NAEP 2019 2020 Summary of Changes Memo

Memorandum United States Department of Education

Institute of Education Sciences

National Center for Education Statistics

Date: May 4, 2018 (revised June 13, 2018)

To: Robert Sivinski, OMB

From: Linda Hamilton, NCES

Through: Kashka Kubzdela, NCES

Re: National Assessment of Educational Progress (NAEP) 2019 and 2020 Changes to Approved Materials (OMB# 1850-0928 v.10)

This request updates the scope, sampling, procedures, and materials to be used in NAEP in 2019, including operational assessments, pilot tests, and special studies from those originally approved in August 2016, with the latest change request approved in March 2018, for NAEP 2017-2019 (OMB# 1850-0928 v.1-9). In addition, this request adds the scope for NAEP 2020. Please note that as in 1991, 1993, and 1995, there will be no operational assessments in 2020 – the year for which the Governing Board has prioritizing piloting certain subjects to be used operationally in future years as part of the transition to DBA.

The library of possible items to be used in the NAEP 2019 and 2020 questionnaires is provided in Appendix F of this submission. The communication and recruitment materials to be used in NAEP 2019 are provided in Appendix D (note that Appendices D1-7 through D1-9 are the 2018 versions). The content of the MyNAEP system used in NAEP 2018 by school coordinators to provide requested administration information is provided in Appendix J.

Some of the assessment, questionnaire, and recruitment materials are translated into Spanish. Specifically, Spanish versions of the student assessments and questionnaires are used when a bilingual accommodation is offered and for all students in Puerto Rico. Typically this is done for all operational grade 4 and 8 assessments (note that a TEL bilingual accommodation has not been offered to date). In years in which a Puerto Rico NAEP assessment is given, such as in 2019, Spanish versions of communication materials for parents, teachers, and school coordinators and teacher and school questionnaires are created by translating their English equivalents into Spanish. In addition, every year, Spanish versions of parent communication materials are used nationwide for Spanish-speaking parents.

Additionally, in early August 2018, the following materials will be provided in the specified Appendices for a 30-day public comment period and OMB review (OMB # 1850-0928 v.11):

• Final NAEP 2019 communication materials in English to be mailed after September 2018 and, where applicable, their Spanish translations (Appendices D1 and D2);

• Final English version of the 2019 MyNAEP system (Appendix J1) and its Spanish translation (Appendix J2);

• Draft content of the HSTS/MSTS version of 2019 MyNAEP in English (note, there will not be a Spanish version) (Appendix J3); and

• Final version of all NAEP 2019 questionnaires in English; final version of the Spanish-translated Student Questionnaires; and draft version of the Spanish-translated Teacher and School Questionnaires (future Appendix K).

The final version of the HSTS/MSTS 2019 MyNAEP system content (Appendix J3) and the final versions of the Spanish-translated Teacher and School Questionnaires (Appendix K) will be submitted to OMB for review as a change request in September 2018 (OMB #1850-0928 v.12). The final versions of 2020 communication materials and data collection instruments will be submitted by October 2019.

Summary of changes to sample sizes for 2019

Change	Rationale
Added sample for the newly added 2019 special studies (CAFS, SES Survey Questionnaire Study, and MSTS).	These are new studies for 2019.
Removed KaSA (US and Puerto Rico).	The implementation of Multi-Stage Testing (MST) has eliminated the need for the KaSA special study.
Removed writing pilot.	The writing pilot has been moved to 2020.
Reduced (a) the operational and pilot reading and (b) science DBA bridge study sample sizes (as such, grade 4 student sample decreased from 427,500 to 375,200; grade 8 from 427,500 to 386,200; and grade 12 from 119,00 to 86,000).	The sample sizes included in the last approved request (OMB# 1850-0928 v.1-9) were based on projections for 2019. As plans for 2019 were finalized, we were able to reduce sample sizes and increase efficiency while still supporting statistical parameters.
Increased the estimated pre-assessment burden for school coordinators from 3 hours to 4.5 hours.	These changes are based on data and feedback from the 2017 and 2018 NAEP assessments.
Updated the estimated percent of schools that would file student lists.
Updated the estimated percent of students with disabilities (SD) and English Language Learners (ELL).

The following below details the changes that were made to each of the approved documents:

Document	Changes
Part A	Removed all references and content specific information pertaining to 2018 Provided additional details for the 2019 special studies previously included: National Indian Education Study (NIES) and High School Transcript Study (HSTS) Added special studies for 2019: Computer Access and Familiarity Study (CAFS), Socioeconomic Status (SES) Survey Questionnaire Study, and Middle School Transcript Study (MSTS) Removed KaSA study and writing pilot Adjusted operational and pilot sample sizes for 2019 Added assessments and special studies to be conducted in 2020 Added confidentiality and regulatory language presented to users on the various log-in screens Added information on Spanish versions of NAEP materials
Part B	Removed screen shots of the MyNAEP system and included this information in new Appendix J Added information about special studies
Part C	Added the 2017 Writing Issues Paper
Appendix A - External Advisory Committees	Updated committee members list to be current Added the NAEP Survey Questionnaire and eNAEP DBA System Translation Review Committee
Appendix B- NAEP 2013 Weighting Procedures	Updated with the 2013 Weighting Procedures, which are the most recently published materials
Appendix C - 2019 Sampling Memo	Replaced sampling memo with 2019 draft sampling memo
Appendix D – 2019 Communications Materials	Updated to include 2019 versions of recruitment and communications materials, with the exception of three materials which are the 2018 version (D1-7, D1-8, and D1-9; see note above regarding future material updates). Appendices D2-17 through D2-25 are new materials.
Appendix E - NAEP 2019 & 2020 Feedback Forms	Updated to include the latest, expanded versions The School Debriefing Interview is now called The School Coordinator Wrap Up Activities High School Students Strategies is now in the content of the MyNAEP system and is included in Appendix J The rating scale system from NAEP 2018 is no longer used
Appendix F - NAEP 2019 & 2020 Questionnaire Items Library	Library of Student, Teacher, and School Administrator questionnaires updated to include the most current content
Appendix G - NAEP 2013 Sampling Design	Updated with the 2013 Sampling Design, which is the most recently published materials
Appendix H - Student Information Instructions	Instructions for entering student information – updated to include the latest versions
Appendix I - High School Transcript Study (HSTS) and Middle School Transcript Study (MSTS) Data Collection Instruments	New appendix Materials for data collection in the 2019 HSTS and MSTS special studies
Appendix J - MyNAEP System Content	New appendix Details on the content of the MyNAEP system, some of which was previously included in Part B

1