Download:
pdf |
pdfNATIONAL CENTER FOR EDUCATION STATISTICS
NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS
NAEP Assessments for 2017-2019
Appendix A-C
Appendix A: External Advisory Committees
Appendix B: Sample Data Security Agreement
Appendix C: 2015 Sampling Memo
OMB# 1850-NEW v.1
(previous OMB# 1850-0790 v.43)
November 23, 2015
Revised February 2016
NATIONAL CENTER FOR EDUCATION STATISTICS
NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS
NAEP Assessments for 2017-2019
Appendix A
External Advisory Committees
OMB# 1850-NEW v.1
(previous OMB# 1850-0790 v.43)
November 23, 2015
Table of Contents
Appendix A-1: NAEP Design and Analysis Committee ........................................................................ 2
Appendix A-2: NAEP Validity Studies Panel ........................................................................................ 2
Appendix A-3: NAEP Quality Assurance Technical Panel .................................................................... 3
Appendix A-4: NAEP National Indian Education Study Technical Review Panel ................................ 3
Appendix A-5: NAEP Civics Standing Committee ................................................................................ 4
Appendix A-6: NAEP Economics Standing Committee ........................................................................ 4
Appendix A-7: Geography Standing Committee .................................................................................... 4
Appendix A-8: NAEP Mathematics Standing Committee ..................................................................... 5
Appendix A-9: NAEP Reading Standing Committee ............................................................................. 5
Appendix A-10: NAEP Science Standing Committee............................................................................ 6
Appendix A-11: NAEP Survey Questionnaires Standing Committee .................................................... 6
Appendix A-12: NAEP Technology and Engineering Literacy Standing Committee ........................... 7
Appendix A-13: NAEP U.S. History Standing Committee .................................................................... 7
Appendix A-14: NAEP Writing Standing Committee............................................................................ 8
Appendix A-15: NAEP Principals’ Panel Standing Committee ............................................................. 8
Appendix A-16: NAEP Mathematics Translation Review Committee .................................................. 9
Appendix A-17: NAEP Science Translation Review Committee .......................................................... 9
NAEP 2017-2019 OMB Clearance: Appendix A
page 1
Appendix A-1: NAEP Design and Analysis Committee
Name
Affiliation
Betsy Becker
Florida State University
Peter Behuniak
University of Connecticut
Johnny Blair
Independent Consultant, Washington, DC
Lloyd Bond
University of North Carolina, Greensboro (Emeritus)/ Carnegie
Foundation (retired)
Derek Briggs
University of Colorado
Kadriye Ercikan
University of British Columbia
Huynh Huynh
University of South Carolina (Emeritus)
Matthew Johnson
Columbia University
Brian Junker
Carnegie Mellon University
David Kaplan
University of Wisconsin-Madison
Kenneth Koedinger
Carnegie Mellon University
Jacqueline Leighton
University of Alberta
Appendix A-2: NAEP Validity Studies Panel
Name
Affiliation
Peter Behuniak
University of Connecticut
George Bohrnstedt
American Institutes for Research, Washington, DC
Jim Chromy
RTI International (Emeritus Fellow), Raleigh, NC
Phil Daro
Strategic Education Research (SERP) Institute, Berkeley, CA
Richard Duran
University of California
David Grissmer
University of Virginia
Larry Hedges
Northwestern University
Gerunda Hughes
Howard University
Ina Mullis
Boston College
Scott Norton
Council of Chief State School Officers, Washington, DC
Jim Pellegrino
University of Chicago/Learning Sciences Research Institute
Gary Phillips
American Institutes for Research, Washington, DC
Lorrie Shepard
University of Colorado at Boulder
David Thissen
The University of North Carolina at Chapel Hill
Karen Wixson
The University of North Carolina at Greensboro
NAEP 2017-2019 OMB Clearance: Appendix A
page 2
Appendix A-3: NAEP Quality Assurance Technical Panel
Name
Affiliation
Jamal Abedi
University of California, Davis
Chuck Cowan
Analytic Focus LLC, San Antonio, TX
Kadriye Ercikan
University of British Columbia
Gail Goldberg
Gail Goldberg Consulting, Ellicott City, MD
Brian Gong
National Center for the Improvement of Educational Assessment,
Dover, NH
James Pellegrino
University of Illinois at Chicago
Mark Reckase
Michigan State University
Michael (Mike) Russell
Boston College
William (Bill) Schafer
Consultant, Mason, OH
Richard Wolfe
University of Toronto (Emeritus), Ontario, Canada
Appendix A-4: NAEP National Indian Education Study Technical Review Panel
Name
Affiliation
Henry Braun
Boston College
Doreen Brown
ASD Education Center, Anchorage, AK
Robert Cook
Native American Initiative/Teach for America, Summerset, SD
Steve Culpepper
University of Illinois at Urbana-Champaign
Susan Faircloth
University of North Carolina Wilmington
Jeremy MacDonald
Rocky Boy Elementary, Box, Elder, MT
Rebecca Izzo-Manymules
Southwest Indian Polytechnic Institute, Albuquerque, NM
Jeannette Muskett Miller
Tohatchi High School, Tohatchi, NM
Debora Norris
Consultant, Phoenix, AZ
CHiXapkaid (Michael Pavel)
University of Oregon
Martin Reinhardt
Northern Michigan University
Tarajean Yazzie-Mintz
Wakanyeja ECE Initiate/American Indian College Fund, Denver, CO
NAEP 2017-2019 OMB Clearance: Appendix A
page 3
Appendix A-5: NAEP Civics Standing Committee
Name
Affiliation
Patricia Avery
University of Minnesota
Christopher Elnicki
Cherry Creek School District, Greenwood Village, CO
Fay Gore
North Carolina Public Schools, Raleigh, NC
Barry Leshinsky
Challenger Middle School, Huntsville, AL
Peter Levine
CIRCLE (Center for Information & Research on Civic Learning
and Engagement), Medford, MA
Clarissa Peterson
Depauw University
Terri Richmond
Golden Valley High School, Bakersville, CA
Jackie Viana
Miami-Dade County Schools, Miami, FL
Appendix A-6: NAEP Economics Standing Committee
Name
Affiliation
Kris Bertelsen
Little Rock Branch-Federal Reserve Bank of St. Louis,
Little Rock, AR
Stephen Buckles
Vanderbilt University
Steven L. Cobb
University of North Texas
Jaime Festa-Daigle
Lake Havasu High School, Lake Havasu City, AZ
Julie Heath
University of Memphis
Richard MacDonald
St. Cloud State University
Andrea Morgan
Oregon Department of Education, Salem, OR
Kevin Smith
Renaissance High School, Detroit, MI
William Walstad
University of Nebraska–Lincoln
Appendix A-7: Geography Standing Committee
Name
Affiliation
Sarah Bednarz
Texas A&M University
Osa Brand
National Council for Geographic Education, Washington, DC
Seth Dixon
Rhode Island College
Charlie Fitzpatrick
ESRI Schools, Arlington, VA
Ruth Luevanos
Pacoima Middle School, Pacoima, CA
Joe Stoltman
Western Michigan University
Kelly Swanson
Johnson Senior High, St. Paul, MN
NAEP 2017-2019 OMB Clearance: Appendix A
page 4
Appendix A-8: NAEP Mathematics Standing Committee
Name
Affiliation
Jennifer Alvarez
Sultana Elementary School, Ontario, CA
Daniel Chazan
University of Maryland, College Park
Carl Cowen
Indiana University–Purdue University
Julie Guthrie
Texas Education Agency
Kathleen Heid
Pennsylvania State University
Mark Howell
Gonzaga College High School, Washington, DC
Russ Keglovits
Nevada Department of Education, Carson City, NV
Carolyn Maher
Rutgers University
Michele Mailhot
Maine Department of Education, Augusta, ME
Brian Nelson
Curtis Corner Middle School, Wakefield, RI
Matthew Owens
Spring Valley High School, Columbia, SC
Carole Philip
Alice Deal Middle School, Washington, DC
Melisa M. Ramos Trinidad
Educación Bilingüe Luis Muñoz Iglesias, Cidra, PR
Ann Trescott
Stella Maris Academy, La Jolla, CA
Appendix A-9: NAEP Reading Standing Committee
Name
Affiliation
Marilyn Adams
Brown University
Peter Afflerbach
University of Maryland
Patricia Alexander
University of Maryland
Margretta Browne
Richard Montgomery High School, Silver Spring, MD
Julie Coiro
University of Rhode Island
Bridget Dalton
University of Colorado Boulder
Valerie Harrison
Claflin University
Karen Malone
Fort Wingate High School, Fort Wingate, NM
Pamela Mason
Harvard Graduate School of Education
Margaret McKeown
University of Pittsburgh
P. David Pearson
University of California, Berkeley
Jenny Thomson
University of Sheffield, Sheffield, UK
Monica Verra-Tirado
Florida Department of Education, Tallahassee, FL
Victoria Young
Texas Education Agency, Austin, TX
Zynia Zepeda
Crane Elementary School District, Yuma, AZ
NAEP 2017-2019 OMB Clearance: Appendix A
page 5
Appendix A-10: NAEP Science Standing Committee
Name
Affiliation
Alicia Cristina Alonzo
Michigan State University
Mary Thandi Buthelezi
Wheaton College
Susan Craft
Hanahan High School, Hanahan, SC
George Deboer
American Association for the Advancement of Science,
Washington, DC
Alex Decaria
Millersville University
Crystal Edwards
Lawrence Township Public Schools, Lawrenceville, NJ
Ibari Igwe
Shrewd Learning, Elkridge, MD
Michele Lombard
Kenmore Middle School, Arlington, VA
Ellen Mingione
Consultant
Brett Moulding
Utah Partnership for Effective Science Teaching and Learning,
Ogden, UT
Amy Pearlmutter
Littlebrook Elementary School, Princeton, NJ
Steve Semken
Arizona State University
Gerald Wheeler
National Science Teacher Association, Arlington, VA
David White
Lamoille North Supervisory Union School District, Hyde Park, VT
Appendix A-11: NAEP Survey Questionnaires Standing Committee
Name
Affiliation
Angela Duckworth
University of Pennsylvania
Hunter Gehlbach
Harvard University
Gerunda Hughes
Howard University
David Kaplan
University of Wisconsin-Madison
Henry Levin
Teachers College, Columbia University
Stanley Presser
University of Maryland
Leslie Rutkowski
Indiana University Bloomington
Rob Santos
Urban Institute, Washington, DC
Norbert Schwarz
University of Michigan
Jonathon Stout
Lock Haven University
Roger Tourangeau
Westat, Rockville, MD
Akane Zusho
Fordham University
NAEP 2017-2019 OMB Clearance: Appendix A
page 6
Appendix A-12: NAEP Technology and Engineering Literacy Standing Committee
Name
Affiliation
Keith Barton
Indiana University Bloomington
John Behrens
Pearson eLEADS Center, Mishawaka, IN
Brooke Bourdelat-Parks
Biological Sciences Curriculum Study (BSCS),
Colorado Springs, CO
Barbara Bratzel
Shady Hill School, Cambridge, MA
Lewis Chappelear
James Monroe High School, North Hills, CA
Britte Haugan Cheng
SRI International, Menlo Park, CA
Meredith Davis
North Carolina State University
Chris Dede
Harvard Graduate School of Education
Richard Duran
University of California, Santa Barbara
Maurice Frazier
Oscar Smith High School, Chesapeake, VA
Camilla Gagliolo
Arlington Public Schools, Arlington, VA
Christopher Hoadley
New York University
Eric Klopfer
Massachusetts Institute of Technology
Beth McGrath
Stevens Institute of Technology
Greg Pearson
National Academy of Engineering, Washington, DC
John Poggio
University of Kansas
Erin Reilly
University of Southern California
Troy Sadler
Missouri University Science Education Center,
Columbia, MO
Kimberly Scott
Arizona State University
Teh-Yuan Wan
New York State Education Department, Albany, NY
Appendix A-13: NAEP U.S. History Standing Committee
Name
Affiliation
Keith Barton
Indiana University Bloomington
Michael Bunitsky
Frederick County Public Schools, Frederick, MD
Teresa Herrera
Shenandoah Middle School, Miami, FL
Cosby Hunt
Center for Inspired Teaching, Washington, DC
Helen Ligh
Macy Intermediate School, Monterey, CA
Amanda Prichard
Green Mountain High School, Lakewood, CO
Kim Rasmussen
Auburn Washburn Unified School District, Topeka, KS
Diana Turk
New York University
NAEP 2017-2019 OMB Clearance: Appendix A
page 7
Appendix A-14: NAEP Writing Standing Committee
Name
Affiliation
Diane August
Center for Applied Linguistics, Washington, DC
Margretta Browne
Montgomery County Public Schools, Silver Spring, MD
Robert Crongeyer
Robla School, Sacramento, CA
Elyse Eidman-Aadahl
National Writing Project, Berkeley, CA
Nikki Elliot-Schuman
Smarter Balanced Assessment Consortium
Rayna Goldfarb
Abraham Lincoln High School, Philadelphia, PA
Charles MacArthur
University of Delaware
Michael McCloskey
Johns Hopkins University
Norma Mota-Altman
San Gabriel High School, Alhambra, CA
Sandra Murphy
University of California, Davis
Drew Sterner
Tamanend Middle School, Warrington, PA
Sherry Swain
National Writing Project, Berkeley, CA
Victoria Young
Texas Education Agency, Austin, TX
Appendix A-15: NAEP Principals’ Panel Standing Committee
Name
Affiliation
David Atherton
Clear Creek Middle School, Gresham, OR
Ardith Bates
Gladden Middle School, Chatsworth, GA
Williams Carozza
Harold Martin Elementary School, Hopkinton, NH
Diane Cooper
St. Joseph’s Academy, Clayton, MO
Brenda Creel
Alta Vista Elementary School, Cheyenne, WY
Rita Graves
Pin Oak Middle School, Bellaire, TX
Don Hoover
Lincoln Junior High School, Springdale, AR
Stephen Jackson
(Formerly with) Paul Laurence Dunbar High School, Washington,
DC
Anthony Lockhart
Lake Shore Middle School, Belle Glade, FL
Susan Martin
Berrendo Middle School, Roswell, NM
Lillie McMillan
Porter Elementary School, San Diego, CA
Jason Mix
Howard Lake–Waverly–Winsted High School, Howard Lake, MN
NAEP 2017-2019 OMB Clearance: Appendix A
page 8
Appendix A-16: NAEP Mathematics Translation Review Committee
Name
Affiliation
Gilberto Cuevas
Texas State University, San Marcos
Néstor Díaz
Coral Gables Senior High School, Coral Gables, FL
David Feliciano
P.S.M.S 29, The Melrose School, Bronx, NY
Yvonne Fuentes
Author and Spanish Linguist, Carrollton, GA
Flor Yanira Gurrola Valenzuela
Washington Middle School, Albuquerque, NM
Melisa M. Ramos Trinidad
Educación Bilingüe Luis Muñoz Iglesias, Cidra, PR
Sonia Suazo
Escuela Salvador Brau Elemental, Cayey, PR
Enid Valle
Kalamazoo College
Appendix A-17: NAEP Science Translation Review Committee
Name
Affiliation
Néstor Díaz
Coral Gables Senior High School, Coral Gables, FL
Yvonne Fuentes
Author and Spanish Linguist, Carrollton, GA
Myrna Rasmussen
Austin Independent School District, Austin, TX
Enid Valle
Kalamazoo College
NAEP 2017-2019 OMB Clearance: Appendix A
page 9
NATIONAL CENTER FOR EDUCATION STATISTICS
NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS
Appendix B
Sample Data Security Agreement
Request for Clearance for
NAEP Assessments for 2017-2019
OMB# 1850-NEW v.1
(previous OMB# 1850-0790 v.43)
NAEP 2017-2019 OMB Clearance: Appendix B
UNITED STATES DEPARTMENT OF EDUCATION
INSTITUTE OF EDUCATION SCIENCES
NATIONAL CENTER FOR EDUCATION STATISTICS
Data Security Agreement for the
2015 National Assessment of Educational Progress (NAEP)
This document establishes a data security agreement between the ___________________ State
Department of Education, and the National Center for Education Statistics of the U.S. Department of
Education for the 2015 NAEP assessment program.
NAEP is a congressionally mandated project of the U.S. Department of Education. P.L 107-279, Title III,
directs the Commissioner for Education Statistics to conduct a National Assessment of Educational
Progress. The law requires the NCES Commissioner for Education Statistics to conduct a national and
state assessment in mathematics and reading in grades four and eight at least once every 2 years. At grade
12, NAEP assesses mathematics and reading at regularly scheduled intervals. Additional national
assessments in grades 4, 8, and 12 take place at regularly scheduled intervals in other subjects such as
writing, science, history, geography, civics, economics, foreign languages, and arts. This agreement
applies to data collection activities under the NAEP program including operational, and pilot and special
studies.
P.L. 107-110, as amended by P.L. 107-279 authorizes NAEP to include, “whenever feasible, information
collected, cross-tabulated, compared, and reported by race, ethnicity, socioeconomic status, gender,
disability, and limited English proficiency.” To fulfill this statutory requirement, in addition to cognitive
questions, NAEP administers background questionnaires that provide information for reporting categories
and that collect non-cognitive data on students, their family background, teachers, and schools.
NCES understands that any improper disclosure or unauthorized use of these materials may violate
Federal statutes, including but not limited to the Family Educational Rights and Privacy Act (FERPA) (20
U.S.C. 1232g) as well as applicable state statutes.
By accepting this agreement, NCES acknowledges that student records and related information constitute
confidential materials and commits to protect and safeguard these data according to NAEP data security
procedures, as applicable, described in NCES Data Confidentiality Procedures Summary (Attachment
A) and incorporated herein.
ACCEPTED AND AGREED TO:
_______________________________________
State authorized agent
Date
____________________________________
NCES authorized agent
Date
_______________________________________
Title
_____________________________________
Title
_______________________________________
Address
_____________________________________
Address
______________________________________________
____________________________________________
NAEP 2017-2019 OMB Clearance: Appendix B
NATIONAL CENTER FOR EDUCATION STATISTICS
NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS
Appendix C
201 Sampling Memo
Request for Clearance for
NAEP Assessments for 2017-2019
OMB# 1850-NEW v.1
(previous OMB# 1850-0790 v.43)
NAEP 2017-2019 OMB Clearance: Appendix C
Date:
February 22, 2016
Memo: 20171.1A/1.1B/1.1D/1.1E/1.1L
To:
William Ward, NCES
Ed Kulick, ETS
David Freund, ETS
Chris Averett
Kavemuii Murangi
Jennifer Kali
Amy Dresher, ETS
David Ferraro
Cathy White, Pearson
Scott Ferguson, Fulcrum
Dianne Walsh
Lauren Byrne
Lisa Rodriguez
Rick Rogers
Rob Dymowski
William Wall
Erin Wiley
Dwight Brock
Amy Lin
David Hubble
Sarah Shore
Yiting Dai
Jing Kang
Sabrina Zhang
From:
Reviewer:
Lloyd Hicks and Keith Rust
John Burke
Subject:
Sample Design for 2017 NAEP - DRAFT
I.
Introduction
For 2017, the NAEP assessment involves the following components:
A.
National assessments in reading, mathematics, and writing at grades 4, and 8;
B.
State-by-state and Trial Urban District Assessment (TUDA) assessments in reading and
mathematics for public schools at grades 4 and 8;
C.
An assessment of mathematics in Puerto Rico at grades 4 and 8;
\\westat.com\dfs\NAEPLIB\2017\Memos\School
Sampling\Public Grades 4 & 8-A\2017-m01v01a.docx
NAEP 2017-2019
OMB Clearance: Appendix C
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
D.
Digital-based assessment (DBA) start-ups, and pilot tests, in US history, civics, and
geography at grade 8;
E.
Pilot assessments in reading and mathematics at grades 4 and 8;
F.
A special study of Multi-Stage Testing (MST) in mathematics at grades 4 and 8;
G.
A special equating study of Knowledge and Skills Appropriate (KaSA) items in
mathematics, at grades 4 and 8, both nationally and in Puerto Rico.
H.
A special study of writing, using laptop computers rather than tablets, at grade 8.
I.
A Computer Access and Familiarity Study at grades 4 and 8. The study will involve a
relatively small subsample of public schools and students selected for the reading and
math operational assessment components described above.
Below is a summary list of the features of the 2017 sample design.
1.
The alpha samples for grades 4 and 8 public, and the delta samples for private schools
at grades 4 and 8, will be used for the operational assessments in reading and
mathematics.
2.
The beta samples at grades 4 and 8 public, and the epsilon samples at grades 4 and 8
private, will be used for the national writing assessment, all start-ups and pilot tests, and
the MST special study. The beta samples will also be used for the DBA portion of the
national KaSA study.
3.
The lambda (‘L is for ‘laptop’) sample at grade 8 public will be used for the writing
laptop study.
4.
As in recent NAEP studies, Trial Urban District Assessment (TUDA) samples will form
part of the corresponding state samples, and the state samples will form part of the
national sample. There are twenty-seven Trial Urban District Assessment (TUDA)
participants. Twenty-one of the twenty-seven also participated in 2013. The six new
districts are: Denver, CO; Clark Co., NV; Guilford Co., NC; Shelby Co., TN; Fort
Worth, TX; Milwaukee, WI.
5.
Schools in the alpha and delta samples will receive a mixture of DBA assessments, using
tablets, and pencil and paper (PBA) assessments. Schools in the beta and epsilon
samples will be assessed using DBA with tablets. Schools in the lambda sample will be
assessed DBA using laptops. In the alpha samples, the proportion of students assigned
to DBA versus PBA will vary depending upon whether the school is in a large TUDA
district, a small TUDA district, or not in a TUDA districts. Schools in the TUDA
districts will have a larger proportion of students assigned to PBA than other alpha
sample schools. In public schools the PBA assessment will include the KaSA study, in
addition to operational reading and math.
6.
There will be no samples in territories other than for Puerto Rico at grades 4 and 8.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 2 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
7.
As in 2015, the Department of Defense Schools are expected to be reported as a single
jurisdiction (DoDEA).
8.
There are no samples at grade 12.
9.
There is no National Indian Education Study. This means that less extensive sampling
of BIE schools is required than in 2015 and other years when NIES has been
conducted. To ensure sound results for AIAN students in reading and mathematics at
the national level, at grades 4 and 8 BIE students will be sampled at the same rate as
students in Oklahoma, the state with the largest AIAN population.
10.
Oversampling of private schools at grades 4 and 8 will be done at the same level as
2015. Response rates permitting, this will allow separate reporting for reading and
mathematics for Catholic and non-Catholic schools, but no further breakdowns by
private school type.
11.
The sample sizes of assessed students for these various components are shown in Table
1 (which also shows the approximate numbers of participating schools).
NAEP 2017-2019 OMB Clearance: Appendix C
Page 3 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
Table 1.
February 22, 2016
Target sample sizes of assessed students, and expected number of participating
schools, for 2017 NAEP
Spiral
Jurisdictions
States (incl.
Spiral Indic. DC, DoDEA)
Grade 4
Nat’l/state reading (DBA)
Nat’l/state math (DBA)
Nat’l/state reading (PBA)
Nat’l/state math (PBA)
math KaSA (PBA)
Puerto Rico (DBA)
Puerto Rico (PBA)
Total - alpha
Total- delta
Typical max. no. students/school
Average assessed students/school
Total schools - alpha, delta
Writing
Mathematics MST
Math Pilot
Reading pilot
KaSA (DBA)
Total - beta
Total - epsilon
Typical max. no. students/school
Average assessed students/school
Total schools - beta, epsilon
DS
DS
PA, PB
PA, PB
PA
DP
PP
4
2
52
52
52
52
1
1
Urban
districts
27
27
27
27
Students
Private
Public school
school
students
students
141,000
141,000
37,500
37,500
3,000
3,000
3,000
366,000
62
49
7,500
DA, DB
DA, DB
DA, DB
DA, DB
DA
1
1
19,800
9,000
13,500
10,800
3,000
56,100
50
43
1,300
Total number of students grade 4
Total number of schools grade 4
NAEP 2017-2019 OMB Clearance: Appendix C
422,100
8,800
Page 4 of 22
Total
3,000 144,000
3,000 144,000
500 38,000
500 38,000
0
3,000
0
3,000
0
3,000
366,000
7,000
7,000
60
23
47
300
7,800
2,200
1,000
1,500
1,200
0
5,900
50
20
300
22,000
10,000
15,000
12,000
3,000
56,100
5,900
39
1,600
12,900 435,000
600
9,400
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
Table 1.
February 22, 2016
Target sample sizes of assessed students, and expected number of participating
schools, for 2017 NAEP (Continued)
Spiral
Jurisdictions
States (incl.
Spiral Indic. DC, DoDEA)
Grade 8
Nat’l/state reading (DBA)
Nat’l/state math (DBA)
Nat’l/state reading (PBA)
Nat’l/state math (PBA)
math KaSA (PBA)
Puerto Rico (DBA)
Puerto Rico (PBA)
Total - alpha
Total- delta
Typical max. no. students/school
Average assessed students/school
Total schools - alpha, delta
DS
DS
PA, PB
PA, PB
PA
DP
PP
4
2
52
52
52
52
1
1
Urban
districts
27
27
27
27
Students
Private
Public school
school
students
students
141,000
141,000
37,500
37,500
3,000
3,000
3,000
366,000
62
50
7,330
Writing
Mathematics MST
Math Pilot
Reading pilot
KaSA (DBA)
U.S. History Start-up/Pilot
Civics Start-up/Pilot
Geography Start-up/Pilot
Total – beta
Total – epsilon
Typical max. no. students/school
Average assessed students/school
Total schools - beta, epsilon
DA, DB
DA, DB
DA, DB
DA, DB
DA
DA, DB
DA, DB
DA, DB
1
1
Writing laptop study
Total- lambda
Typical max. no. students/school
Average assessed students/school
Total schools - lambda
LW
1
19,800
9,000
13,500
10,800
3,000
8,100
6,300
6,300
76,800
Total
3,000 144,000
3,000 144,000
500 38,000
500 38,000
0
3,000
0
3,000
0
3,000
366,000
7,000
7,000
60
26
49
270
7,600
2,200
1,000
1,500
1,200
0
900
700
700
50
43
1,800
8,200
50
23
350
3,000
3,000
30
25
120
0
0
0
0
0
22,000
10,000
15,000
12,000
3,000
9,000
7,000
7,000
76,800
8,200
40
2,150
3,000
3,000
30
25
120
Total number of students grade 8
Total number of schools grade 8
445,800
9,250
15,200 461,000
620
9,870
GRAND TOTAL STUDENTS
GRAND TOTAL SCHOOLS
867,900
18,050
28,100 896,000
1,220 19,270
NAEP 2017-2019 OMB Clearance: Appendix C
Page 5 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
II.
February 22, 2016
Assessment Types
The assessment spiral types are shown in Table 2. Seven different spirals will be used at grade 4 and
eight at grade 8. Session IDs contain six characters, traditionally. The first two characters identify the
assessment “type” (subjects and type of spiral in a general way). Grade is contained in the second
pair of characters, and the session sequential number (within schools) in the last two characters. For
example, session DS0401 denotes the first grade 4 reading and mathematics operational DBA
assessment in a given school.
Table 2.
ID
NAEP 2017 assessment types and IDs
Type
Subjects
Grades
Schools
Comments
All schools in the alpha (except
Puerto Rico) and delta samples,
except a few small schools
All alpha sample schools, except
Puerto Rico and a few small
schools
All delta sample schools, except
a few small schools
DS
Operational
DBA
Reading, math (1:1)
4, 8
Public,
Private
PA
Operational
PBA
Reading, math, KaSA
(10:10:1)
4, 8
Public
Reading, math (1:1)
4, 8
Private
4, 8
Public
All beta sample schools
4, 8
Private
All epsilon sample schools
KaSA Mathematics
4, 8
Public
Puerto Rico only
KaSA Mathematics
4, 8
Public
Puerto Rico only
Writing
8
Public
All lambda sample schools
PB
DA
DB
DP
PP
LW
III.
Operational
PBA
Operational,
start-up, and
pilot DBA
Operational,
start-up, and
pilot DBA
Operational
DBA
Operational
PBA
Laptop study
Writing, reading, math,
math MST, math KaSA, US
history, civics, geography
Writing, reading, math,
math MST, US history,
civics, geography
Sample Types and Sizes
In similar fashion to past years (but somewhat different), we will identify five different types of
school samples: Alpha, Beta, Delta, Epsilon, and Lambda. These distinguish sets of schools that will
be conducting distinct portions of the assessment.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 6 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
1.
February 22, 2016
Alpha Samples at Grades 4 and 8
These are public school samples for grades 4 and 8. They will be used for the operational state-bystate assessments in reading and mathematics, and contribute to the national samples for these
subjects as well. They will also be used for the national KaSA special study (PBA mode). There will
be alpha samples for each state, DC, DoDEA, BIE, and Puerto Rico.
The details of the target student sample sizes for the alpha samples are as follows:
A.
At each grade, the target student sample size is 6,400: roughly 2,700 assessed each for math,
and reading, and 50 for KaSA PBA. Of these 2,700 per subject, 2,200 will be DBA and 500
will be PBA. Thus the goal in each state (before considering the contribution of TUDA
districts) is to assess 4,400 students in DBA and 1,050 PBA. The DS session type will be used
for DBA and the PA session type for PBA.
B.
There will be samples for twenty-seven TUDA districts. For the six large TUDA districts
(New York, Los Angeles, Chicago, Miami-Dade, Clark Co., and Houston) the assessed
student target sample sizes are three-quarters the size of a state sample for DBA (3,300) and
the same size as a state sample for PBA (1,050), for a total of 4,350. Therefore the target
sample size is 5,100.
C.
For the remaining 21 TUDA districts the assessed student target sample sizes are half the size
of a state sample for DBA (2,200) and the same size as a state sample for PBA (1,050), for a
total of 3,250. Therefore the target sample size is 3,800.
D.
Note that, above, there is a conflict between sample size requirements at the state level, and
the TUDA district level. This will be resolved as in previous years: the districts will have the
target samples indicated in B and C, and reflected in Table 3. For the states that contain one
or more of these districts, the target sample size indicated in A (and shown in Table 3) will be
used to determine a school sampling rate for the state, which will be applied to the balance of
the state outside the TUDA district(s). Thus the target student sample sizes, shown in Table 3,
for states that contain a TUDA district, are only ‘design targets’, and are smaller than the final
total sample size for the state, but larger than the sample for the balance of the state, exclusive
of its TUDA districts. In the case of the District of Columbia, the state sample size
requirement is that all schools and students be included. This renders moot any requirements
for the DC TUDA sample, which by default consists of all schools operated by the DCPS
district (but excludes charter schools in DC, even though those are all included in the state
sample, as these are not operated by DCPS).
E.
In Puerto Rico, the target sample size is 7,000 per grade (grades 4 and 8), with the goal of
assessing 3,000 students each for DBA and PBA.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 7 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
As in past state-by-state assessments, schools with fewer than 20 students in the grade in question
will be sampled at a moderately lower rate than other schools (at least half, and often higher,
depending upon the size of the school). This is in implicit recognition of the greater cost and burden
associated with surveying these schools.
Table 3 shows the target student sample sizes, and the approximate counts of schools to be selected
in the alpha samples, along with the school and student frame counts, by state and TUDA districts
for grades 4 and 8. The table also identifies the jurisdictions where we take all schools and where we
take all students.
Table 4 consolidates the target student (and resulting school) sample size numbers, to show the total
target sample sizes in each state, combining the TUDA targets with those for the balance of the
state.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 8 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
Table 3.
February 22, 2016
Grade 4 and 8 school and student frame counts, expected school sample sizes, and initial target student sample sizes for
the 2017 state-by-state and TUDA district assessments (Alpha samples)
Grade 4
Jurisdiction
Alabama
Alaska
Arizona
Arkansas
Bureau of Indian Education
California
Colorado
Connecticut
Delaware
District of Columbia
DoDEA Schools
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Maryland
Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Schools in
frame
709
361
1,193
480
137
5,977
1,054
602
119
119
110
2,226
1,248
205
381
2,205
1,050
638
704
721
760
320
903
958
1,711
956
423
1,166
NAEP 2017-2019 OMB Clearance: Appendix C
Schools in
sample
113
188
116
115
15
111
118
115
96
119
110
109
105
113
124
119
113
126
131
114
116
153
112
114
118
120
111
126
Students
in frame
57,548
9,990
86,472
36,937
3,357
472,139
67,814
39,544
10,393
5,536
7,554
212,541
133,243
15,494
22,864
149,235
78,837
37,147
37,202
52,221
55,735
13,444
67,399
70,968
111,240
65,262
38,316
69,574
Grade 8
Overall
target
student
sample
size
6,400
6,400
6,400
6,400
399
6,400
6,400
6,400
6,400
5,536
7,554
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
Page 9 of 22
**
**
Schools in
frame
456
280
793
303
113
2,901
567
339
61
69
65
1,220
562
83
209
1,561
489
368
393
417
488
202
373
485
1,083
712
287
709
Schools in
sample
109
141
115
108
13
110
113
111
51
69
65
110
105
61
100
117
106
114
120
113
112
114
107
106
115
122
105
121
Students in
frame
55,820
9,651
83,469
36,503
2,936
455,002
65,088
40,679
10,105
4,520
5,636
202,256
129,475
13,314
22,319
151,830
79,653
35,691
36,033
50,755
51,981
13,473
61,983
71,662
114,211
63,732
36,486
67,833
Overall
target
student
sample size
6,400
6,400
6,400
6,400
366
6,400
6,400
6,400
6,400
4,520
5,636
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
**
**
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
Table 3.
February 22, 2016
Grade 4 and 8 school and student frame counts, expected school sample sizes, and initial target student sample sizes for
the 2017 state-by-state and TUDA district assessments (Alpha samples) (Continued)
Grade 4
Jurisdiction
Montana
Nebraska
Nevada
New Hampshire
New Jersey
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Puerto Rico
Rhode Island
South Carolina
South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
West Virginia
Wisconsin
Wyoming
Schools in
frame
392
532
394
270
1,368
444
2,471
1,457
261
1,740
869
747
1,607
931
164
644
312
995
4,433
623
216
1,109
1,231
417
1,099
192
NAEP 2017-2019 OMB Clearance: Appendix C
Schools in
sample
183
149
110
136
115
125
111
111
178
115
130
124
110
211
112
109
170
113
109
109
216
108
115
142
127
145
Grade 8
Students
in frame
11,534
23,315
35,875
13,734
99,727
26,208
201,226
118,118
8,471
129,087
50,988
43,816
130,442
31,308
10,777
58,089
10,517
77,202
399,614
50,112
6,204
97,550
81,904
20,578
61,686
7,639
Overall
target
student
sample
size
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
7,000
6,400
6,400
6,400
6,400
6,400
6,400
6,204
6,400
6,400
6,400
6,400
6,400
Page 10 of 22
**
Schools in
frame
271
294
171
142
762
232
1,498
728
184
1,093
583
429
888
398
60
307
246
584
2,256
258
121
379
609
190
649
89
Schools in
sample
142
117
90
90
109
108
108
108
153
111
123
118
106
155
60
105
143
110
109
106
121
104
113
106
117
89
Students in
frame
10,811
22,561
34,346
14,078
99,169
25,079
196,197
117,176
7,789
131,562
48,784
43,051
131,525
30,211
10,720
54,828
9,657
73,441
384,210
47,422
5,999
95,187
79,084
20,464
61,152
7,042
Overall
target
student
sample size
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
6,400
7,000
6,400
6,400
6,400
6,400
6,400
6,400
5,999
6,400
6,400
6,400
6,400
7,042
*
**
**
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
Table 3.
February 22, 2016
Grade 4 and 8 school and student frame counts, expected school sample sizes, and initial target student sample sizes for
the 2017 state-by-state and TUDA district assessments (Alpha samples) (Continued)
Grade 4
Jurisdiction
Albuquerque
Atlanta
Austin
Baltimore City
Boston
Charlotte
Chicago
Clark County, NV
Cleveland
Dallas
Denver
Detroit
Duval County, FL
Fresno
Fort Worth
Guilford County, NC
Hillsborough County, FL
Houston
Jefferson County, KY
Los Angeles
Miami
Milwaukee
New York City
Philadelphia
San Diego
Shelby County, TN
District of Columbia PS
Schools in
frame
95
55
81
128
72
105
433
226
71
151
102
65
119
72
85
74
176
175
100
566
286
111
788
148
144
120
76
Schools in
sample
58
55
56
75
72
52
95
79
71
54
60
65
57
55
54
57
55
78
55
81
83
76
83
60
61
60
76
Grade 8
Students
in frame
7,412
4,285
6,878
6,716
4,086
11,696
27,360
25,311
2,754
13,325
7,108
3,889
10,313
5,905
7,073
5,492
16,522
18,049
7,718
50,210
26,711
5,668
73,248
11,227
10,245
9,250
3,584
Overall
target
student
sample
size
3,800
4,285
3,800
3,800
4,086
3,800
5,100
5,100
2,754
3,800
3,800
3,889
3,800
3,800
3,800
3,800
3,800
5,100
3,800
5,100
5,100
3,800
5,100
3,800
3,800
3,800
3,584
Counts for states do not reflect the oversampling for their constituent TUDA districts.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 11 of 22
**
**
**
**
**
Schools in
frame
40
23
25
96
43
46
434
80
70
42
60
49
50
23
32
29
87
62
43
204
178
83
524
112
68
61
32
Schools in
sample
40
23
25
69
43
35
94
58
70
42
48
49
36
23
32
29
49
48
31
77
78
64
82
57
44
43
32
Students in
frame
6,691
3,554
5,463
5,504
3,667
11,007
27,895
24,676
2,685
10,878
6,060
2,963
8,873
5,237
5,977
5,339
15,096
13,383
7,306
43,901
26,978
4,977
66,513
8,849
9,398
8,277
2,394
Overall
target
student
sample size
3,800
3,554
3,800
3,800
3,667
3,800
5,100
5,100
2,685
3,800
3,800
2,963
3,800
3,800
3,800
3,800
3,800
5,100
3,800
5,100
5,100
3,800
5,100
3,800
3,800
3,800
2,394
*
**
*
**
**
*
**
*
*
*
**
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
Target student sample sizes reflect sample sizes prior to attrition due to exclusion, ineligibility, and nonresponse.
* identifies jurisdictions where all schools (but not all students) for the given grade are included in the NAEP sample.
** identifies jurisdictions where all students for the given grade are included in the NAEP sample.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 12 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
Table 4.
February 22, 2016
Total sample sizes, combining state and TUDA samples
Grade 4
Jurisdiction
Alabama
Alaska
Arizona
Arkansas
Bureau Of Indian Education
California
Colorado
Connecticut
Delaware
District Of Columbia
DoDEA Schools
Florida
Georgia
Hawaii
Idaho
Illinois
Indiana
Iowa
Kansas
Kentucky
Louisiana
Maine
Maryland
Massachusetts
Michigan
Minnesota
Mississippi
Missouri
Montana
Schools
in frame
709
361
1,193
480
137
5,977
1,054
602
119
119
110
2,226
1,248
205
381
2,205
1,050
638
704
721
760
320
903
958
1,711
956
423
1,166
392
NAEP 2017-2019 OMB Clearance: Appendix C
Schools
in sample
113
188
116
115
15
293
165
115
96
119
110
276
156
113
124
191
113
126
131
153
116
153
173
178
179
120
111
126
183
Students
in frame
57,548
9,990
86,472
36,937
3,357
472,139
67,814
39,544
10,393
5,536
7,554
212,541
133,243
15,494
22,864
149,235
78,837
37,147
37,202
52,221
55,735
13,444
67,399
70,968
111,240
65,262
38,316
69,574
11,534
Grade 8
Overall
target
student
sample
size
6,400
6,400
6,400
6,400
399
18,198
9,526
6,400
6,400
5,536
7,554
17,486
10,479
6,400
6,400
10,323
6,400
6,400
6,400
9,256
6,400
6,400
9,563
10,118
10,065
6,400
6,400
6,400
6,400
Page 13 of 22
**
**
Schools in
frame
456
280
793
303
113
2,901
567
339
61
69
65
1,220
562
83
209
1,561
489
368
393
417
488
202
373
485
1,083
712
287
709
271
Schools in
sample
109
141
115
108
13
240
151
111
51
69
65
245
125
61
100
189
106
114
120
128
112
114
166
143
161
122
105
120
142
Students in
frame
55,820
9,651
83,469
36,503
2,936
455,002
65,088
40,679
10,105
4,520
5,636
202,256
129,475
13,314
22,319
151,830
79,653
35,691
36,033
50,755
51,981
13,473
61,983
71,662
114,211
63,732
36,486
67,833
10,811
Overall
target
student
sample size
6,400
6,400
6,400
6,400
366
18,274
9,603
6,400
6,400
4,520
5,636
17,486
9,778
6,400
6,400
10,320
6,400
6,400
6,400
9,279
6,400
6,400
9,632
9,740
9,197
6,400
6,400
6,400
6,400
**
**
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
Table 4.
February 22, 2016
Total sample sizes, combining state and TUDA samples (Continued)
Grade 4
Jurisdiction
Nebraska
Nevada
New Hampshire
New Jersey
New Mexico
New York
North Carolina
North Dakota
Ohio
Oklahoma
Oregon
Pennsylvania
Puerto Rico
Rhode Island
South Carolina
South Dakota
Tennessee
Texas
Utah
Vermont
Virginia
Washington
West Virginia
Wisconsin
Wyoming
Total
Schools
in frame
532
394
270
1,368
444
2,471
1,457
261
1,740
869
747
1,607
931
164
644
312
995
4,433
623
216
1,109
1,231
417
1,099
192
52,354
Schools
in sample
149
114
136
115
151
154
204
178
183
130
123
161
211
112
109
170
160
339
109
216
108
115
142
190
145
8,089
Grade 8
Overall
target
student
sample
size
6,400
6,970
6,400
6,400
8,373
9,169
13,068
6,400
9,017
6,400
6,400
9,649
7,000
6,400
6,400
6,400
9,432
22,173
6,400
6,204
6,400
6,400
6,400
9,609
6,400
427,568
Students
in frame
23,315
35,875
13,734
99,727
26,208
201,226
118,118
8,471
129,087
50,988
43,816
130,442
31,308
10,777
58,089
10,517
77,202
399,614
50,112
6,204
97,550
81,904
20,578
61,686
7,639
3,833,727
**
Schools in
frame
294
171
142
762
232
1,498
728
184
1,093
583
429
888
398
60
307
246
584
2,256
258
121
379
609
190
649
89
29,009
Sample sizes for each state do reflect the samples in the TUDA districts within the state.
* identifies jurisdictions where all schools (but not all students) for the given grade are included in the NAEP sample.
** identifies jurisdictions where all students for the given grade are included in the NAEP sample.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 14 of 22
Schools in
sample
117
90
90
109
122
153
157
153
177
123
118
155
155
60
105
143
141
247
106
121
104
113
106
171
89
6,770
Students in
frame
22,561
34,346
14,078
99,169
25,079
196,197
117,176
7,789
131,562
48,784
43,051
131,525
30,211
10,720
54,828
9,657
73,441
384,210
47,422
5,999
95,187
79,084
20,464
61,152
7,042
3,733,641
Overall
target
student
sample size
6,400
6,894
6,400
6,400
8,483
9,329
13,106
6,400
8,954
6,400
6,400
9,769
7,000
6,400
6,400
6,400
9,478
22,304
6,400
5,999
6,400
6,400
6,400
9,678
7,042
423,869
*
**
**
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
Stratification
Each state and grade will be stratified separately, but using a common approach in all cases. TUDA
districts will be separated from the balance of their state, and each part stratified separately. The first
level of stratification will be based on urban-centered type of location. This variable has 12 levels
(some of which may not be present in a given state or TUDA district), and these will be collapsed so
that each of the resulting location categories contains at least 9 percent of the student population (12
percent for large TUDA districts and 18 percent for small TUDA districts). Within each of the
resulting location categories, schools will be assigned a minority enrollment status. This is based on
the two race/ethnic groups that are the second and third most prevalent within the location
category. If these groups are both low in percentage terms, no minority classification will be used.
Otherwise three (or occasionally four) equal-sized groups (generally high, medium, and low
minority) will be formed based on the distribution across schools of the two minority groups.
Within the resulting location and minority group classes (of which there are likely to be from three
to fifteen, depending upon the jurisdiction), schools will be sorted by a measure derived from school
level results from the most recent available state achievement tests at the relevant grade. In general,
mathematics test results will be used, but where these are not available, reading results will be used.
In the few states that do not have math or reading tests at grades 4 and 8 (or where we are unable to
match the results to the NAEP school frame), instead of achievement data, schools will be sorted
using a measure of socio-economic status. This is the median household income of the 5-digit ZIP
Code area where the school is located, based on the 2014 ACS (5-year) data. For BIE and DoDEA
schools neither achievement data nor income data are available, and so grade enrollment is used in
these cases.
Once the schools are sorted by location class, minority enrollment class, and achievement data (or
household income), a systematic sample of schools will be selected using a random start. Schools
will be sampled with probability proportional to size. The exact details of this process are described
in the individual sampling specification memos.
2.
Beta Sample
The beta sample comprises the national public school samples at grades 4 and 8. This sample will be
used to conduct the DBA assessments of operational writing, and various pilots, start-ups, and
special studies as shown in Table 1. Each of these samples will be nationally representative, selected
NAEP 2017-2019 OMB Clearance: Appendix C
Page 15 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
to have minimal overlap with the alpha sample schools at the same grade. The number of students
targeted per school will be 50.
Stratification
The Beta samples will have an implicit stratification, using a hierarchy of stratifiers and a serpentine
sort. The highest level of the hierarchy is Census division (9 implicit strata). The next stratifier in the
hierarchy is type of location, which has twelve categories. Many of the type of location strata nested
within Census divisions will be collapsed with neighboring type of location cells (this will occur if
the expected school sample size within the cell is less than 4.0). These geographic strata will be
subdivided using a dichotomous high minority status category for oversampling purposes. Schools
with more than 10 Black or Hispanic students and greater than 15 percent Black or Hispanic
students will be considered high minority and placed in an oversampling stratum. All other schools
will be considered low minority and placed in a regular sampling stratum. If the expected sample size
for a high or low minority stratum is less than 8.0, it will be left as is. If the expected sample size is
greater than 8.0, then the stratum will be subdivided into up to four substrata (two for expected
sample size up to 12.0, three for expected sample size up to 16.0, and four for expected sample size
greater than 16.0). For the regular sampling strata, the subdivision will be by state or groups of
contiguous states. For the oversampling strata, the subdivision will be by percentage Black and
Hispanic. Within these substrata, the schools are to be sorted by school type (public, BIE, DoDEA)
and median household income from the 2014 5-year ACS (using a serpentine sort within the school
type substrata).
Schools with more than 15 percent black or Hispanic students will be sampled at twice the rate of
other schools, so as to increase the student sample sizes for these two groups.
3.
Delta Samples
These are the private school samples at grades 4 and 8 for conducting the operational assessments in
reading and mathematics. The sample sizes are large enough to report results by Catholic and nonCatholic at grades 4 and 8. Approximately half the sample at each grade will be from Catholic
schools. The number of students targeted per school will be 60 at each grade.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 16 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
Stratification
The private schools are to be explicitly stratified by private school type (Catholic/Other). Within
each private school type, stratification will be by Census region (4 categories), type of location (12
categories), race/ethnicity composition, and enrollment size. In general, where there are few or no
schools in a given stratum, categories will be collapsed together, always preserving the private school
type.
4.
Epsilon Sample
This sample is analogous to the beta sample, but for private schools, at grades 4 and 8. The same
stratification variables will be used as for the delta samples. The epsilon sample schools will have
minimum overlap with the delta sample schools which, given the respective sample sizes, means that
no schools will be selected for both the delta and epsilon samples at the same grade.
5.
Lambda Sample
This will be a sample of grade 8 public schools, and consist of only about 120 schools. The sample
will be selected from a sample of approximately 35 geographic Primary Sampling Units (PSUs). This
PSU component is needed because of the operational complexities of administering the laptopbased assessment. A select group of staff will be trained to administer those assessments, at a
somewhat later time than the rest of the assessments. The PSUs will be selected so as to minimize
overlap with the PSU samples used in recent NAEP assessments. The school stratification of the
lambda sample within PSUs will be by type of location and median household income. The number
of students targeted per school will be 30.
IV.
New Schools
To compensate for the fact that files used to create the NAEP school sampling frames are at least
two years out of date at the time of frame construction, we will supplement the Alpha, Beta, Delta,
and Epsilon samples with new school samples at each grade.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 17 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
The new school samples will be drawn using a two-stage design. At the first stage, a minimum of ten
school districts (in states with at least ten districts) will be selected from each state for public
schools, and ten Catholic dioceses will be selected nationally for the private schools. The sampled
districts and dioceses will be asked to review lists of their respective schools and identify new
schools. Frames of new schools will be constructed from these updates, and new schools will be
drawn with probability proportional to size using the same sample rates as their corresponding
original school samples.
The school sample sizes in the above tables do not reflect new school samples.
V.
Substitute Samples
Substitute samples will be selected for each of the Beta, Delta, Epsilon and Lambda samples. The
substitute school for each original will be the next “available” school on the sorted sampling frame,
with the following exceptions:
A.
Schools selected for any NAEP samples will not be used as substitutes.
B.
Private schools whose school affiliation is unknown will not be used as substitutes. Also,
unknown affiliated private schools in the original samples will not get substitutes.
C.
A school can be a substitute for one and only one sample. (If a school is selected as a
substitute school for grade 8, for example, it cannot be used as a substitute for grade 4.)
D.
A public school substitute will always be in the same state as its original school.
E.
A catholic school substitute will always be a Catholic school, and the same for non-Catholic
schools.
VI.
Contingency Samples
The districts that are taking part in the TUDA program are volunteers. Thus it is possible that at
some point over the next few months, a given district might choose to opt out of the TUDA
program for 2017. However, it is not acceptable for all schools in such a district to decline NAEP,
as then the state estimates will be adversely affected. Thus to deal with this possibility, in each
TUDA district, subsamples of the alpha sample schools will be identified as contingency samples. In
the event that the district withdraws from the TUDA program prior to the selection of the student
NAEP 2017-2019 OMB Clearance: Appendix C
Page 18 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
sample, all alpha sampled schools from that district will be dropped from the sample, with the
exception of those selected in the contingency sample. The contingency sample will provide a
proportional representation of the district, within the aggregate state sample. Student sampling in
those schools will then proceed in the same way as for the other schools within the same state.
VII.
Student Sampling
Students within the sampled schools will be selected with equal probability, except in low minority
schools in the Beta samples where oversampling of Black and Hispanic students will take place. The
student sampling parameters vary by sample type (Alpha, Beta, Delta, Epsilon and Lambda) and
grade, as described below.
Alpha Sample, Grades 4 and 8 Schools (Except Puerto Rico)
A.
The sample size for each school will depend upon whether the school is in a Large TUDA
district, a Small TUDA district, or not involved in TUDA.
B.
In schools not involved in TUDA all students will be selected, up to 70. If the school has
more than 70 students, 62 will be selected. Of these students, 50 will be assigned to DBA and
the rest to PBA. In schools with fewer than 21 students, all will be assigned to DBA or all to
PBA. In schools with 32 to 37 students, 25 will be assigned to DBA and the rest to PBA. In
all other schools (with a single hit, see below), 25/31 of the students will be assigned to DBA
with the rest to PBA. In some schools, the school may be assigned more than one ‘hit’ in
sampling. In these schools we will select a sample of size 62 times the number of hits, taking
all students if this target is greater than or equal to 62/70 of the total enrollment.
C.
In schools from Large TUDA districts (New York City, Los Angeles, Chicago, Miami-Dade,
Clark Co., and Houston), all students, up to 74, will be selected. If the school has more than
74 students, 66 will be selected. In schools where 66 to 74 students are selected, 50 students
will be assigned to DBA and the rest to PBA. In schools with fewer than 21 students, all will
be assigned to DBA or all to PBA. In schools with 34 to 39 students, 25 will be assigned to
DBA and the rest to PBA. In all other schools (with a single hit, see below), 25/33 of the
students will be assigned to DBA with the rest to PBA. In some schools, the school may be
assigned more than one ‘hit’ in sampling. In these schools we will select a sample of size 66
times the number of hits, taking all students if this target is greater than or equal to 66/74 of
the total enrollment.
D.
In schools from Small TUDA districts all students, up to 80, will be selected. If the school has
more than 80 students, 74 will be selected. In schools where 74 to 80 students are selected, 50
students will be assigned to DBA and the rest to PBA. In schools with fewer than 21 students,
all will be assigned to DBA or all to PBA. In schools with 38 to 44 students, 25 will be
assigned to DBA and the rest to PBA. In all other schools (with a single hit, see below), 25/37
NAEP 2017-2019 OMB Clearance: Appendix C
Page 19 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
of the students will be assigned to DBA with the rest to PBA. In some schools, the school
may be assigned more than one ‘hit’ in sampling. In these schools we will select a sample of
size 74 times the number of hits, taking all students if this target is greater than or equal to
74/80 of the total enrollment.
Alpha Sample, Puerto Rico Grades 4 and 8
A.
All students, up to 55, will be selected.
B.
If the school has more than 55 students, a systematic sample of 50 students will be selected
with no oversampling.
C.
25 students will be assigned to DBA and the rest to PBA.
For schools with between 20 and 49 students, half of the students will be assigned to DBA
and half to PBA.
D.
Schools with fewer than 20 students in the selected grade will be assigned to have either all
students assessed with DBA, or all with PBA.
E.
Delta Samples, Grades 4 and 8
A.
All students, up to 70, will be selected.
B.
If the school has more than 70 students, a systematic sample of 60 students will be selected.
In schools with 60 students or more, 50 students will be assigned to DBA, and the remainder
to PBA. In schools with fewer than 18 students, all will be assigned to DBA or all to PBA. In
schools with 31 to 35 students, 25 will be assigned to DBA and the rest to PBA. In all other
schools 5/6 of the students will be assigned to DBA with the rest to PBA.
Beta and Epsilon Samples, Grades 4 and 8
A.
In each school, a sample will be selected as follows: All students up to 50 will be selected. If
there are more than 50 students enrolled, a sample of 50 students will be selected.
Lambda Sample, Grade 8
A.
In each school, a sample will be selected as follows: All students up to 30 will be selected. If
there are more than 30 students enrolled, a sample of 30 students will be selected.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 20 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
VIII.
February 22, 2016
Weighting Requirements
The Operational Reading and Mathematics Samples
The exact weighting requirements for these samples have yet to be determined. One possibility is
that three sets of weights will be required – for DBA alone, PBA alone, and DBA/PBA combined.
These weights will reflect probabilities of selection, school and student nonresponse, any trimming,
and the random assignment to the particular subject. There will be a separate replication schemes by
grade and public/private. Such weights will also be derived for the Puerto Rico KaSA assessment
and the national KaSA special studies at grades 4 and 8.
The Operational Writing Assessment
The sample weights will reflect probabilities of selection, school and student nonresponse, any
trimming, and the random assignment to the particular subject (necessary because the writing
assessment is spiraled in with other assessment components).
Start-up/Pilot Assessments in US History, Civics, and Geography, at Grade 8
Weighting will be implemented in full for these assessments. The sample weights will reflect
probabilities of selection, school and student nonresponse, any trimming, and the random
assignment to the particular subject (necessary because these assessments are spiraled in with other
assessment components).
Math MST Special Study
Weighting will be implemented in full for these assessments. The sample weights will reflect
probabilities of selection, school and student nonresponse, any trimming, and the random
assignment to the particular subject (necessary because these assessments are spiraled in with other
assessment components).
NAEP 2017-2019 OMB Clearance: Appendix C
Page 21 of 22
Memorandum: 2017 - 1.1A/1.1B/1.1D/1.1E/1.1L
February 22, 2016
Pilot Tests for Reading and Mathematics
As is standard practice, only preliminary weights will be provided for these assessments. The sample
weights will reflect probabilities of selection, and the random assignment to the particular subject
(necessary because these assessments are spiraled in with other assessment components).
Writing Laptop Special Study at Grade 8
Weighting will be implemented in full for this assessment. The sample weights will reflect
probabilities of selection, school and student nonresponse, and any trimming.
NAEP 2017-2019 OMB Clearance: Appendix C
Page 22 of 22
| File Type | application/pdf |
| File Title | Appendix A (Statute Authorizing NAEP) |
| Author | joconnell |
| File Modified | 2016-06-20 |
| File Created | 2016-02-17 |