Appendix G NAEP 2013 Sampling Design

NATIONAL CENTER FOR EDUCATION STATISTICS

NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS

National Assessment of Educational Progress (NAEP)

2022 Materials Update #1

Appendix G

NAEP 2013 Sample Design

OMB# 1850-0928 v.23

June 2021

No changes since November 2020

(1850-0928 v.21)

The 2013 Sample Design documentation is the most current version available to the public. At this time, there is not a timeline for when the details for later assessment years will be publicly available.

NAEP Technical Documentation Website

NAEP Technical Documentation NAEP 2013 Sample Design

The sample design for NAEP 2013 included samples for various operational, special study, and pilot test assessments.

Representative samples were drawn for the following operational assessments:

national assessments in mathematics

2013 State Assessment Sample Design

2013 National Assessment Sample Design

and reading in public and private schools at grades 4, 8, and 12;

state-by-state and Trial Urban District Assessments (TUDA) assessments in mathematics and reading in public schools at grades 4 and 8; and

state-by-state assessments in mathematics and reading in public schools at grade 12 in 13 states.

Representative samples were drawn for the following special studies and pilot test assessments:

pilot test of the computer-based assessment of Technical and Engineering Literacy (TEL) in public schools at grade 8;

a special mathematics assessment in public and private schools in Puerto Rico at grades 4 and 8;

Accessible Booklet Study in reading in public and private schools at grades 4 and 8;

study to examine the link between Lexile and NAEP reading in public and private schools at grade 8;

study to obtain NAEP grade 12 mathematics scores for students in the National High School Longitudinal Study (HSLS) in public schools;

study to examine the relationship between NAEP grade 8 and grade 12 mathematics scales, conducted in public schools at grades 9, 10, and 11 in two states that conducted PISA assessments in 2012; and

pilot tests in reading and mathematics in public and private schools at grades 4 and 8.

The samples for the operational assessments were organized into four distinct components and sampled separately. The samples for the special studies and pilot tests were integrated into these various components:

mathematics and reading assessments in public schools at grades 4 and 8; mathematics and reading assessments in public schools at grade 12; mathematics and reading assessments in private schools at grades 4, 8, and 12; and

computer-based TEL pilot assessment in public schools at grade 8.

The national assessments were designed to achieve nationally representative samples of public and private school students in the fourth, eighth, and twelfth grades. Their

target populations included all students in public, private, Bureau of Indian Education (BIE), and Department of Defense Education Activity (DoDEA) schools, who were enrolled in grades 4, 8, and 12 at the time of assessment.

For the fourth- and eighth-grade mathematics and reading assessments in public schools, the NAEP state student samples and assessments constituted the NAEP national student samples and assessments. Nationally representative samples were drawn for the remaining populations of private school students, DoDEA students, and BIE students in the fourth and eighth grades.

The TUDA samples formed part of the corresponding state public school samples, and the state samples formed the public school grades 4 and 8 part of the national sample.

At grade 12, the national samples for mathematics and reading consisted of 13 state samples of public schools and additional samples of public, private, BIE, and DoDEA schools to represent the balance of the nation.

All samples except the TEL pilot sample were based on a two-stage sample design: selection of schools within strata; and

selection of students within schools.

The computer-based TEL pilot sample was based on a three-stage sample design: selection of primary sampling units (PSUs);

selection of schools within strata; and selection of students within schools.

In the three-stage design for the TEL pilot sample, schools were stratified and selected within the sampled PSUs. The sample of schools was selected with probability proportional to a measure of size based on the estimated grade 8 student enrollment.

The state assessments were designed to achieve representative samples of students in the respective grade. At grades 4 and 8, the target populations included all students in each participating jurisdiction, which included states, District of Columbia, DoDEA, and school districts chosen for the TUDA assessments. At grade 12, the target population consisted of all students in each of the 13 participating states. Each sample was designed to produce aggregate estimates with reliable precision for all the participating jurisdictions, as well as estimates for various student subpopulations of interest.

In the PISA linking study, samples of students in grades 9 through 11 were selected from the schools selected for the grade 12 public school samples in Florida and Massachusetts.

The figure below illustrates the various sample types and subjects.

Components of the NAEP samples, by assessment subject, grade, and school type: 2013

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Assessments.

NAEP Technical Documentation Website

NAEP Technical Documentation Sample Design for the 2013 State Assessment

Each assessment cycle, a sample of students in designated grades within both public and private schools throughout the United States is selected for assessment. In state assessment years, of which 2013 is an example, the samples of public schools and their students in each state are large enough to support state-level estimates.

The NAEP 2013 state assessments covered fourth- and eighth-grade students in public schools for operational mathematics and reading. A representative sample of students was drawn in each participating jurisdiction, including the 50 states, the District of Columbia, Puerto Rico, Bureau of Indian Education (BIE) schools, Department of Defense Education Activity (DoDEA) schools, and in school districts chosen for the Trial Urban District Assessment (TUDA) study. The state operational mathematics and reading assessments also

Target Population Sampling Frame Stratification of Schools School Sample Selection Substitute Schools Ineligible Schools Student Sample Selection

School and Student Participation

covered twelfth-grade students in public schools in 13 states for each subject. A representative sample of public school students was drawn in each participating jurisdiction.

All jurisdictions, including the TUDA districts, were included in the mathematics and reading assessments at grades 4 and 8, with the exception of Puerto Rico, where a special mathematics assessment was conducted instead of the operational mathematics and reading assessments. Also, BIE was not designed as a reportable jurisdiction for the 2013 state assessments, but a nationally representative sample of students in BIE schools was selected.

Generally for the state assessments, each non-TUDA jurisdiction sample is designed to produce aggregate estimates with approximately equal precision for all the participating jurisdictions, as well as estimates for various subpopulations of interest. The target sample size for these jurisdictions is 3,150 for each operational subject. In 2013, the samples for operational mathematics and reading at grades 4 and 8 were designed in this fashion. At grades 4 and 8, the overall target student sample size for the operational samples in each non-TUDA jurisdiction was 6,600—3,150 each for mathematics and reading and 300 for pilot tests. For the mathematics assessment in Puerto Rico, the target sample size was 5,800 at grades 4 and 8. At grade 12, the target sample sizes varied by jurisdiction. Details can be found in the school sample selection.

The target population for the NAEP 2013 state assessment included students in public schools who were enrolled in grades 4, 8, and 12 at the time of assessment. The sampling frame included public schools having the relevant grade in each jurisdiction. The samples were selected based on a two-stage sample design:

selection of schools within participating jurisdictions; and

selection of students within schools.

From the stratified frame of public schools for each grade within each jurisdiction, a systematic random sample of grade-eligible schools was drawn with probability proportional to a measure of size based on the estimated grade-specific enrollment of the school.

For the TUDA study, schools were sampled from the 21 participating TUDA districts at the same time schools were selected for the jurisdiction samples. The TUDA districts are listed below:

Albuquerque Public Schools, New Mexico; Atlanta Public Schools, Georgia;

Austin Independent School District, Texas; Baltimore City Public Schools, Maryland; Boston Public Schools, Massachusetts; Charlotte-Mecklenburg Schools, North Carolina; Chicago Public Schools, Illinois;

Cleveland Metropolitan School District, Ohio; Dallas Independent School District, Texas; Detroit Public Schools, Michigan;

District of Columbia Public Schools, District of Columbia; Fresno Unified School District, California;

Hillsborough County Public Schools, Florida; Houston Independent School District, Texas;

Jefferson County Public Schools (Louisville), Kentucky; Los Angeles Unified School District, California;

Miami-Dade County Public Schools, Florida; Milwaukee Public Schools, Wisconsin;

New York City Department of Education, New York; School District of Philadelphia, Pennsylvania; and San Diego Unified School District, California.

These subsamples affected the design of the state samples in those states where TUDA districts were oversampled. In each of these states, there were distinct sampling rates for each TUDA district and for the balance of the state (i.e., the rest of the state not in a TUDA district).

Each selected school provided a list of eligible enrolled students from which a systematic sample of students was drawn. In fourth- and eighth-grade schools, 63 students, if possible, were selected from each school: 30 for mathematics, 30 for reading, and 3 for the pilot tests. In twelfth-grade schools, 60 students, if possible, were selected from each school: 30 for mathematics and 30 for reading. Details can be found in the student sample selection.

NAEP Technical Documentation Target Population for the 2013 State Assessment

The target population for the 2013 state assessment included all students in public schools in the United States who were enrolled in fourth or eighth grade and, for 13 states, students enrolled in twelfth grade. In addition, students enrolled in fourth and eighth grades attending Bureau of Indian Education (BIE) schools, Department of Defense Education Activity (DoDEA) schools, and public schools in Puerto Rico were included. BIE was not designed as a reportable jurisdiction for the 2013 state assessments, but a nationally representative sample of students in BIE schools was selected.

NAEP Technical

Documentation Sampling Frame for the 2013 State Assessment

Drawing the school samples for the 2013 assessments required a comprehensive list of public schools in each jurisdiction containing information for stratification purposes. As in previous NAEP assessments,

the Common Core of Data (CCD) file developed by NCES was used to construct the sampling frame. The CCD file corresponding to the 2009-2010 school year provided the frame for all regular and state-operated public, Bureau of Indian Education (BIE), Department of Defense Education Activity (DoDEA) schools, and schools in Puerto Rico.

Fourth- and Eighth-Grade Schools and Enrollment in Public School Sampling Frame

Twelfth-Grade Schools and Enrollment Public School Sampling Frame in 13 States

New-School Sampling Frame

The sampling frame excluded ungraded schools, vocational schools with no enrollment, special education-only schools, prison and hospital schools, virtual or online schools, home- school entities, and juvenile correctional institutions.

For quality control purposes, school and student counts from the NAEP 2013 sampling frame were compared to school and student counts from the previous frame (2012). No revisions to the frame were needed as a result of this check.

NAEP Technical Documentation Website

NAEP Technical Documentation Fourth- and Eighth-Grade Schools and Enrollment in the 2013 Public School Sampling Frame

The following table displays, by jurisdiction, the number of fourth- and eighth-grade public schools and their estimated enrollment, as contained in the Common Core of Data (CCD) sampling frame. Grade-specific enrollment was estimated for each school as the average grade enrollment for grades 1 through 8.

Number of schools and enrollment in public school sampling frame, state assessment, by grade and jurisdiction: 2013

Jurisdiction	Grade 4		Grade 8
Schools	Enrollment	Schools	Enrollment
Total	52,652	3,755,038	28,515	3,664,355
Alabama	748	59,269	484	57,283
Alaska	365	9,827	285	9,701
Arizona	1,179	83,555	769	81,283
Arkansas	496	37,017	315	35,831
California–Fresno	70	5,823	26	5,457
California–Los Angeles	516	51,516	162	46,818
California–San Diego	138	10,040	63	9,691
California–Balance	5,122	396,594	2,567	394,159
Colorado	1,013	63,266	520	59,357
Connecticut	604	41,489	306	42,667
Delaware	115	9,647	60	9,398
Florida–Hillsborough County	164	15,161	80	15,020
Florida–Miami	274	26,903	160	25,573
Florida–Balance	1,659	160,638	910	156,199
Georgia–Atlanta	61	4,264	26	3,369
Georgia–Balance	1,181	125,981	522	120,938
Hawaii	202	14,155	80	12,843
Idaho	368	21,351	200	20,466
Illinois–Chicago	488	29,942	470	29,469
Illinois–Balance	1,855	124,553	1,133	124,839
Indiana	1,101	80,245	485	79,856
Iowa	674	35,421	387	34,991
Kansas	729	35,907	412	34,645
Kentucky–Jefferson County	96	7,617	42	7,030
Kentucky–Balance	635	44,068	349	42,373
Louisiana	789	55,300	531	50,584
Maine	341	13,945	211	14,205
Maryland–Baltimore	124	6,292	93	5,501
Maryland–Balance	763	55,578	263	55,486
Massachusetts–Boston	79	4,106	38	3,808
Massachusetts–Balance	894	67,192	444	68,154
Michigan–Detroit	121	7,584	74	5,001

Michigan–Balance	1,735	110,658	968	115,376
Minnesota	952	61,086	695	61,246
Mississippi	436	38,958	290	36,999
Missouri	1,170	68,189	727	68,007
Montana	400	10,863	289	10,912
Nebraska	568	22,085	339	21,503
Nevada	379	33,851	158	33,028
New Hampshire	265	14,495	137	15,191
New Jersey	1,366	100,453	741	99,535
New Mexico–Albuquerque	99	7,594	42	6,903
New Mexico–Balance	332	18,137	161	17,280
New York–New York City	709	63,731	458	61,278
New York–Balance	1,659	128,564	877	131,957
North Carolina–Charlotte	105	11,245	38	9,852
North Carolina–Balance	1,309	106,884	656	101,491
North Dakota	260	6,995	188	7,330
Ohio–Cleveland	83	3,573	81	3,550
Ohio–Balance	1,794	129,785	1,009	129,136
Oklahoma	895	49,300	593	46,433
Oregon	767	42,827	414	42,949
Pennsylvania–Philadelphia	177	12,098	142	10,970
Pennsylvania–Balance	1,565	117,452	772	121,829
Rhode Island	170	10,437	57	10,842
South Carolina	618	55,228	298	52,433
South Dakota	328	9,380	254	9,306
Tennessee	998	75,934	565	71,570
Texas–Austin	80	6,862	24	5,360
Texas–Dallas	147	12,932	38	10,113
Texas–Houston	179	16,525	64	12,738
Texas–Balance	3,871	337,085	2,039	320,696
Utah	582	46,508	232	42,593
Vermont	224	6,419	122	6,364
Virginia	1,137	93,610	388	92,179
Washington	1,214	77,826	606	77,099
West Virginia	425	20,875	202	20,637
Wisconsin–Milwaukee	115	5,807	89	5,371
Wisconsin–Balance	996	55,190	544	55,692
Wyoming	188	6,849	93	6,568
Other jurisdictions
Bureau of Indian Education (BIE)	135	3,246	109	2,785
Department of Defense Education Activity (DoDEA)	108	7,507	63	5,589
District of Columbia (TUDA)	87	3,369	37	2,357
District of Columbia–Balance	44	1,538	42	1,950
Puerto Rico	1,017	38,842	407	37,363
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation Twelfth-Grade Schools and Enrollment in the 2013 Public School Sampling Frame in 13 States

The following table presents the number of schools and estimated enrollment for the twelfth-grade CCD frame for the 13 state assessments.

Number of schools and enrollment in public school sampling frame, grade 12 state assessment, by jurisdiction: 2013

Jurisdiction	Schools	Enrollment
Total 1	5,710	859,758
Arkansas	297	32,035
Connecticut	245	41,607
Florida	965	176,821
Idaho	210	19,057
Illinois	954	149,998
Iowa	408	37,793
Massachusetts	371	67,923
Michigan	1,032	126,382
New Hampshire	89	15,749
New Jersey	432	97,690
South Dakota	191	8,796
Tennessee	369	67,111
West Virginia	147	18,796
1 The aggregate of the 13 states participating in the state assessments at grade 12. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation New-School Sampling Frame for the 2013 State Assessment

The Common Core of Data (CCD) file used for the frame corresponds to the 2009-2010 school year, whereas the assessment year is the 2012-2013 school year. During this 3-year period, some

schools closed, some changed structure (one school becoming two schools, for example), and others came into existence.

As was done in previous years, to achieve as close to full coverage as possible, the school frame was supplemented by a sample of new schools obtained from a sample of districts. Each sampled district was sent a list of the CCD schools and asked to add in any new schools or old schools that had become newly eligible for grades 4, 8, and 12.

Since asking every school district to list new and newly-eligible schools would have generated too much of a burden, a sample of districts was contacted to obtain a list of new schools. To represent the unsampled districts in the full sample of schools, weights for schools included in the new-school sample were adjusted to reflect the district selection probability.

The goal was to allow every new school a chance of selection, thereby fully covering the target population of schools in operation during the 2012-2013 school year. The first step in this process was the development of a new-school frame through the construction of a district-level file from the CCD school-level file. To develop the frame, the district-level file was divided into two files: one

for small districts and a second for medium and large districts.

Small districts contained no more than three schools on the frame in total, with no more than one school at each targeted grade (4, 8, and 12). New schools in small districts were identified during school recruitment and added to the sample if the old school was sampled. From a sampling perspective, the new school was viewed as an “annex” to the sampled school that had a well-defined probability of selection equal to that of the old school. The “frame” in this case was, in fact, the original frame; when the old school was sampled in a small district, the new school was automatically sampled as well.

The remaining districts were defined as medium and large districts. In these districts, a frame of new schools was developed based on information provided by the district. To limit the required effort, the new-school frame was created through developing information on a sample of medium and large public school districts in each jurisdiction.

Prior to district sampling, specific districts were in sample with certainty. They included the following districts:

districts in jurisdictions where all schools were selected for sample at any of grades 4, 8 or 12; state-operated districts;

districts in states with fewer than 10 districts;

charter-only districts (that is, districts containing no schools other than charter schools); and

TUDA districts.

The remaining districts in each jurisdiction (except the certainty jurisdictions) were separated into two strata of large- and medium-size districts. These strata were defined by computing an aggregate percentage of enrollment for each district within the state (removing districts in the certainty strata defined above) and sorting in descending order by percentage of jurisdiction enrollment represented by the district. All districts up to and including the first district at or above the 80th cumulative percentage were defined as large districts. The remaining districts were defined as medium districts.

An example is given below. A state's districts are ordered by percentage enrollment. The first six become large districts and the last six become medium districts.

Large and medium districts example, state assessment, by enrollment, stratum, and district: 2013

District	Percentage enrollment	Cumulative percentage enrollment	Stratum
1	20	20	L
2	20	40	L
3	15	55	L
4	10	65	L
5	10	75	L
6	10	85	L
7	5	90	M
8	2	92	M
9	2	94	M
10	2	96	M
11	2	98	M
12	2	100	M
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

The target sample size for each jurisdiction was 10 districts. Where possible, eight large and two medium districts were selected. However, in the example above, since there are only six large districts, all of the large districts and four of the medium districts were selected for the new-school inquiry.

If sampling was needed in the medium stratum, the medium districts were selected with equal probability. If sampling was needed in the large stratum, the large districts were sampled with probability proportional to enrollment. These probabilities were retained and used in later stages of sampling and weighting, as the district probability then represented the number of other districts that were not sampled to be surveyed for new schools.

The selected districts in each jurisdiction were then sent a listing of all their schools that appeared on the 2009-2010 CCD file and were asked to provide information about the new schools not included in the file and grade span changes of existing schools. These listings provided by the selected districts were used as sampling frames for selection of new public schools and updates of existing schools.

This process was conducted through the NAEP State Coordinator in each jurisdiction. The coordinators were sent the information for all sampled districts in their respective states and were responsible for returning the completed updates.

The eligibility of a school was determined based on the grade span. A school also was classified as “newly-eligible” if a change of grade span had occurred such that the school status changed from ineligible to eligible in a particular grade.

The purpose of school stratification is to increase the efficiency and ensure the representativeness of the school samples in terms of important school-level characteristics, such as geography (e.g., states and TUDA districts), urbanicity, and race/ethnicity classification. NAEP school sampling utilizes two types of stratification: explicit and implicit.

Stratification Variables

Explicit stratification partitions the sampling frame into mutually exclusive groupings called strata. The systematic samples selected from these strata are independent, meaning that each is selected with its own unique random start. The explicit school strata for the 2013 NAEP state assessments were usually states. If a state contained Trial Urban District Assessment (TUDA) districts, the explicit strata were each individual TUDA district and the balance of the state. In 2013, there were 21 participating TUDA districts in the NAEP state assessment program. They are listed below:

Albuquerque Public Schools, New Mexico; Atlanta Public Schools, Georgia;

Austin Independent School District, Texas; Baltimore City Public Schools, Maryland; Boston Public Schools, Massachusetts; Charlotte-Mecklenburg Schools, North Carolina; Chicago Public Schools, Illinois;

Cleveland Metropolitan School District, Ohio; Dallas Independent School District, Texas; Detroit Public Schools, Michigan;

District of Columbia Public Schools, District of Columbia; Fresno Unified School District, California;

Hillsborough County Public Schools, Florida; Houston Independent School District, Texas;

Jefferson County Public Schools (Louisville), Kentucky; Los Angeles Unified School District, California;

Miami-Dade County Public Schools, Florida; Milwaukee Public Schools, Wisconsin;

New York City Department of Education, New York; School District of Philadelphia, Pennsylvania; and San Diego Unified School District, California.

Implicit stratification involves sorting the sampling frame, as opposed to grouping the frame. For NAEP, schools are sorted by key school characteristics within explicit strata and sampled systematically using this ordering. This type of stratification ensures the representativeness of the school samples with respect to the key school characteristics. The implicit school stratification variables for the 2013 state assessments included urbanicity, race/ethnicity classification, and achievement score/median income. Further details about these variables can be found here.

The implicit stratification of public schools for the NAEP 2013 state assessments involved three dimensions:

urbanicity classification (urban-centric locale); race/ethnicity classification; and

achievement level or median income.

The urbanicity stratum is the top-level implicit stratification variable and is assigned within each explicit stratum. It is derived from the NCES urban-centric locale variable and classifies schools based on location (city, suburb, town, rural) and proximity to urbanized areas.

It has 12 possible values.

Stratification by Urbanicity Classification

Stratification by Race/ethnicity Classification

Stratification by Achievement Data and Median Income

Missing Stratification Variables

The race/ethnicity stratum classifies schools by the relative magnitude of enrollment of non- Hispanic White, non-Hispanic Black, Hispanic, Asian, American Indian/Alaska Native, Hawaiian/Pacific Islander, and students classified as two or more races represented in schools. The source of the race/ethnicity data is the Common Core of Data (CCD). The race/ethnicity stratum is the second-level variable in the stratification hierarchy and is nested within the urbanicity stratum.

The last stratification dimension is a classification of schools based on either achievement data or median household income. For most states, it is based on achievement data. However, not all states provide achievement data. In these cases, median household income is used instead.

Median income comes from the 2000 Census and it corresponds to the zip code area where the school is located.

Missing values for stratification variables were imputed.

The implicit stratification in this three-fold hierarchical procedure was achieved via a "serpentine sort" within a given explicit stratum. This sort was accomplished by alternating between ascending and descending sort order on each variable successively through the sort hierarchy.

Within this sorted list the schools were arranged in serpentine order by achievement data (or median household income) within each cell determined by the two higher stratification variables (urbanicity and race/ethnicity classifications), with ascending order for achievement data/median household income used in every other cell, and descending order for achievement data/median household income used in the remaining cells, giving an ascending-descending-ascending- descending pattern. Schools in these urbanicity and race/ethnicity classification cells were also sorted in serpentine order. Within each urbanicity and race/ethnicity classification cells, schools were sorted in ascending order within one urbanicity stratum, by descending order within the next urbanicity stratum, and so on. The following table shows an oversimplified example to illustrate the ascending-descending-ascending-descending pattern of the serpentine sort.

Stratification variables sorted by serpentine sort: 2013

TUDA	Urbanicity	Race/ethnicity level	Achievement score
Yes	Large City	High minority	20
			22
		27
		30
	Low minority	29
		26
		20
		18
	Mid-size City	Low minority	15
			25
27
31
High minority	35
	32
	30
	28
No	Mid-size City	High minority	20
			22
			27
			30
		Low minority	29
			26
	20
	18
	Large City	Low minority	15
			25
			27
31
High minority		35
		32
	30
	28
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation Stratification by Urbanization Classification for the 2013 State Assessment

The creation of the urbanicity classification variable was based on the NCES urban-centric locale and was defined within each explicit stratum. The NCES urban-centric locale contains the following categories:

1. Large City: Territory inside an urbanized area and inside a principal city with population of 250,000 or more;

2. Mid-size City: Territory inside an urbanized area and inside a principal city with population less than 250,000 and greater than or equal to 100,000;

3. Small City: Territory inside an urbanized area and inside a principal city with population less than 100,000;

4. Large Suburb: Territory outside a principal city and inside an urbanized area with population of 250,000 or more;

5. Mid-size Suburb: Territory outside a principal city and inside an urbanized area with population less than 250,000 and greater than or equal to 100,000;

6. Small Suburb: Territory outside a principal city and inside an urbanized area with population less than 100,000;

7. Fringe Town: Territory inside an urban cluster that is less than or equal to 10 miles from an urbanized area;

8. Distant Town: Territory inside an urban cluster that is more than 10 miles and less than or equal to 35 miles from an urbanized area.

9. Remote Town: Territory inside an urban cluster that is more than 35 miles of an urbanized area;

10. Fringe Rural: Census-defined rural territory that is less than or equal to 5 miles from an urbanized area, as well as rural territory that is less than or equal to 2.5 miles from an urban cluster;

11. Distant Rural: Census-defined rural territory that is more than 5 miles but less than or equal to 25

miles from an urbanized area, as well as rural territory that is more than 2.5 miles but less than or equal to 10 miles from an urban cluster;

12. Remote Rural: Census-defined rural territory that is more than 25 miles from an urbanized area and is also more than 10 miles from an urban cluster; and

13. Outside of the United States: Department of Defense Education Activity (DoDEA) overseas schools or Puerto Rico.

For the definitions of the geographic terms used in these descriptions, please refer to the Census Bureau’s website (for example, www.census.gov/population/www/metroareas/aboutmetro.html)

The urbanicity classification cells were created by starting with the original NCES urban-centric locale categories. Urbanicity strata were collapsed with neighboring strata until a minimum cell size criterion, in terms of the percentage of students, was met. The minimum cell size criterion varied by type of explicit stratum. The

criterion for explicit strata comprising the largest TUDA districts (Los Angeles, New York City, Chicago, Miami, and Houston) was 12 percent; for the other TUDA districts, it was 18 percent; and for all other explicit strata, it was 9 percent.

The urbanicity classification variable was equal to the original NCES urban-centric locale if no collapsing was necessary. If collapsing was necessary, the collapsing scheme first collapsed within the four major strata (city, suburbs, town, rural). For example, urbanicity categories 1, 2, and 3 within city were collapsed (1 with 2, 2 with

3) if cells 1 or 3 were deficient. If the middle cell (e.g., 2) was deficient, then it was collapsed with the smaller of the two end cells. If a collapsed pair was still deficient, it was collapsed with the remaining unit within the major stratum. That is, a single city cell would be created by collapsing the large city, mid-size city, and small city cells. If a cell was still deficient after collapsing within major stratum, further collapsing across major strata occurred as needed until the deficiency was resolved. The values of the urbanicity classification variable were set equal to

the cell value of the final level of collapsing.

Prior experience with this type of stratification has shown that the greatest efficiency of stratification results when cities and suburb fringe areas are always kept separate from towns and rural areas, even if the enrollment criterion is violated.

NAEP Technical Documentation Website

NAEP Technical Documentation Stratification by Race/Ethnicity Classification for the 2013 State Assessment

Race/ethnicity classification was based on the second and third largest race/ethnicity percentages (among non- Hispanic White, non-Hispanic Black, Hispanic, Asian, American Indian/Alaska Native, Hawaiian/Pacific Islander, and students classified as two or more races) within each urbanicity classification stratum. The race/ethnicity strata were formed using one of three classification schemes as follows:

Case 1: Urbanicity cells where both the second and third largest race/ethnicity groups contained less than 7 percent of students in the urbanicity cell were not stratified by race/ethnicity enrollment (race/ethnicity stratification value was set to 0). There were no race/ethnicity strata formed within these urbanicity cells.

Case 2: Urbanicity cells where the second largest race/ethnicity group contained at least 7 percent but no more than 15 percent of students in the urbanicity cell were stratified into three race/ethnicity cells. Schools were ordered by the sum of the percentage of race/ethnicity enrollment for the second and third largest groups within the urbanicity cell and then divided into three approximately equal size groups in terms of students.

Case 3: Urbanicity cells where both the second and third largest race/ethnicity groups contained more than 15 percent of students in the urbanicity cell were stratified into four race/ethnicity cells. The second largest group provided the primary stratification variable; the third largest group provided the secondary stratification variable. Within an urbanicity cell, schools were first sorted based on the primary stratification variable. Then they were divided into two strata of schools containing approximately equal numbers of students. Within each of these two strata, the schools were sorted by the secondary stratification variable and subdivided into two substrata of schools containing approximately equal numbers of students. The four race/ethnicity classifications consisted of the following values; low primary variable/low secondary variable, low primary variable/high secondary variable, high primary variable/low secondary variable, and high primary variable/high secondary variable.

NAEP Technical Documentation Website

NAEP Technical Documentation Stratification by Achievement Data and Median Income for the 2013 State Assessment

The achievement data obtained from each jurisdiction are derived from the results of state assessment programs. The contents of the achievement data files varied by jurisdiction and included achievement measures for a variety of subjects, grades, and multiple assessment programs. One achievement measure was selected for

Jurisdictions Using Achievement Data or Median Household Income in Stratification

each responding jurisdiction to be used in the stratification process. Where available, the achievement data were used

for implicit stratification by grade. Since the achievement data are more current than the median household income data, as well as more likely to be well-correlated to NAEP assessment scores, they were judged to be a more effective stratification variable. The achievement measures were selected according to the following criteria:

Achievement measures from state assessments conducted in mathematics and reading (in that order of priority) were utilized, if available. For grade 4, data from fourth-grade assessments were used, if available; otherwise, data from third-grade assessments. For grade 8, data from eighth-grade assessments were used, if available; otherwise, data from seventh-grade assessments. For both grades, data from 2009 assessments (the latest available) were used. For grade 12, achievement measures were not available.

Achievement measures should match to at least 70 percent of the schools on the sampling frames.

Achievement measures should differentiate schools from one another. For example, district-level measures, those with high missing rates or pass/fail indicators, were judged not to be useful for differentiating schools. In addition, achievement measures that did not have good dispersion were not used for stratification.

All other things being equal, the possibilities for score types were average scale score, median scale score, percentile rank, median percentile rank, normal curve equivalent, raw score, index score, and percentage above a particular cut score or quartile. In general, the availability varied for any given state/grade/subject/year.

Achievement data useful for implicit stratification were obtained from 50 of 52 jurisdictions for both fourth- and eighth- grade assessments. Where achievement data were not used, median household income was used based on the zip code area in which the school is located. The source of median household income is the 2000 Census.

NAEP Technical Documentation Website

NAEP Technical Documentation Jurisdictions Using Achievement Data or Median Household Income in Stratification for the 2013 State Assessment

This table shows whether achievement data or median household income was used as a stratification variable for participating

jurisdictions. Neither achievement nor median income data was available for stratification of Bureau of Indian Education (BIE) and Department of Defense Education Activity (DoDEA) schools. The estimated grade enrollment was used in these two jurisdictions.

Type of data, achievement or median household income, used for stratification, state assessment, by grade and jurisdiction: 2013

Jurisdiction	Grade 4		Grade 8
Achievement	Income	Achievement	Income
Alabama	YES	NO	YES	NO
Alaska	YES	NO	YES	NO
Arizona	YES	NO	YES	NO
Arkansas	YES	NO	YES	NO
California	YES	NO	YES	NO
Colorado	YES	NO	YES	NO
Connecticut	YES	NO	YES	NO
Delaware	YES	NO	YES	NO
Florida	YES	NO	YES	NO
Georgia	YES	NO	YES	NO
Hawaii	YES	NO	YES	NO
Idaho	YES	NO	YES	NO
Illinois	YES	NO	YES	NO
Indiana	YES	NO	YES	NO
Iowa	YES	NO	YES	NO
Kansas	YES	NO	YES	NO
Kentucky	YES	NO	YES	NO
Louisiana	YES	NO	YES	NO
Maine	YES	NO	YES	NO
Maryland	YES	NO	YES	NO
Massachusetts	YES	NO	YES	NO
Michigan	YES	NO	YES	NO
Minnesota	YES	NO	YES	NO
Mississippi	YES	NO	YES	NO
Missouri	YES	NO	YES	NO
Montana	YES	NO	YES	NO
Nebraska	NO	YES	NO	YES
Nevada	YES	NO	YES	NO
New Hampshire	YES	NO	YES	NO
New Jersey	YES	NO	YES	NO
New Mexico	YES	NO	YES	NO
New York	YES	NO	YES	NO
North Carolina	YES	NO	YES	NO
North Dakota	YES	NO	YES	NO
Ohio	YES	NO	YES	NO
Oklahoma	YES	NO	YES	NO
Oregon	YES	NO	YES	NO
Pennsylvania	YES	NO	YES	NO
Rhode Island	YES	NO	YES	NO
South Carolina	YES	NO	YES	NO
South Dakota	YES	NO	YES	NO
Tennessee	YES	NO	YES	NO

Texas	YES	NO	YES	NO
Utah	YES	NO	YES	NO
Vermont	YES	NO	YES	NO
Virginia	YES	NO	YES	NO
Washington	YES	NO	YES	NO
West Virginia	YES	NO	YES	NO
Wisconsin	YES	NO	YES	NO
Wyoming	YES	NO	YES	NO
Other jurisdictions
Bureau of Indian Education (BIE)	—	—	—	—
Department of Defense Education Activity (DoDEA)	—	—	—	—
District of Columbia	YES	NO	YES	NO
Puerto Rico	NO	YES	NO	YES
— Not available. NOTE: With the exception of the state of Nebraska, and the jurisdiction of Puerto Rico, in all other states and the District of Columbia achievement data was used as a stratification variable for the 2013 state assessment. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Missing Stratification Variables for the 2013 State Assessment

Schools with missing stratification variables had their data imputed as follows:

Schools with missing estimated grade enrollment had their estimated grade enrollment set to 20. Schools missing the urbanicity (urban-centric locale) variable were assigned the modal value of urbanicity for schools in the same five-digit zip code or the same city. The mean ethnicity percentage was imputed at the five-digit zip code level only if all schools were missing ethnicity at the district level, and only at the three-digit zip code prefix if the five-digit zip code mean was missing as well.

Schools with missing or questionable values in race/ethnicity enrollment data—those in which the summation of the ethnicity percentages did not fall in the range 97 through 103, indicating a gross error—were assigned the average race/ethnicity enrollment within their school district, five-digit zip code, or three-digit zip code prefix.

Schools with missing achievement data in jurisdictions and grades for which achievement data were used in stratification were assigned the mean achievement data value within their urbanization and race/ethnicity classification. The achievement data were imputed only for those schools in jurisdictions and grades in which achievement data were used for stratification.

Schools missing median household income were assigned the mean value of median household income for the three-digit zip code prefix in which they were located. In some cases, imputation was not possible at the three-digit zip code level, and needed to be done at the city and state level.

NAEP Technical Documentation School Sample Selection for the 2013 State Assessment

For the grades 4, 8, and 12 public school state assessment samples, schools were sampled independently from each jurisdiction with probability proportional-to-size (PPS) using systematic sampling. Prior to sampling, schools in each jurisdiction were sorted by the appropriate implicit stratification variables (urbanicity status, race/ethnicity status, and achievement score or zip code-based median household income) in a serpentine order. A school's measure of size was a complex function of the school's estimated grade enrollment. Schools whose measure of size was larger than

Computation of Measures of Size

School Sample Sizes: Frame and New School

Evaluation of the Samples Using State Achievement Data

the sampling interval could be selected or “hit” multiple times. Schools with multiple hits were selected with certainty and had larger student sample sizes.

The sampled schools for the public school state assessment samples came from two frames: the public school sample frame (as constructed from the Common Core of Data (CCD)) and the new- school sampling frame.

Schools from the CCD-based frame were sampled at a rate that would yield specific target student sample sizes for each jurisdiction. At grades 4 and 8, jurisdictions had a target sample size of 6,600 students - 3,150 students each for the reading and mathematics operational

assessments and 300 students for pilot tests. For the special mathematics assessment in Puerto Rico, the target sample size was 5,800 students. By design, Bureau of Indian Education

(BIE) schools were not part of the state assessments this year. However, separate BIE school samples were selected based on target student sample sizes that were large enough to ensure that BIE schools were sufficiently represented in the national samples.

At grade 12, the target sample sizes of students differed by jurisdiction and are shown in the following table. These numbers reflect the desired number of assessed students for the reading and mathematics operational assessments (2,300 students per subject) and an upward adjustment to offset expected rates of school and student attrition due to nonresponse and ineligibility.

Target sample sizes of assessed students, grade 12 state assessment, by jurisdiction: 2013

Jurisdiction	Target student sample size
Arkansas	6,200
Connecticut	6,750
Florida	6,600
Idaho	6,250

Illinois	7,250
Iowa	6,850
Massachusetts	6,250
Michigan	7,400
New Hampshire	8,350
New Jersey	6,500
South Dakota	6,500
Tennessee	7,400
West Virginia	6,650
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

Prior to selection, schools were deeply stratified in each jurisdiction to ensure that the school sample distribution reflected the school population distribution as closely as possible, with regard to the stratification variables, to miminimize sampling error. The success of this approach was shown by comparing the proportion of minorities enrolled in schools (based on CCD values for each school), median income, and urban-centric locale (viewed as an interval variable) reported in the original frame against the school sample.

In addition, the distribution of state assessment achievement scores for the original frame can be compared with that of the school sample for those jurisdictions for which state assessment achievement data are available, as was done in the evaluation of the samples using state achievement data. The number of significant differences found in this analysis was smaller than what would be expected to occur by chance, given the large number of comparisons that were made. The number of significant differences remained small even with the use of a finite population correction factor in the calculation of the sampling variances. The close adherence of sample values to frame values suggested there is little evidence that the school sample for NAEP 2013 is not representative of the frame from which it was selected. The achievement/median income variable is used as the third-level sort order variable in the school systematic selection procedure. While it may be a rather low-level sort variable, it still helps control how representative the sampled schools are in terms of achievement. The close agreement between frame and sample values of these achievement/median income variables provided assurance that the selected sample is representative of the frame with respect to achievement status.

NAEP Technical Documentation Website

NAEP Technical Documentation Computation of Measures of Size for the 2013 State Assessment

In designing each school sample, five objectives underlie the process of determining the probability of selection for each school and how many students are to be sampled from each selected school containing the respective grade:

to meet the target student sample size for each grade; to select an equal-probability sample of students;

to limit the number of students selected from any one school;

to ensure that the sample within a school does not include a very high percentage of the students in the school, unless all students are included; and

to reduce the rate of sampling of small schools, in recognition of the greater cost and burden per student of conducting assessments in such schools.

The goal in determining the school's measure of size is to optimize across the last four objectives in terms of maintaining the accuracy of estimates and the cost effectiveness of the sample design. In certain jurisdictions, a census of students was taken so as to meet, as nearly as possible, the target student sample size. Elsewhere, to meet the target student sample and achieve a reasonable compromise among the other four objectives above, the following algorithm was used to assign a measure of size to each school based on its enrollment per grade as indicated on the sampling frame.

The preliminary measures of size (MOS_js) were set as follows:

where x_js is the estimated grade enrollment for school s in jurisdiction j, y_j the target within-school student sample size for jurisdiction j, and z_js the within-school take-all student cutoff for jurisdiction j to which school s belongs.

For grades 4 and 8, the target sample size and take-all cutoff were 63 students and 70 students for all jurisdictions, respectively, with the exception of Puerto Rico, where the target sample size was 50 students, and the take-all cutoff was 55 students. For grade 12, the target sample size and take-all cutoff were 60 students and 66 students, respectively.

The preliminary measure of size reflects the need to lower the expected number of very small schools in

the sample, as the marginal cost for each assessed student in these schools is higher. These very small schools are sampled at half the rate of the larger schools, and their weights are doubled to account for the half sampling.

The next task in this development is to describe b_j, the constant of proportionality for a specified jurisdiction. It is a sampling parameter that, when multiplied by a school’s preliminary measure of size (MOS_js), yields the school’s final measure of size. It is computed in such a way that, when used with the systematic sampling procedure, the target student sample size is achieved.

The final measure of size, E_js, is defined as:

The quantity u_j (the maximum number of “hits” allowed) in this formula is designed to put an upper bound on the burden for the sampled schools. In most jurisdictions, u_j was set to 3. In Alaska, u_j was set to 8, and in Puerto Rico, u_j was set to 1.

In addition, new and newly-eligible schools were sampled from the new-school frame. The assigned measures of size for these schools,

,

used the b_j and u_j values from the CCD-based school frame for the jurisdiction (i.e., the same sampling rate as for the CCD-based school sample within each jurisdiction). The variable π_djs is the probability of selection of the district into the new-school district (d) sample.

NAEP Technical Documentation Website

NAEP Technical Documentation School Sample Sizes: List Frame-Based and New School for the 2013 State Assessment

The following table lists the number of sampled schools taken from the public school sampling frame (as constructed from the Common Core of Data) and the new-school sampling frame, for both fourth and eighth grades, by participating jurisdiction. The school counts shown are at the time of sampling. After school sampling, it was determined that in some Trial Urban District Assessments (TUDAs) a few schools did not contribute to the TUDA's Adequate Yearly Progress (AYP). These schools were then classified as out of scope for the TUDA but in scope for the state.

NAEP state frame-based and new public school samples, state assessment, by grade and jurisdiction: 2013

Jurisdiction	Grade 4			Grade 8
Total school sample	Frame school sample	New school sample	Total school sample	Frame school sample	New school sample
Total	8,350	8,170	190	6,970	6,760	210
Alabama	120	120	0	110	110	0
Alaska	200	200	#	150	140	#
Arizona	120	120	#	120	120	10
Arkansas	120	120	10	110	110	#
California–Fresno	50	50	0	30	30	0
California–Los Angeles	90	80	#	80	80	10
California–San Diego	60	60	0	40	40	#
California–Balance	100	100	#	110	100	10
Colorado	120	120	0	120	120	#
Connecticut	120	120	#	110	110	#
Delaware	100	100	10	70	60	10
Florida–Hillsborourgh County	60	60	#	50	50	#
Florida–Miami	90	80	#	80	80	10
Florida–Balance	90	90	#	90	90	#
Georgia–Atlanta	60	60	0	30	30	0
Georgia–Balance	100	100	#	100	100	#
Hawaii	120	120	#	60	60	#
Idaho	130	130	#	100	100	#
Illinois–Chicago	100	100	#	100	100	#
Illinois–Balance	100	100	0	100	100	0
Indiana	120	120	#	110	110	#
Iowa	140	140	#	120	120	#
Kansas	150	140	10	130	130	10
Kentucky–Jefferson County	50	50	0	40	40	0
Kentucky–Balance	100	100	#	100	100	#
Louisiana	130	120	10	150	120	30
Maine	160	160	#	120	120	0
Maryland–Baltimore	70	70	#	70	60	#

Maryland–Balance	100	100	#	100	100	0
Massachusetts–Boston	80	80	0	40	40	0
Massachusetts–Balance	110	110	0	100	100	#
Michigan–Detroit	80	80	0	70	70	0
Michigan–Balance	110	110	0	110	110	0
Minnesota	130	130	#	130	130	10
Mississippi	120	110	10	110	110	0
Missouri	130	130	#	130	120	#
Montana	200	200	0	150	150	0
Nebraska	170	170	0	130	130	0
Nevada	120	110	#	90	90	#
New Hampshire	130	130	0	90	90	0
New Jersey	120	120	#	110	110	#
New Mexico–Albuquerque	60	60	0	40	40	#
New Mexico–Balance	100	90	#	80	80	#
New York–New York City	80	80	#	90	80	#
New York–Balance	80	80	0	70	70	0
North Carolina–Charlotte	50	50	0	40	40	#
North Carolina–Balance	110	100	10	100	100	#
North Dakota	270	260	10	190	190	#
Ohio–Cleveland	90	80	#	90	80	#
Ohio–Balance	120	110	#	110	110	#
Oklahoma	140	140	10	130	130	10
Oregon	130	130	#	130	120	10
Pennsylvania–Philadelphia	60	60	0	60	60	#
Pennsylvania–Balance	110	110	0	100	100	0
Rhode Island	120	120	#	60	60	10
South Carolina	120	110	#	110	110	#
South Dakota	190	190	0	150	150	0
Tennessee	120	120	#	110	110	#
Texas–Austin	60	50	#	30	20	#
Texas–Dallas	60	50	#	40	40	#
Texas–Houston	80	80	#	50	50	#
Texas–Balance	110	100	10	110	110	10
Utah	120	110	10	120	110	10
Vermont	220	220	0	120	120	#
Virginia	110	110	0	110	110	#
Washington	120	120	#	120	120	0
West Virginia	150	150	0	110	110	#
Wisconsin–Milwaukee	70	70	#	60	60	#
Wisconsin–Balance	120	120	#	110	110	#
Wyoming	200	190	10	100	90	10
Other jurisdictions
Bureau of Indian Education (BIE)	20	20	0	10	10	#
Department of Defense Education Activity (DoDEA)	120	110	10	70	60	10
District of Columbia (TUDA)	90	90	#	40	40	#
District of Columbia–Balance	50	40	10	50	40	10
Puerto Rico	170	160	10	130	120	10
# Rounds to zero. NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding.

SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

The following table lists the number of sampled schools taken from the public school sampling frame (as constructed from the Common Core of Data) and the new-school sampling frame, for twelfth grade, by participating jurisdiction.

NAEP state frame-based and new public school samples, grade 12 state assessment, by jurisdiction: 2013

Jurisdiction	Total school sample	Frame school sample	New school sample
Total 1	1,460	1,460	10
Arkansas	100	100	0
Connecticut	110	110	0
Florida	120	120	#
Idaho	100	100	0
Illinois	130	130	0
Iowa	120	120	0
Massachusetts	110	110	#
Michigan	140	140	#
New Hampshire	80	80	0
New Jersey	110	110	#
South Dakota	140	140	0
Tennessee	130	130	0
West Virginia	90	90	0
1 The aggregate of the 13 states participating in the state assessments at grade 12. # Rounds to zero. NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation Evaluation of the Samples for the 2013 State Assessment Using State Achievement Data

The purpose of this analysis was to determine whether public schools selected for the 2013 samples were representative of the schools on the NAEP sampling frames in terms of student achievement. Percentiles of the achievement distributions were compared between the frame and sample schools for each public school jurisdiction in grades 4, 8, and 12.

Achievement Data

For grades 4 and 8, the achievement variable used in the analysis was the same variable used in the NAEP sample design to stratify the public school frame. For most jurisdictions, the variable was an achievement score provided by the jurisdiction.

However, for some jurisdictions where achievement data were not available, median household income from the 2000 Census was used. (In 2000, the Census determined median household income based on the five-digit zip code area in which the school was located.) The achievement data consisted of various types of school-specific achievement measures from state assessment programs. The type of achievement data available varied by jurisdiction. For instance, in some states, the measure was the average score for a given state assessment. In other states, the measure was a percentile rank or percentage of students above a specific score. For grade 12, since achievement data was not available, median household income based on zip code area from the 2000 Census was used.

During frame development, not every record on the Common Core of Data (CCD) file matched to the achievement data files created for the National Center for Education Statistics (NCES), even in jurisdictions where those data were generally available. For schools that did not match, their achievement score was imputed by a mean matching imputation approach using the mean achievement score for schools with complete achievement data within the same jurisdiction-urbanicity- race/ethnicity stratum combination.

Methodology

To determine whether the distributions between the frame and sample schools were different, comparisons of percentile estimates were made for the 10th, 25th, 50th, 75th, and 90th percentile levels as well as the mean for each public school jurisdiction by grade. Frame and sample school estimates were considered statistically different if the frame value fell outside the 95 percent confidence interval of the corresponding sample estimate. The percentile values for the frame schools were calculated by weighting each school by the estimated number of students in the given grade. The percentile estimates for the sample schools were calculated using school weights and weighted by the school measure of size (estimated number of students in the given grade). The 95 percent confidence intervals for the school sample estimates were calculated in WesVar—software for computing estimates of sampling variance from complex sample survey (Westat, 2000b —using the Woodruff method (Sarndal, Swensson, and Wretman 1992) with the use of a finite population correction factor.

Results

As mentioned above, sample and frame achievement distributions were determined to be different if at least one of the percentile estimates or the mean differed significantly at the 95 percent confidence level. Out of all the jurisdiction and grade comparisons (excluding jurisdictions where all schools in the frame were selected), only 14 of the 810 distributions compared were found to be significantly different. They are shown in the table below

Summary of significant differences in achievement measures between the sample and the frame, state assessment, by jurisdiction and grade: 2013

Grade	Jurisdiction	Achievement data / median income	Estimate	Frame	Sample	Confidence interval
4	Delaware	Achievement data	75th percentile	87.23	86.47	(86.38, 87.20)
	Fresno TUDA	Achievement	25th	46.52	46.42	(46.36, 46.51)

		data	percentile
	San Diego TUDA	Achievement data	mean	66.19	66.97	(66.26, 67.69)
8	Maine	Achievement data	25th percentile	45.94	46.28	(46.16, 46.38)
	New Mexico	Achievement data	50th percentile	39.98	37.58	(37.19, 39.85)
	New Mexico	Achievement data	mean	42.15	41.21	(40.47, 41.95)
	South Dakota	Achievement data	25th percentile	68.86	69.00	(68.87, 70.00)
	South Dakota	Achievement data	90th percentile	90.93	89.53	(89.21, 90.54)
	Detroit TUDA	Achievement data	75th percentile	62.68	61.45	(59.36, 62.48)
	Detroit TUDA	Achievement data	90th percentile	75.26	74.67	(74.15, 75.21)
	Hillsborough TUDA	Achievement data	75th percentile	76.35	76.46	(76.36, 76.58)
	Houston TUDA	Achievement data	mean	78.83	79.11	(78.91, 79.31)
12	Illinois	Median income	10th percentile	31,564.65	30,157.62	(28,203.36, 31,475.48)
	Tennessee	Median income	90th percentile	55,748.34	52,008.12	(51,304.59, 55,454.43)
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

The number of significant differences found in this analysis was smaller than what would be expected to occur by chance, given the large number of comparisons that were made. Also, the number of significant differences remained small even with the added use of a finite population correction factor in the calculation of the sampling variances. Even in the statistically significant cases, the close adherence of sample values to frame values suggests there is little evidence that the school sample for NAEP 2013 is not representative of the frame from which it was selected. The achievement/median income variable is used as the fourth-level sort order variable in the school systematic selection procedure. While it may be a rather low level sort variable, it still helps control how representative the sampled schools are in terms of achievement. The close agreement between frame and sample values of these achievement/median income variables provided assurance that the selected sample is representative of the frame with respect to achievement or income status.

NAEP Technical

Documentation Substitute Schools for the 2013 State Assessment

As participation is effectively mandatory by law at fourth and eighth grades, substitute schools for nonresponding schools were not provided. However, participation was not mandatory at twelfth grade, and substitute schools were used. Substitutes were preselected for the twelfth-grade public school sample by sorting the school frame file according to the actual order used in the sampling process (the implicit stratification). Each sampled school had each of its nearest neighbors on the school frame file selected as a potential substitute. The last sort ordering was by grade enrollment. The result was that the nearest neighbors had grade enrollment values very close to that of the sampled school. To be eligible as a potential substitute, the neighbor needed to be a nonsampled school (for any grade). The school also needed to be in the same implicit stratum as the sampled school. If both nearest neighbors were eligible to be substitutes, the one with a closer grade enrollment was chosen.

Five substitutes participated in the twelfth-grade public school sample in the 13 states.

NAEP Technical Documentation Ineligible Schools for the 2013 State Assessment

The Common Core of Data (CCD) public school file from which most of the sampled schools were drawn corresponds to the 2009-2010 school year, some 3 years prior to the assessment school year. During the intervening period, some of these

Eligible Schools Sampled

Ineligible Sampled Schools by Ineligibility Type

schools either closed, no longer offered the grade of interest, or were ineligible for other reasons. In such cases, the sampled school was coded as ineligible.

NAEP Technical Documentation Website

NAEP Technical Documentation Eligible Schools Sampled for the 2013 State Assessment

The following table shows the number of eligible fourth- and eighth-grade schools sampled for each NAEP 2013 state assessment jurisdiction.

Eligible sampled schools, state assessment, by grade and jurisdiction: 2013

Jurisdiction	Grade 4		Grade 8
Total school sample	Eligible school sample	Total school sample	Eligible school sample
Total	8,350	7,860	6,970	6,440
Alabama	120	110	110	100
Alaska	200	180	150	120
Arizona	120	120	120	110
Arkansas	120	120	110	110
California–Fresno	50	50	30	20
California–Los Angeles	90	80	80	80
California–San Diego	60	60	40	40
California–Balance	100	100	110	100
Colorado	120	120	120	110
Connecticut	120	110	110	110
Delaware	100	90	70	50
Florida–Hillsborourgh County	60	60	50	50
Florida–Miami	90	80	80	70
Florida–Balance	90	90	90	80
Georgia–Atlanta	60	50	30	20
Georgia–Balance	100	100	100	100
Hawaii	120	120	60	60
Idaho	130	120	100	100
Illinois–Chicago	100	90	100	90
Illinois–Balance	100	90	100	90
Indiana	120	110	110	100
Iowa	140	130	120	110
Kansas	150	140	130	120
Kentucky–Jefferson County	50	50	40	30
Kentucky–Balance	100	100	100	100
Louisiana	130	110	150	120

Maine	160	150	120	110
Maryland–Baltimore	70	70	70	60
Maryland–Balance	100	100	100	100
Massachusetts–Boston	80	70	40	40
Massachusetts–Balance	110	100	100	100
Michigan–Detroit	80	60	70	40
Michigan–Balance	110	100	110	100
Minnesota	130	120	130	110
Mississippi	120	110	110	110
Missouri	130	130	130	120
Montana	200	190	150	140
Nebraska	170	160	130	120
Nevada	120	120	90	90
New Hampshire	130	130	90	90
New Jersey	120	120	110	110
New Mexico–Albuquerque	60	60	40	40
New Mexico–Balance	100	100	80	80
New York–New York City	80	80	90	80
New York–Balance	80	70	70	70
North Carolina–Charlotte	50	50	40	30
North Carolina–Balance	110	100	100	100
North Dakota	270	250	190	180
Ohio–Cleveland	90	70	90	70
Ohio–Balance	120	110	110	110
Oklahoma	140	140	130	130
Oregon	130	130	130	120
Pennsylvania–Philadelphia	60	60	60	50
Pennsylvania–Balance	110	100	100	90
Rhode Island	120	120	60	60
South Carolina	120	110	110	110
South Dakota	190	180	150	140
Tennessee	120	110	110	110
Texas–Austin	60	50	30	20
Texas–Dallas	60	50	40	40
Texas–Houston	80	80	50	50
Texas–Balance	110	110	110	110
Utah	120	110	120	110
Vermont	220	220	120	120
Virginia	110	110	110	110
Washington	120	120	120	110
West Virginia	150	140	110	100
Wisconsin–Milwaukee	70	60	60	50
Wisconsin–Balance	120	120	110	100
Wyoming	200	180	100	90
Other jurisdictions
Bureau of Indian Education	20	20	10	10

(BIE)
Department of Defense Education Activity (DoDEA)	120	100	70	60
District of Columbia (TUDA)	90	80	40	30
District of Columbia–Balance	50	40	50	40
Puerto Rico	170	150	130	120
NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. "Balance" refers to the part of the state outside of the TUDA district(s). SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

The following table shows the number of eligible twelfth-grade schools sampled for each NAEP 2013 state assessment jurisdiction.

Eligible sampled schools, grade 12 state assessment, by jurisdiction: 2013

Jurisdiction	Total school sample	Eligible school sample
Total 1	1,460	1,390
Arkansas	100	100
Connecticut	110	100
Florida	120	110
Idaho	100	90
Illinois	130	120
Iowa	120	120
Massachusetts	110	110
Michigan	140	130
New Hampshire	80	80
New Jersey	110	110
South Dakota	140	130
Tennessee	130	120
West Virginia	90	90
1 The aggregate of the 13 states participating in the state assessments at grade 12. NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation Ineligible Sampled Schools by Ineligibility Type for the 2013 State Assessment

The following table shows the unweighted counts and percentages of NAEP 2013 state assessment fourth- and eighth-grade schools that were eligible and ineligible, by reason for ineligibility.

School eligibility status, state assessment, by grade and eligibility status: 2013

Eligibility status	Grade 4		Grade 8
Unweighted count of schools	Unweighted percentage	Unweighted count of schools	Unweighted percentage
All sampled public schools	8,350	100.00	6,970	100.00
Eligible	7,860	94.13	6,400	92.40
No eligible students in grade	56	0.67	48	0.69
Does not have sampled grade	109	1.31	144	2.07
School closed	259	3.10	186	2.67
Not a regular school	55	0.66	116	1.66
Other ineligible school	16	0.19	33	0.47
Duplicate on sampling frame	1	0.01	2	0.03
NOTE: Numbers of schools are rounded to nearest ten, except those pertaining to ineligible schools. Detail may not sum to totals because of rounding. Percentages are based on rounded counts. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

The following table shows the unweighted counts and percentages of NAEP 2013 state assessment twelfth-grade schools that were eligible and ineligible, by reason for ineligibility.

School eligibility status, grade 12 state assessment, by eligibility status: 2013

Eligibility status	Unweighted count of schools	Unweighted percentage
Total1	1,460	100.00
Eligible	1,390	95.21
No eligible students in grade	7	0.48
Does not have sampled grade	9	0.62
School closed	19	1.30
Not a regular school	22	1.51
Other ineligible school	13	0.89
Duplicate on sampling frame	0	0.00
1The aggregate of the 13 states participating in the state assessments at grade 12. NOTE: Numbers of schools are rounded to nearest ten, except those pertaining to ineligible schools. Detail may not sum to totals because of rounding. Percentages are based on rounded counts. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation Student Sample Selection for the 2013 State Assessment

Within each sampled school, a sample of students was selected from a listing of the students in the sampled grade such that every student had an equal chance of selection. The student lists were submitted either electronically using a system known as E-filing or on paper. In E-

filing, student lists are submitted in Excel files by either school coordinators or NAEP State Coordinators. The files can be submitted for one school at a time (known as single school E-file submission) or for an entire jurisdiction at once (known as multiple school E-file submission). E- filing allows schools to easily submit student demographic data electronically with the student lists, easing the burden on NAEP field supervisors and school coordinators. Schools that are unable to submit their student lists using the E-filing system provide hardcopy lists via the student listing form to NAEP field supervisors. In 2013, there were 16,898 schools that E-filed their student lists, while 548 lists were submitted using the student listing form.

In year-round, multi-track schools, students who were not scheduled to be in school on the assessment day were removed from the student lists prior to sampling. Student base weights were adjusted to account for these students.

The sampling process was the same, regardless of list submission type. The sampling process was systematic(e.g., if the sampling rate was one-half, a random starting point of one or two was chosen, and every other student on the list was selected). For E-filed schools only, where demographic data was submitted for every student on the frame, students were sorted by gender and race/ethnicity before the sample was selected to implicitly stratify the sample.

In the certainty jurisdictions, all students were sampled in all schools. Otherwise, the sample size for grades 4 and 8 was 63 students (with the exception of 50 students in Puerto Rico), and the sample size for grade 12 was 60 students. Larger schools may have been selected with certainty in the sampling process and thus may have a larger sample size. In addition, most fourth-grade schools chose the option of taking all students when enrollment was less than 90 students. This increased the fourth-grade sample size in many states beyond the designated target.

Some students enrolled in the school after the sample was selected. In such cases, new enrollees were sampled at the same rate as the students on the original list.

In fourth- and eighth-grade schools, sampled students were randomly assigned to mathematics, reading, and pilot as follows: 30 students for mathematics, 30 students for reading, and 3 students for pilot. However, for schools in Puerto Rico, only the special mathematics assessment was conducted. In twelfth-grade schools, sampled students were randomly assigned to mathematics and reading as follows: 30 students for mathematics and 30 students for reading. This was implemented by spiraling: the booklets assigned to sampled students were provided from booklet packets that had, on average, the correct ratio of each of the relevant assessments in a randomized order.

Some of the students who were English language learners (ELL) or students with disabilities

(SD) were excluded from the assessment because they could not be assessed with the accommodations NAEP provides.

NAEP Technical Documentation Website

NAEP Technical Documentation School and Student Participation in the 2013 State Assessment

In all cases in the 2013 state assessment for grades 4 and 8, the weighted response rates for schools in each jurisdiction exceeded the 85 percent standard established by the National Center for Education Statistics (NCES). As participation is effectively mandatory, substitute schools for nonresponding schools were not provided. Participation was not mandatory at grade 12, and substitute schools were used.

In every NAEP survey, some of the sampled students are not assessed for the following reasons:

withdrawn students;

excluded students with disabilities (SD);

excluded English language learner (ELL) students; or

Weighted Response Rates of Fourth-Grade School Sample by Participating Jurisdiction

Weighted Response Rates of Eighth-Grade School Sample by Participating Jurisdiction

Weighted Response Rates of Twelfth-Grade School Sample by Participating State

Weighted Student Response and Exclusion Rates, Mathematics Assessment

Weighted Student Response and Exclusion Rates, Reading Assessment

students absent from both the original session and the make-up session (not excluded but not assessed).

Withdrawn students are those who have left the school before the original assessment. Excluded students were determined by their school to be unable to meaningfully take the NAEP assessment in their assigned subject, even with an accommodation. Excluded students must also be classified as SD and/or ELL. Other students who were absent for the initial session are assessed in the makeup session. The last category includes students who were not excluded (i.e., “were to be assessed”) but were not assessed either due to absence from both sessions or because of a refusal to participate. Assessed students are also classified as assessed without an accommodation or assessed with an accommodation. The latter group can be divided into SD students assessed with an accommodation, ELL students assessed with an accommodation, or students who are both SD and ELL and accommodated. Note that some SD and ELL students are assessed without accommodations, and students who are neither SD nor ELL can only be assessed without an accommodation.

The weighted response rates utilize the student base weights and indicate the weighted percentage of assessed students among all students to be assessed. The exclusion rates, in contrast, provide the weighted percentage of excluded SD or ELL students among all absent, assessed, and excluded students.

NAEP Technical Documentation Website

NAEP Technical Documentation Weighted Response Rates of Fourth-Grade School Sample by Participating Jurisdiction for the 2013 State Assessment

The following table presents unweighted counts and weighted response rates at grade 4 for sampled eligible and participating schools. States with Trial Urban District Assessment (TUDA) districts are shown in multiple rows: for the TUDA district(s) and for the state as a whole (the TUDA district[s] plus the rest of the state). The weighted school response rates estimate the proportion of the student population that is represented by the participating school sample prior to substitution.

Participation is effectively mandatory for all states and districts, but not for Bureau of Indian Education (BIE) or Department of Defense Education Activity (DoDEA) schools.

School counts and response rates of sampled eligible schools, grade 4 state assessment, by jurisdiction: 2013

Jurisdiction	Number of sampled eligible schools	Number of participating schools	Weighted school response rates (percent)
Total	7,860	7,830	99.67
Alabama	110	110	100.00
Alaska	180	170	99.48
Arizona	120	120	100.00
Arkansas	120	120	100.00
California–Fresno	50	50	100.00
California–Los Angeles	80	80	100.00
California–San Diego	50	50	100.00
California	290	290	99.17
Colorado	120	120	100.00
Connecticut	110	110	97.22
Delaware	90	90	100.00
Florida–Hillsborough County	60	60	100.00
Florida–Miami	80	80	100.00
Florida	220	220	100.00
Georgia–Atlanta	50	50	100.00
Georgia	150	150	100.00
Hawaii	120	120	100.00
Idaho	120	120	100.00
Illinois–Chicago	90	90	100.00
Illinois	180	180	97.98
Indiana	110	110	100.00
Iowa	130	130	100.00
Kansas	140	140	100.00
Kentucky–Jefferson County	50	50	100.00
Kentucky	150	150	100.00
Louisiana	110	110	100.00
Maine	150	150	100.00
Maryland–Baltimore	70	70	100.00
Maryland	170	170	100.00
Massachusetts–Boston	70	70	100.00
Massachusetts	170	170	100.00
Michigan–Detroit	50	50	100.00
Michigan	150	150	100.00
Minnesota	120	120	100.00
Mississippi	110	110	100.00
Missouri	130	130	100.00
Montana	190	190	99.85
Nebraska	160	160	100.00

Nevada	120	120	100.00
New Hampshire	130	130	100.00
New Jersey	120	120	100.00
New Mexico–Albuquerque	50	50	100.00
New Mexico	150	150	99.69
New York–New York City	80	80	100.00
New York	160	150	98.84
North Carolina–Charlotte	50	50	100.00
North Carolina	160	160	100.00
North Dakota	250	250	99.86
Ohio–Cleveland	70	70	100.00
Ohio	180	180	100.00
Oklahoma	140	140	100.00
Oregon	130	130	100.00
Pennsylvania–Philadelphia	60	60	100.00
Pennsylvania	160	160	100.00
Rhode Island	120	120	100.00
South Carolina	110	110	100.00
South Dakota	180	180	100.00
Tennessee	110	110	100.00
Texas–Austin	50	50	100.00
Texas–Dallas	50	50	100.00
Texas–Houston	80	80	100.00
Texas	290	290	100.00
Utah	110	110	99.08
Vermont	220	220	100.00
Virginia	110	110	100.00
Washington	120	120	99.09
West Virginia	140	140	100.00
Wisconsin–Milwaukee	60	60	100.00
Wisconsin	180	180	100.00
Wyoming	180	180	100.00
Other jurisdictions
Bureau of Indian Education (BIE)	20	10	80.19
Department of Defense Education Activity (DoDEA)	100	100	99.23
District of Columbia (TUDA)	80	80	100.00
District of Columbia	120	120	100.00
Puerto Rico	150	150	100.00
NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation Weighted Response Rates of Eighth-Grade School Sample by Participating Jurisdiction for the 2013 State Assessment

The following table presents unweighted counts and weighted response rates at grade 8 for sampled eligible and participating schools. States with Trial Urban District Assessment (TUDA) districts are shown in multiple rows: for the TUDA district(s) and for the state as a whole (the TUDA district[s] plus the rest of the state). The weighted school response rates estimate the proportion of the student population that is represented by the participating school sample prior to substitution.

Participation is effectively mandatory for all states and districts, but not for Bureau of Indian Education (BIE) or Department of Defense Education Activity (DoDEA) schools.

School counts and response rates of sampled eligible schools, grade 8 state assessment, by jurisdiction: 2013

Jurisdiction	Number of sampled eligible schools	Number of participating schools	Weighted school response rates (percent)
Total	6,440	6,420	99.47
Alabama	100	100	100.00
Alaska	120	120	99.91
Arizona	110	110	99.03
Arkansas	110	110	100.00
California–Fresno	20	20	100.00
California–Los Angeles	70	70	100.00
California–San Diego	30	30	100.00
California	230	230	100.00
Colorado	110	110	100.00
Connecticut	110	110	98.00
Delaware	50	50	100.00
Florida–Hillsborourgh County	50	50	100.00
Florida–Miami	70	70	100.00
Florida	200	200	100.00
Georgia–Atlanta	20	20	100.00
Georgia	120	120	100.00
Hawaii	60	60	100.00
Idaho	100	100	100.00
Illinois–Chicago	90	90	100.00
Illinois	180	180	100.00
Indiana	100	100	97.06
Iowa	110	110	100.00
Kansas	120	120	100.00
Kentucky–Jefferson County	30	30	100.00
Kentucky	130	130	99.04
Louisiana	120	120	100.00
Maine	110	110	100.00
Maryland–Baltimore	50	50	100.00
Maryland	160	160	100.00
Massachusetts–Boston	40	40	100.00
Massachusetts	140	140	100.00
Michigan–Detroit	40	40	100.00
Michigan	150	150	100.00
Minnesota	110	110	98.99
Mississippi	110	110	100.00
Missouri	120	120	100.00
Montana	140	140	99.80

Nebraska	120	120	100.00
Nevada	90	90	100.00
New Hampshire	90	90	100.00
New Jersey	110	110	100.00
New Mexico–Albuquerque	30	30	100.00
New Mexico	120	120	99.68
New York–New York City	80	80	99.00
New York	160	150	93.08
North Carolina–Charlotte	30	30	100.00
North Carolina	130	130	100.00
North Dakota	180	180	99.92
Ohio–Cleveland	70	70	100.00
Ohio	170	170	100.00
Oklahoma	130	130	100.00
Oregon	120	120	100.00
Pennsylvania–Philadelphia	50	50	100.00
Pennsylvania	150	150	100.00
Rhode Island	60	60	100.00
South Carolina	110	110	100.00
South Dakota	140	140	100.00
Tennessee	110	110	100.00
Texas–Austin	20	20	100.00
Texas–Dallas	40	40	100.00
Texas–Houston	50	50	100.00
Texas	210	210	100.00
Utah	110	110	100.00
Vermont	120	120	100.00
Virginia	110	110	100.00
Washington	110	110	100.00
West Virginia	100	100	100.00
Wisconsin–Milwaukee	50	50	100.00
Wisconsin	150	150	100.00
Wyoming	90	90	100.00
Other jurisdictions
Bureau of Indian Education (BIE)	10	10	69.29
Department of Defense Education Activity (DoDEA)	60	60	99.40
District of Columbia (TUDA)	30	30	100.00
District of Columbia	70	70	100.00
Puerto Rico	120	120	100.00
NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation Weighted Response Rates of Twelfth-Grade School Sample by Participating State for the 2013 State Assessment

The following table presents unweighted counts and weighted response rates at grade 12 for sampled eligible and participating schools. The weighted school response rates estimate the proportion of the student population that is represented by the participating school sample prior to substitution.

School counts and response rates of sampled eligible schools, grade 12 state assessment, by jurisdiction: 2013

Jurisdiction	Number of sampled eligible schools	Number of participating schools	Weighted school response rates (percent)
Total 1	1,390	1,380	98.78
Arkansas	100	100	100.00
Connecticut	100	100	98.93
Florida	110	100	99.05
Idaho	90	90	100.00
Illinois	120	110	90.38
Iowa	120	120	100.00
Massachusetts	110	110	99.04
Michigan	130	130	100.00
New Hampshire	80	80	100.00
New Jersey	110	110	98.14
South Dakota	130	130	99.74
Tennessee	120	120	100.00
West Virginia	90	90	100.00
1The aggregate of the 13 states participating in the state assessments at grade 12. NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation Weighted Student Response and Exclusion Rates for the 2013 State Mathematics Assessment

The following table presents the weighted student response and exclusion rates for the mathematics assessment. States with Trial Urban District Assessment (TUDA) districts are shown in multiple rows: for the TUDA district(s) and for the state as a whole (the TUDA district[s] plus the rest of the state). The weighted response rates utilize the student base weights and indicate the weighted percentage of assessed students as a percentage of all students to be assessed. The exclusion rates give the weighted percentage of excluded students, those with disabilities (SD) or students who were English language learners (ELL), among all absent, assessed, and excluded students.

Weighted student response and exclusion rates, state mathematics assessment, by grade and jurisdiction: 2013

Jurisdiction	Grade 4			Grade 8
Weighted student response rates (percent)	Weighted percentage of all students who were SD and excluded	Weighted percentage of all students who were ELL and excluded	Weighted student response rates (percent)	Weighted percentage of all students who were SD and excluded	Weighted percentage of all students who were ELL and excluded
Total	94.49	1.25	0.46	93.02	1.28	0.40
Alabama	94.82	1.03	0.10	94.23	0.91	0.13
Alaska	93.18	0.98	0.22	91.72	1.01	0.23
Arizona	95.07	0.88	0.34	93.42	0.98	0.32
Arkansas	94.66	1.16	0.10	95.00	1.80	0.24
California–Fresno	93.58	0.90	0.22	92.52	1.65	0.79
California–Los Angeles	95.80	1.57	1.07	94.39	1.14	0.82
California–San Diego	95.18	1.11	0.74	92.60	1.89	1.00
California	94.79	1.54	1.20	93.59	1.20	0.70
Colorado	92.34	1.04	0.35	93.47	1.05	0.23
Connecticut	93.85	1.19	0.22	92.44	1.81	0.34
Delaware	94.36	1.98	0.22	90.65	1.03	0.32
Florida– Hillsborough County	95.74	1.11	0.10	93.78	1.35	0.13
Florida–Miami	95.07	0.93	1.66	92.63	0.96	1.29
Florida	94.11	1.25	0.76	91.06	1.14	0.64
Georgia–Atlanta	95.42	0.80	0.19	91.57	0.72	0.00
Georgia	94.18	1.34	0.15	93.38	1.30	0.25
Hawaii	94.70	0.81	0.53	90.26	0.97	0.88
Idaho	95.24	1.00	0.44	94.15	1.03	0.11
Illinois–Chicago	94.85	0.71	0.58	94.80	0.77	0.65
Illinois	94.40	0.72	0.39	94.48	0.79	0.25
Indiana	95.18	1.31	0.21	92.49	1.58	0.05
Iowa	95.16	0.53	0.20	93.74	0.73	0.04
Kansas	94.79	1.43	0.31	93.94	1.57	0.10

Kentucky– Jefferson County	94.66	1.03	0.71	93.37	1.60	0.22
Kentucky	94.67	1.26	0.19	94.54	1.98	0.18
Louisiana	94.49	0.97	0.12	94.14	1.03	0.03
Maine	93.95	1.94	0.29	92.79	1.21	0.15
Maryland– Baltimore	94.32	1.24	0.34	89.54	1.50	0.20
Maryland	94.22	0.76	0.24	92.08	1.21	0.52
Massachusetts– Boston	93.72	2.83	1.46	91.61	2.25	0.88
Massachusetts	93.74	1.75	0.46	91.98	1.40	0.77
Michigan–Detroit	90.92	4.33	0.83	91.58	4.29	0.00
Michigan	94.14	1.58	0.44	92.93	1.96	0.58
Minnesota	94.85	1.27	0.18	91.58	1.50	0.27
Mississippi	95.44	0.67	0.10	93.80	0.77	0.03
Missouri	95.42	1.32	0.09	94.25	1.24	0.03
Montana	93.92	1.64	0.18	92.28	1.44	0.03
Nebraska	95.37	1.50	0.25	93.41	1.59	0.26
Nevada	95.75	1.14	0.40	92.80	0.75	0.30
New Hampshire	93.74	1.14	0.08	91.60	0.99	0.07
New Jersey	94.85	0.94	0.26	92.26	1.20	0.47
New Mexico– Albuquerque	94.71	0.93	0.51	90.76	1.45	0.31
New Mexico	95.06	1.07	0.42	93.07	1.52	0.32
New York–New York City	91.74	0.44	1.12	91.78	0.99	1.05
New York	92.27	0.88	0.51	91.15	1.59	0.43
North Carolina– Charlotte	94.18	0.91	0.56	90.94	1.00	0.39
North Carolina	94.19	1.12	0.31	92.95	1.02	0.28
North Dakota	95.57	2.39	0.22	94.98	2.71	0.33
Ohio–Cleveland	93.62	3.70	0.73	91.57	2.15	0.54
Ohio	94.29	1.20	0.13	93.07	1.47	0.05
Oklahoma	94.35	1.77	0.17	92.97	1.41	0.30
Oregon	94.18	1.95	0.51	92.91	1.38	0.12
Pennsylvania– Philadelphia	94.71	2.84	0.95	92.67	2.79	1.02
Pennsylvania	94.30	1.43	0.28	92.17	1.40	0.30
Rhode Island	94.98	0.95	0.21	93.93	0.72	0.39
South Carolina	96.08	1.02	0.10	94.19	1.17	0.18
South Dakota	95.36	1.42	0.09	94.44	1.17	0.25
Tennessee	94.21	1.08	0.37	92.81	1.62	0.22
Texas–Austin	93.69	1.57	0.78	90.97	1.60	0.51
Texas–Dallas	95.79	1.93	0.84	93.81	1.97	1.11
Texas–Houston	96.62	1.22	1.03	92.37	1.74	0.77
Texas	95.36	1.34	0.62	93.82	1.32	0.80
Utah	94.79	1.08	0.43	92.07	1.32	0.30
Vermont	95.04	1.26	0.14	93.91	0.70	0.19
Virginia	94.35	1.23	0.38	93.39	0.76	0.29
Washington	93.50	2.00	0.36	90.87	1.70	0.42
West Virginia	94.77	1.65	0.09	92.62	1.69	0.00
Wisconsin– Milwaukee	94.68	3.07	0.60	91.60	3.56	0.93
Wisconsin	95.42	1.64	0.31	94.25	1.43	0.19
Wyoming	94.65	0.89	0.18	93.66	1.43	0.07

Other jurisdictions
Bureau of Indian Education (BIE)	93.34	1.42	0.00	92.02	3.28	0.00
Department of Defense Education Activity (DoDEA)	95.05	1.17	0.74	94.47	0.82	0.49
District of Columbia (TUDA)	95.52	1.27	0.85	90.15	0.75	1.23
District of Columbia	95.09	0.86	0.62	91.26	0.41	0.71
Puerto Rico 1	94.47	0.00	0.24	92.75	0.01	0.02
1 In Puerto Rico, a special mathematics assessment was conducted instead of the operational mathematics assessment. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Mathematics Assessment.

Similarly, the following table presents the weighted student response and exclusion rates for the twelfth-grade mathematics assessment.

Weighted student response and exclusion rates, grade 12 state mathematics assessment, by jurisdiction: 2013

Jurisdiction	Weighted student response rates (percent)	Weighted percentage of all students who were SD and excluded	Weighted percentage of all students who were ELL and excluded
Arkansas	92.09	2.78	0.30
Connecticut	81.22	1.62	0.18
Florida	77.25	3.01	0.29
Idaho	89.17	1.61	0.04
Illinois	85.16	1.82	0.14
Iowa	83.05	1.13	0.00
Massachusetts	81.71	2.12	0.46
Michigan	86.94	1.84	0.10
New Hampshire	76.64	1.58	0.02
New Jersey	84.10	1.56	0.33
South Dakota	87.48	1.45	0.06
Tennessee	88.15	2.45	0.15
West Virginia	83.68	2.00	0.00
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Mathematics Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation Weighted Student Response and Exclusion Rates for the 2013 State Reading Assessment

The following table presents the weighted student response and exclusion rates for the reading assessment. States with Trial Urban District Assessment (TUDA) districts are shown in multiple rows: for the TUDA district(s) and for the state as a whole (the TUDA district[s] plus the rest of the state). The weighted response rates utilize the student base weights and indicate the weighted percentage of assessed students as a percentage of all students to be assessed. The exclusion rates give the weighted percentage of excluded students, those with disabilities (SD) or students who were English language learners (ELL), among all absent, assessed, and excluded students.

Weighted student response and exclusion rates, state reading assessment, by grade and jurisdiction: 2013

Jurisdiction	Grade 4			Grade 8
Weighted student response rates (percent)	Weighted percentage of all students who were SD and excluded	Weighted percentage of all students who were ELL and excluded	Weighted student response rates (percent)	Weighted percentage of all students who were SD and excluded	Weighted percentage of all students who were ELL and excluded
Total	94.70	2.14	0.90	92.93	1.94	0.57
Alabama	95.49	1.02	0.11	94.26	0.99	0.18
Alaska	93.65	1.22	0.45	91.91	1.03	0.48
Arizona	95.46	0.77	0.31	93.67	1.21	0.26
Arkansas	95.16	0.96	0.15	93.21	1.82	0.27
California–Fresno	94.94	2.19	1.26	93.27	3.10	1.15
California–Los Angeles	94.63	1.78	1.08	94.30	2.19	1.38
California–San Diego	94.74	2.02	1.02	93.78	2.19	1.08
California	94.88	2.26	1.29	93.42	2.09	1.17
Colorado	93.66	1.22	0.45	93.46	0.97	0.31
Connecticut	94.29	1.09	0.63	91.38	1.76	0.52
Delaware	94.34	4.17	0.99	91.59	2.92	0.71
Florida– Hillsborough County	94.92	0.64	0.43	91.85	1.34	1.02
Florida–Miami	95.37	1.57	3.42	94.21	0.73	2.15
Florida	93.98	2.08	1.10	91.72	1.28	0.68
Georgia–Atlanta	95.96	0.99	0.13	92.20	1.02	0.00
Georgia	95.34	4.02	0.98	93.67	3.66	0.21
Hawaii	93.97	1.33	0.79	90.58	1.22	0.90
Idaho	94.99	1.35	0.21	93.64	1.41	0.27
Illinois–Chicago	94.58	1.01	0.95	94.72	0.87	0.98
Illinois	95.13	0.96	0.41	93.76	1.16	0.39
Indiana	94.40	1.97	0.71	93.12	1.75	0.15

Iowa	95.11	0.85	0.23	93.44	1.09	0.22
Kansas	95.07	1.60	0.31	93.42	1.53	0.19
Kentucky– Jefferson County	95.03	3.97	1.71	94.71	3.84	0.75
Kentucky	94.97	2.53	0.66	93.93	3.06	0.29
Louisiana	94.73	1.04	0.12	93.78	1.05	0.19
Maine	93.65	1.60	0.13	92.34	1.30	0.28
Maryland– Baltimore	93.62	13.70	2.37	89.73	15.33	1.14
Maryland	94.40	8.97	5.04	93.77	7.85	1.86
Massachusetts– Boston	94.03	3.08	1.83	93.05	2.09	1.95
Massachusetts	93.77	2.05	0.85	91.82	1.35	1.04
Michigan–Detroit	92.09	5.17	0.93	91.37	5.56	0.27
Michigan	94.64	3.31	0.77	93.66	2.99	0.64
Minnesota	94.93	2.26	0.59	91.30	2.10	0.27
Mississippi	94.99	0.50	0.02	93.72	0.57	0.13
Missouri	95.26	1.18	0.05	92.55	0.86	0.16
Montana	94.40	2.63	0.38	91.61	2.26	0.15
Nebraska	95.83	3.19	0.48	92.32	2.52	0.50
Nevada	95.10	1.38	0.48	92.19	0.89	0.14
New Hampshire	93.45	2.43	0.19	91.46	2.55	0.55
New Jersey	94.87	1.43	0.32	92.01	2.22	0.42
New Mexico– Albuquerque	93.43	0.68	0.11	93.46	1.20	1.19
New Mexico	94.55	0.89	0.32	93.39	1.22	0.89
New York–New York City	92.44	0.65	1.13	91.17	0.58	1.17
New York	93.06	0.88	0.55	90.46	0.48	0.58
North Carolina– Charlotte	94.49	0.58	0.38	92.20	1.06	0.99
North Carolina	94.88	1.56	0.46	92.51	1.43	0.40
North Dakota	96.28	3.81	0.46	94.07	4.00	0.37
Ohio–Cleveland	94.08	4.10	0.73	91.90	2.74	1.01
Ohio	94.58	2.43	0.21	93.08	2.08	0.14
Oklahoma	94.58	1.56	0.34	93.43	1.26	0.16
Oregon	93.98	2.32	0.95	92.62	1.36	0.24
Pennsylvania– Philadelphia	94.61	2.92	1.10	91.35	2.87	1.00
Pennsylvania	94.42	2.13	0.25	91.94	1.34	0.47
Rhode Island	94.78	0.68	0.70	92.96	0.74	0.62
South Carolina	94.64	1.57	0.22	94.03	1.75	0.16
South Dakota	95.69	2.04	0.30	95.01	2.41	0.57
Tennessee	95.34	2.61	0.66	93.54	3.03	0.20
Texas–Austin	94.12	2.77	2.33	88.54	3.07	1.21
Texas–Dallas	96.08	3.48	15.48	93.98	2.38	1.75
Texas–Houston	96.63	2.82	4.69	93.58	2.89	1.60
Texas	95.50	3.21	2.46	93.78	2.85	0.90
Utah	93.71	2.72	0.92	93.00	2.69	0.69
Vermont	95.05	1.07	0.10	92.93	0.59	0.37
Virginia	94.93	1.10	0.58	92.97	1.04	0.36
Washington	93.71	2.43	0.73	91.22	2.09	0.57
West Virginia	93.62	1.67	0.11	93.10	1.79	0.03

Wisconsin– Milwaukee	93.65	3.73	0.55	93.15	3.62	0.72
Wisconsin	94.97	1.43	0.23	94.11	1.44	0.26
Wyoming	94.38	1.11	0.29	93.15	1.05	0.12
Other jurisdictions
Bureau of Indian Education (BIE)	95.63	0.69	0.00	92.93	2.84	0.95
Department of Defense Education Activity (DoDEA)	95.48	5.03	1.41	94.13	3.32	0.87
District of Columbia (TUDA)	94.50	1.56	0.99	90.18	0.94	1.59
District of Columbia	94.46	1.18	0.70	91.33	0.97	0.86
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Reading Assessment.

Similarly, the following table presents the weighted student response and exclusion rates for the twelfth-grade reading assessment.

Weighted student response and exclusion rates, grade 12 state reading assessment, by jurisdiction: 2013

Jurisdiction	Weighted student response rates (percent)	Weighted percentage of all students who were SD and excluded	Weighted percentage of all students who were ELL and excluded
Arkansas	90.21	2.49	0.20
Connecticut	79.77	2.28	0.27
Florida	77.34	2.97	0.70
Idaho	88.68	1.55	0.17
Illinois	83.72	2.23	0.20
Iowa	84.26	1.41	0.13
Massachusetts	79.84	1.65	0.43
Michigan	87.21	3.96	0.14
New Hampshire	76.91	2.15	0.44
New Jersey	84.67	1.61	0.18
South Dakota	86.17	1.60	0.06
Tennessee	88.82	2.86	0.08
West Virginia	84.28	2.37	0.00
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Reading Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation Sample Design for the 2013 National Assessment

The 2013 national assessment included mathematics and reading assessments in public and private schools at grades 4, 8, and 12.

The sample designs aimed to achieve nationally representative samples of students in the defined populations who were enrolled at the time of assessment.

The samples were based on a two-stage sample design: selection of schools within strata; and

selection of students within schools.

Fourth- and Eighth-Grade Public School National Assessments

Twelfth-Grade Public School National Assessment

Private School National Assessment

The samples of schools were selected with probability proportional to a measure of size based on the estimated grade-specific enrollment in the schools.

For fourth- and eighth-grade public schools, the NAEP state student samples and assessments constitute the NAEP national student samples and assessments.

For the twelfth-grade public schools, the national sample consisted of 13 state samples and an additional sample that represented the remaining 37 states and the District of Columbia.

Nationally representative samples were also drawn for the private school students in grades 4, 8, and 12.

NAEP Technical Documentation Website

NAEP Technical Documentation 2013 Fourth- and Eighth-Grade Public School National Assessment

For the mathematics and reading assessments in fourth- and eighth-grade public schools, the national samples were the state assessment samples for each jurisdiction. All jurisdictions participated in the mathematics and reading assessments, with the exception of Puerto Rico, where a special mathematics assessment was conducted instead of the operational mathematics and reading assessments. Also, Bureau of Indian Education (BIE) was not designed as a reportable jurisdiction for the 2013 state assessments, but a nationally representative sample of students in BIE schools was selected.

NAEP Technical Documentation Website

NAEP Technical Documentation 2013 Twelfth-Grade Public School National Assessment

The twelfth-grade public school sample for the NAEP 2013 study was designed to achieve a nationally representative sample of twelfth-grade students enrolled in public schools in the United States. The sample was also designed to achieve state-level representative samples in 13 specific states. These states were Arkansas, Connecticut, Florida, Idaho, Illinois, Iowa, Massachusetts, Michigan, New Hampshire, New Jersey, South Dakota, Tennessee, and West Virginia.

The target sample size of assessed students for the twelfth- grade public school sample was 80,000 assessed

students: 4,600 students in each of the 13 state-assessment states (approximately 60,000 students combined) and 20,000 students from the the remaining 37 states, the District of Columbia, the Bureau of Indian Education (BIE) schools,

and Department of Defense Education Activity

Target Population Sampling Frame Stratification of Schools School Sample Selection Substitute Schools Ineligible Schools Student Sample Selection

School and Student Participation

(DoDEA) schools located within the 50 states and the District of Columbia. Prior to sampling, the target sample sizes were adjusted upward to offset expected school and student attrition due to nonresponse and ineligibility.

The twelfth-grade public school sample was based on a two-stage design that involved selection of schools within strata and selection of students within schools. The first-stage sample of schools was selected with probability proportional to a measure of size based on estimated grade 12 student enrollment in the schools.

The students in the twelfth-grade public school sample were assessed in mathematics and reading.

NAEP Technical Documentation Target Population for the 2013 Twelfth-Grade Public School National Assessment

The target population for the 2013 twelfth-grade public school national assessment included all students who were enrolled in twelfth-grade public schools in the 50 states and the District of Columbia. The sample included Bureau of Indian Education (BIE) schools and Department of Defense Education Activity (DoDEA) schools located within the 50 states and the District of Columbia.

NAEP Technical Documentation Sampling Frame for the 2013 Twelfth-Grade Public School National Assessment

As with the NAEP state sample, the sampling frame for public schools was derived from the Common Core of Data (CCD) file corresponding to the 2009-2010 school year. The CCD files provided the frame for all regular public, state-operated public, Bureau of Indian Education (BIE), and Department of Defense Education Activity (DoDEA) schools open during the 2009-2010 school year.

The sampling frame excluded ungraded schools, vocational schools with no enrollment, special education-only schools, prison and hospital schools, home

Twelfth-Grade Schools and Enrollment in Public School Sampling Frame

New-School Sampling Frame for the National Assessment

school entities, virtual or online schools, adult and evening schools, and juvenile correctional institutions.

NAEP Technical Documentation Twelfth-Grade Schools and Enrollment in the 2013 Public School Sampling Frame

The following table presents the number of schools and estimated enrollment for the twelfth-grade Common Core of Data (CCD) frame by sampling stratum.

NAEP twelfth-grade Common Core of Data (CCD) frame public school enrollment and counts, national assessment, by sampling stratum: 2013

Sampling Stratum	Schools	Percent	Estimated enrollment	Percent
Total	23,433	100.00	3,476,820	100.00
Arkansas	297	1.27	32,035	0.92
Connecticut	245	1.05	41,607	1.20
Florida	965	4.12	176,821	5.09
Idaho	210	0.90	19,057	0.55
Illinois	954	4.07	149,998	4.31
Iowa	408	1.74	37,793	1.09
Massachusetts	371	1.58	67,923	1.95
Michigan	1,032	4.40	126,382	3.63
New Hampshire	89	0.38	15,749	0.45
New Jersey	432	1.84	97,690	2.81
South Dakota	191	0.82	8,796	0.25
Tennessee	369	1.57	67,111	1.93
West Virginia	147	0.63	18,796	0.54
Remainder	17,723	75.63	2,617,062	75.27
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Assessment.

NAEP Technical Documentation New-School Sampling Frame for the 2013 Twelfth-Grade Public School National Assessment

The Common Core of Data (CCD) file used for the CCD-based sampling frame corresponds to the 2009-2010 school year, whereas the assessment year is the 2012-2013 school year. During this 3-year period, some schools closed, some changed structure (one school becoming two schools, for example), and others came into existence.

To achieve as close to full coverage as possible, the CCD-based school sampling frame was supplemented by a sample of new schools obtained from a sample of districts. Each sampled district was sent a list of the CCD schools and asked to add in any new schools or old schools that had become newly eligible for grades 4, 8, or 12.

Since asking every school district to list new and newly-eligible schools would have generated too much of a burden, a sample of districts was contacted to obtain a list of new schools. To represent the unsampled districts in the full sample of schools, weights for schools included in the new-school sample were adjusted to reflect the district selection probability. This was done for fourth-, eighth-, and twelfth-grade schools in one step, and this step is described in the new-school frame.

The following table presents the number of schools and estimated enrollment for the twelfth-grade new school frame by sampling stratum.

NAEP twelfth-grade new school frame for the public school national assessment: school counts and estimated enrollment by sampling stratum: 2013

Sampling Stratum	Schools	Percent	Estimated enrollment	Percent
Total	398	100.00	21,953	100.00
Arkansas	2	0.50	64	0.29
Connecticut	0	0.00	0	0.00
Florida	62	15.58	2,440	11.11
Idaho	1	0.25	10	0.05
Illinois	4	1.01	177	0.81
Iowa	0	0.00	0	0.00
Massachusetts	2	0.50	187	0.85
Michigan	2	0.50	53	0.24
New Hampshire	0	0.00	0	0.00
New Jersey	4	1.01	135	0.61
South Dakota	0	0.00	0	0.00
Tennessee	3	0.75	199	0.91
West Virginia	0	0.00	0	0.00
Remainder	318	79.90	18,688	85.13
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Assessment.

NAEP Technical Documentation Stratification of Schools for the 2013 Twelfth-Grade Public School National Assessment

Prior to sampling, the twelfth-grade public school frame was stratified to increase the efficiency and ensure the representativeness of the school sample in terms of important school-level characteristics, such as geography (e.g., states), urbanicity, and race/ethnicity composition. The school frame was stratified using two types of stratification, explicit and implicit.

Explicit stratification partitions the sampling frame into mutually exclusive groupings called sampling strata. The systematic samples selected from these strata are independent, meaning that each is selected with its own unique random start. The twelfth-grade public school sample had 14 sampling strata: one for each of the 13 states with state assessments and an additional stratum representing the remainder of the frame (schools in the remaining 37 states, District of Columbia, and all BIE and DoDEA schools).

Implicit stratification involves sorting the sampling frame, as opposed to grouping the frame. For NAEP, schools are sorted in serpentine fashion by key school characteristics within sampling strata and sampled systematically using this ordering. This type of stratification ensures the representativeness of the school samples with respect to the key school characteristics.

Schools in each state stratum were implicitly stratified by urbanicity classification, race/ethnicity classification, and median income, similarly to the grades 4 and 8 public school samples. (See stratification of schools of grades 4 and 8 public school samples for details.)

Schools in the remainder stratum were implicitly stratified by: census division;

urbanicity classification; race/ethnicity classification;

school type (public, BIE, DoDEA); and median income.

The New England and Mid-Atlantic census divisions were collapsed into a single implicit stratum comprising the census region Northeast, as Connecticut, Massachusetts, New Hampshire, and New Jersey were all in the twelfth-grade public school state assessment. The remaining census divisions were not collapsed.

The urbanicity classification strata were derived from the NCES urban-centric locale variable from the Common Core of Data (CCD), which classifies schools based on location (city, suburb, town, rural) and proximity to urbanized areas. Urban-centric locale has 12 possible values.

The urbanicity classification cells were created by starting with the original 12 NCES urban-centric locale categories within each census division stratum. Any cell with an expected school sample size less than four was combined with a neighboring cell within the same census division stratum. Collapsing was first done among the subcategories within a location class. (For example, the subcategories for location class city are 1:large, 2:mid-size, and 3:small. If one of these subcategories was deficient then either 1:large was collapsed with 2:mid-size; 3:small collapsed with 2:mid-size; or 2:mid-size collapsed with the smaller of 1:large or 3:small.) If the collapsed cell was still too small, all three subcategories within a location class were combined.

If a collapsed location class still had an expected school sample size less than four, then it was collapsed with a neighboring collapsed location class. That is, 1:city would be collapsed with 2:suburb or 3:town would be collapsed with 4:rural. If additional collapsing was necessary all location classes were combined. No collapsing across census division strata was allowed or necessary.

The final result of this was a set of census division-urbanicity strata with all strata having expected school sample sizes of at least four schools.

Schools within the urbanicity classification strata were further stratified into race/ethnicity classification strata. The first division was a dichotomization of each urbanicity stratum into a low and a high Black/Hispanic stratum (the cutoff was 15 percent Black and Hispanic students). If the expected school sample size of resultant strata was less than or equal to 8.0, then this was the final urbanicity-race/ethnicity stratum. If the expected school sample size exceeded 8.0, a further division was made.

For the low Black/Hispanic stratum, there were only five urbanicity strata that had a large enough expected school sample size, and these were dichotomized by state. The table below describes the dichotomization.

Strata for low race/ethnicity strata with expected school sample sizes greater than 8: 2013

Census division stratum	Urbanicity stratum	Group 1 states	Group 2 states
East North Central division	Rural Fringe	Indiana, Wisconsin	Ohio
West North Central division	Rural Distant	Iowa, Kansas, Missouri, Nebraska	North Dakota, South Dakota
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Assessment.

Within the high Black/Hispanic stratum, the number of substrata was based on the expected school sample size. If the expected sample size was between 8.0 and 12.0, there were two substrata; if the expected sample size was between 12.0 and 16.0, there were three substrata; and if the expected sample size was over 16.0, there were four substrata.

The substrata were defined by percent Black and Hispanic students, with the cutoffs for substrata defined by weighted percentiles (with the weight equal to expected hits for each school). For two substrata, the cutoff was the weighted median; for three substrata, the weighted 33rd and 67th percentiles; for four substrata, the weighted median and quartiles.

The implicit stratification within these census division-urbanicity-race/ethnicity status strata was based on school type (public, BIE, DoDEA) and median income of the ZIP code area containing the school.

NAEP Technical Documentation Website

NAEP Technical Documentation Sampling of Schools for the 2013 Twelfth-Grade Public School National Assessment

For the twelfth-grade public school assessment sample, schools were sampled independently from each sampling stratum with probability proportional to size using systematic sampling. Prior to sampling, schools in each sampling stratum

were sorted by the appropriate implicit stratification variables in a serpentine order. A school's measure of size was a complex function of the school's estimated grade enrollment. As with the grades 4 and 8 public school state assessment samples, multiple hits were allowed for each school in the state-based

Computation of Measures of Size

School Sample Sizes: Frame and New School

sampling strata, but not in the remainder stratum containing the schools in the remaining states and District of Columbia.

The sampled schools for the twelfth-grade public school assessment came from two frames: the public school sample frame (as constructed from the Common Core of Data (CCD) and the new-school sampling frame.

For the CCD-based school frame, schools in the state-based sampling strata were sampled at a rate that would

yield 4,600 assessed students per stratum. Schools in the remainder stratum were sampled at a rate that would yield a national sample of 26,100 assessed students.

The schools in the new school frame were sampled at the same rate as the CCD-based school frame.

NAEP Technical Documentation Website

NAEP Technical Documentation Computation of Measures of Size for the 2013 Twelfth-Grade Public School National Assessment

In designing the twelfth-grade public school sample, six objectives underlie the process of determining the probability of selection for each school and the number of students to be sampled from each selected school:

to meet the target student sample size for each explicit sampling stratum;

to select an equal-probability sample of students from each explicit sampling stratum; to limit the number of students selected from any one school;

to ensure that the sample within a school does not include a very high percentage of the students in the school, unless all students are included;

to reduce the rate of sampling of small schools, in recognition of the greater cost and burden per student of conducting assessments in such schools; and

to increase the number of Black and Hispanic students in the sample.

The goal in determining the school's measure of size is to optimize across the middle four objectives in terms of maintaining the accuracy of estimates and the cost effectiveness of the sample design.

To increase the number of Black and Hispanic students in the sample, the measure of size for schools with relatively high proportions of Black and Hispanic students (15 percent or more) were doubled. This oversampling was limited to only the remainder stratum, (that is, the stratum comprising schools that are not in states with state assessments). The target student sample sizes for the state-based strata are large (4,600 assessed) and should yield a sufficient number of Black and Hispanic students.

For schools with high proportions of Black and Hispanic students in the remainder stratum, the preliminary measures of size (MOS) were calculated as follows:

where x_js is the estimated grade 12 student enrollment for school s in stratum j.

For all other schools (those in the state-based strata or with a low proportion of Black and Hispanic students in the remainder stratum), the preliminary measures of size (MOS) were calculated set as follows:.

where x_js is the estimated grade 12 student enrollment for school s in stratum j.

The preliminary school measure of size was rescaled to create an expected number of hits by applying a multiplicative constant b_j, which varies by stratum j. The design for the twelfth grade school sample allowed multiple hits. For example, a school with two hits will have twice as many students sampled as a single-hit school. To limit respondent burden, constraints were placed on the number of hits allowed per school. For schools in the state-based sampling strata, the limit was three hits. For schools in the remainder stratum, it was one hit.

It follows that the final measure of size, E_js, was defined as:

where u_j is the maximum number of hits allowed.

In addition, new and newly-eligible schools were sampled from the new-school frame. The assigned measures of size for these schools,

,

used the b_j and u_j values from the CCD-based school frame for stratum j (i.e., the same sampling rate as for the CCD-based school sample within each stratum). The variable π_djs is the probability of selection of the district into the new-school district (d) sample.

In addition, an adjustment was made to the initial measures of size in an attempt to reduce school burden by minimizing the number of schools selected for both the High School Longitudinal Study (HSLS) and the grade 12 public school NAEP assessments. The NAEP sampling procedures used an adaptation of the Keyfitz process to compute conditional measures of size that, by design, minimized the overlap of schools selected for both the NAEP and HSLS assessments.

NAEP Technical Documentation Website

NAEP Technical Documentation School Sample Sizes: List Frame- Based and New School for the 2013 Twelfth-Grade Public School National Assessment

The following table presents the number of schools selected for the twelfth-grade public school sample by sampling frame (Common Core of Data (CCD) and new school ) and sampling stratum.

NAEP public school sample counts for grade 12 national assessment, by sampling stratum and sampling frame (CCD, new school): 2013

State	Total school sample	CCD-based school frame sample	New-school frame sample
Total	2,030	2,020	10
Arkansas	100	100	0
Connecticut	110	110	0
Florida	120	120	#
Idaho	100	100	0
Illinois	130	130	0
Iowa	120	120	0
Massachusetts	110	110	#
Michigan	140	140	#
New Hampshire	80	80	0
New Jersey	110	110	#
South Dakota	140	140	0
Tennessee	130	130	0
West Virginia	90	90	0
Remainder	570	560	10
# Rounds to zero. NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 State Assessment.

NAEP Technical Documentation Substitute Schools for the 2013 Twelfth-Grade Public School National Assessment

Though efforts were made to secure the participation of all schools selected, it was anticipated that not all schools would choose to participate. NAEP uses school substitution to mitigate the effect of bias due to nonresponse. A nonparticipating sampled school is replaced by its substitute when the original school is considered a final refusal.

For the twelfth-grade public school sample, substitute schools were preselected for all sampled schools by sorting the school frame file according to a sort order very close to that used in sample selection (the implicit stratification). The two exceptions to this were as follows: (1) estimated grade enrollment replaces median income as the last sort variable, and (2) school type in the stratification hierarchy was crossed with state (rather than used alone) in the stratum comprising the remaining states and the District of Columbia. The first change guaranteed that the selected substitute would have a grade enrollment very close to that of the originally selected school. The second change guaranteed that any selected substitutes would be within the same state as the originally sampled nonresponding school.

The two candidates for substitutes were then the two nearest neighbors of the originally sampled school on this revised sort order. To be eligible as a potential substitute, the neighbor needed to be a nonsampled school (for any grade) and within the same sampling stratum. If both nearest neighbors were eligible to be substitutes, the one with a closer grade enrollment was chosen.

Nationally, 11 substitutes ultimately participated in the twelfth-grade public school sample.

NAEP Technical Documentation Eligibility Status

of Schools for the 2013 Twelfth-Grade Public School National Assessment

The Common Core of Data (CCD) public school frame from which most of the sampled schools were drawn corresponds to the 2009-2010 school year, some three years prior to the assessment school year. During the intervening period, some of these schools either closed, no longer offered grade 12, or were ineligible for other reasons. In such cases, the sampled schools were considered to be ineligible.

The table below presents unweighted counts of ineligible schools and their eligibility, by status, for the twelfth-grade public school sample.

NAEP twelfth-grade sample public schools, national assessment, by eligibility status: 2013

Eligibility status	Unweighted count of schools	Unweighted percent
Total	2,030	100.00
Eligible	1,940	95.57
Has sampled grade, but no eligible students	7	0.34
Does not have sampled grade	15	0.74
Closed	22	1.08
Not a regular school	37	1.82
Duplicate on sampling frame	0	0.00
Other ineligible	13	0.64
NOTE: Numbers of schools are rounded to nearest ten, except those pertaining to ineligible schools. Detail may not sum to totals due to rounding. Percentages are based on rounded counts. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Assessment.

NAEP Technical Documentation Student Sample Selection for the 2013 Twelfth-Grade Public School National Assessment

The target student sample size within the sampled schools for the twelfth-grade public sample was 60 students. However, schools with 66 or fewer students automatically had all students sampled.

There was only one spiral type for the twelfth-grade public school sample. In the spiral, 51.5 percent of the booklets were reading and 48.5 percent were mathematics.

The process of list submission, sampling students from year-round schools, sampling new enrollees, and determining student eligibility and exclusion status was the same as the process used for the NAEP state student sample.

NAEP Technical Documentation School and Student Participation in the 2013 Twelfth-Grade Public School National Assessment

Twelfth-grade public school participation in NAEP is not mandatory. Although a small portion of the participating school sample consisted of substitute schools, it is preferable when calculating school response rates to do so on the basis of school participation before substitution.

In every NAEP administration, some of the sampled students are not assessed for the following reasons:

withdrawn students;

excluded students with disabilities (SD); excluded English language learners (ELL); or

Weighted School Response Rates

Weighted Student Response and Exclusion Rates for Mathematics

Weighted Student Response and Exclusion Rates for Reading

students absent from both the original session and the make-up session (not excluded but not assessed).

Withdrawn students are those who have left the school before the original assessment.

Excluded students were determined by their school to be unable to meaningfully take the NAEP assessment in their assigned subject, even with an accommodation. Excluded students must also be classified as SD and/or ELL. Other students who were absent for the initial session are assessed in the makeup session. The last category includes students who were not excluded (i.e., “were to be assessed”) but were not assessed either due to absence from both sessions or because of a refusal to participate. Assessed students are also classified as assessed without an accommodation or assessed with an accommodation. The latter group can be divided into SD students assessed with an accommodation, ELL students assessed with an accommodation, or students who are both SD and ELL and accommodated. Note that some SD and ELL students are assessed without accommodations, and students who are neither SD nor ELL can only be assessed without an accommodation.

NAEP Technical Documentation Weighted School Response Rates for the 2013 Twelfth-Grade Public School National Assessment

The following table presents unweighted counts of eligible sampled schools and participating schools, as well as weighted school response rates, for the twelfth-grade public school samples in which the 2013 mathematics and reading assessments were conducted. The weighted school response rates estimate the proportion of the student population that is represented by the participating school sample prior to substitution.

School response counts and rates before substitution, twelfth-grade public schools, national assessment, by region: 2013

Region	Number of sample eligible schools	Number of participating schools	Weighted school response rate prior to substitution (percent)
National	1,940	1,880	92.80
Northeast	460	450	94.59
Midwest	580	560	90.22
South	620	600	91.65
West	270	270	96.17
NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Assessment.

The following table presents the weighted student response and exclusion rates for the 2013 mathematics assessment for twelfth-grade public schools. The exclusion rates give the percentage excluded, among all eligible students. Excluded students must necessarily be either students with disabilities (SD) or English language learners (ELL). The response rates indicate the percentage of students assessed among those who it was intended would take the assessment within the participating schools. Thus, students who were excluded are not included in the denominators of the response rates.

Weighted student response and exclusion rates for twelfth-grade public schools, national mathematics assessment, by region: 2013

Region	Weighted student response rates (percent)	Weighted percentage of all students who were SD and excluded	Weighted percentage of all students who were ELL and excluded
National	84.17	2.22	0.23
Northeast	81.09	2.15	0.26
Midwest	84.02	1.74	0.11
South	86.34	2.45	0.10
West	83.34	2.35	0.51
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Mathematics Assessment.

The following table presents the weighted student response and exclusion rates for the 2013 reading assessment for twelfth-grade public schools. The exclusion rates give the percentage excluded, among all eligible students. Excluded students must necessarily be either students with disabilities (SD) or English language learners (ELL). The response rates indicate the percentage of students assessed among those who it was intended would take the assessment within the participating schools. Thus, students who were excluded are not included in the denominators of the response rates.

Weighted student response and exclusion rates for twelfth-grade public schools, national reading assessment, by region: 2013

Region	Weighted student response rates (percent)	Weighted percentage of all students who were SD and excluded	Weighted percentage of all students who were ELL and excluded
National	83.77	2.38	0.33
Northeast	80.11	1.84	0.31
Midwest	84.17	2.14	0.17
South	85.50	2.91	0.27
West	83.58	2.17	0.60
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Reading Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation2013 Private School National Assessment

The private school samples were designed

to produce nationally representative samples of students enrolled in private schools in the United States. Fourth-, eighth-, and twelfth-grade students were assessed in mathematics and reading.

Mathematics and reading pilots, a Knowledge and Skills Appropriate (KaSA) Study, and a Reading accessible booklet study were also conducted in the private school samples for fourth- and eighth-grade.

The target sample sizes of assessed students for each grade and subject are shown in the table below. Prior to sampling, these target sample sizes were adjusted upward to offset expected rates of school and student attrition due to nonresponse and ineligibility.

Target Population Sampling Frame Stratification of Schools School Sample Selection Substitute Schools Ineligible Schools Student Sample Selection

School and Student Participation

Samples were based on a two-stage design that involved selection of schools within strata and selection of students within schools. The first-stage samples of

schools were selected with probability proportional to a measure of size based on the estimated grade-specific enrollment in the schools.

Target sample sizes of assessed students, private school national assessment, by subject and grade: 2013

Grade	Total	Mathematics	Reading	Pilot/Special Studies
Total	15,730	7,400	7,500	830
4	6,335	3,000	3,000	335
8	6,495	3,000	3,000	495
12	2,900	1,400	1,500	†
† Not applicable. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Assessment.

NAEP Technical Documentation Target Population for the 2013 Private School National Assessment

The target population for the 2013 Private School National Assessment included all students enrolled in private schools in grades 4, 8, and 12 within the 50 states and the District of Columbia.

NAEP Technical Documentation Sampling Frame for the 2013 Private School National Assessment

The frame of the private schools in all three grades was developed from the 2009-2010 Private School Universe Survey (PSS), a survey conducted by the U.S. Census Bureau for the National Center for Education Statistics (NCES). The PSS is a biennial mail survey of all private schools in the 50 states and the District of Columbia. The PSS frame of schools comprises both a list frame and an area frame. The 2009-2010 list frame is an assembly of the 2007-2008 PSS

Fourth-, Eighth-, and Twelfth-Grade Schools and Enrollment in the Private School Sampling Frame

New-School Sampling Frame for the Private School Assessment

frame and more up-to-date lists from state education agencies, private school associations, and other easily accessible sources. To improve the coverage of the PSS list frame, the Census Bureau also conducted a survey to locate private schools in a random sample of geographic areas throughout the United States. The areas were single counties or groups of counties sampled from an area frame constructed from all counties in the nation. Within each selected area a complete list of private schools was gathered using information from the Yellow Pages, religious institutions, local education agencies, chambers of commerce, and local government offices. Schools not already on the list frame were identified and added to the frame of private schools. A weighting component was computed by the Census Bureau so that the additional area-frame schools would represent all schools absent from the list frame, not just those in the selected areas.

The sampling frame excluded schools that were ungraded, provided only special education, were part of hospital or treatment center programs, were juvenile correctional institutions, were home-school entities, or were for adult education.

Private school affiliation is unknown for nonrespondents to the PSS. Because NAEP response rates differ vastly by affiliation, to better estimate the target sample size of schools for each affiliation, additional work was done to obtain affiliation for nonrespondents to the PSS. If a nonresponding school responded to a previous PSS (either two or four years prior), affiliation was obtained from the previous response. For those schools that were nonrespondents for the last three cycles of the PSS, in some cases Internet research was used to establish affiliation. There were still schools with unknown affiliation remaining after this process.

For quality control purposes, school and student counts from the 2013 sampling frame were compared to school and student counts from previous NAEP frames (2011 and 2009). No major issues were found.

NAEP Technical Documentation Website

NAEP Technical Documentation Fourth-, Eighth-, and Twelfth-Grade Schools and Enrollment in the 2013 Private School Sampling Frame

The following table displays, by grade and affiliation, the number of private schools in the sampling frame and their estimated enrollment. For grades 4 and 8, enrollment was estimated for each school as the Private School Universe Survey (PSS)-reported enrollment averaged across grades 1 through 8. For grade 12, the average was computed over grades 9 through 12.

The counts presented below are of schools with known affiliation. Schools with unknown affiliation do not appear in the table because their grade span, affiliation, and enrollment were unknown. Although PSS is a school universe survey, participation is voluntary and not all private schools respond. Since the NAEP sample must represent all private schools, not just PSS respondents, a small sample of PSS nonrespondents with unknown affiliation was selected for each of the targeted grades to improve NAEP coverage.

Number of schools and enrollment in private school sampling frame, national assessment, by affiliation and grade: 2013

Grade	Affiliation	Number of schools	Estimated enrollment
4	Total	19,553	354,543
	Catholic	5,669	156,505
	Non-Catholic private	13,884	198,038
8	Total	17,607	342,303
	Catholic	5,214	156,583
	Non-Catholic private	12,393	185,720
12	Total	9,138	317,449
	Catholic	1,295	150,454
	Non-Catholic private	7,843	166,995
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Assessment.

NAEP Technical Documentation Website

NAEP Technical Documentation New-School Sampling Frame for the 2013 Private School National Assessment

Whereas the Private School Universe Survey (PSS) file used for the frame corresponds to the 2009-2010 school year, the NAEP assessment year was the 2012-2013 school year. During this 3-year period, some schools closed, some changed their grade span, and still others came into existence.

To achieve as close to full coverage as possible, the private school frame was supplemented by a sample of new Catholic schools. The goal was to allow every such school a chance of selection, thereby fully covering the target population of Catholic schools in operation during the 2012-2013 school year. The first step in this process was the development of a new-school frame through the construction of a diocesan-level file from the PSS school-level file. To develop the frame, the diocesan-level file was divided into two files: one for small dioceses and the other for medium and large dioceses.

Small dioceses contained no more than three schools on the frame in total, with no more than one school at each grade (fourth, eighth, and twelfth). New schools in small dioceses were identified during school recruitment and added to the sample if the old school in the same diocese was sampled at the relevant grade. From a sampling perspective, the new school was viewed as an “annex” to the sampled school that had a well-defined probability of selection equal to that of the old school. The “frame” in this case was, in fact, the original frame; when the old school was sampled in a small diocese, the new school was automatically sampled as well.

To limit respondent burden and keep the level of effort within reasonable bounds, the new-school frame was created using information obtained from a sample of the remaining dioceses. The remaining dioceses were separated into two strata of large- and medium-size dioceses. These strata were defined by computing the percentage of the nation’s total Catholic school enrollment each diocese represents, sorting the dioceses in descending order by that percentage, and cumulating the percentages across the sorted file. All dioceses up to and including the first diocese at or above the 80th cumulative percentage were defined as large dioceses. The remaining dioceses were defined as medium dioceses.

A simplified example is given below. Dioceses are ordered by percentage enrollment. The first six become large dioceses and the last six become medium dioceses.

Example showing assignment of Catholic dioceses to the large and medium strata, private school national assessment: 2013

Diocese	Percent enrollment	Cumulative percentage enrollment	Stratum
Diocese 1	20	20	L
Diocese 2	20	40	L
Diocese 3	15	55	L
Diocese 4	10	65	L
Diocese 5	10	75	L
Diocese 6	10	85	L
Diocese 7	5	90	M
Diocese 8	2	92	M
Diocese 9	2	94	M
Diocese 10	2	96	M
Diocese 11	2	98	M

Diocese 12	2	100	M
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Assessment.

In actuality, there were 72 large and 102 medium dioceses in the sampling frame.

The target sample size was 10 dioceses total: 8 large and 2 medium. In the medium stratum, the dioceses were selected with equal probability. In the large stratum, dioceses were sampled with probability proportional to enrollment. These probabilities were retained and used in all later stages of sampling and weighting in order to represent all dioceses, whether or not they had been selected as new school samples for the assessment.

Each selected diocese was sent a listing of its schools extracted from the 2009-2010 PSS file and was asked to provide information about new schools and any changes to grade span in existing schools. This information provided by the selected dioceses was used to create sampling frames for the selection of new Catholic schools. The process of obtaining the information was conducted with the help of the National Catholic Educational Association (NCEA).

NCEA was sent the school lists for the 10 sampled dioceses and was responsible for returning the completed updates.

The eligibility of a new school at a particular grade was determined by its grade span. A school already on PSS also was classified as “new” if a change of grade span had occurred such that the school status changed from ineligible to eligible at a particular grade.

NAEP Technical Documentation Stratification of Schools for the 2013 Private School National Assessment

Explicit stratification for the NAEP 2013 private school samples was by private school type: Catholic, non- Catholic, and unknown affiliation. Private school affiliation was unknown for nonrespondents to the NCES Private School Universe Survey (PSS) for the past three cycles.

The implicit stratification of the schools involved four dimensions. Within each explicit stratum, the private schools were hierarchically sorted by census region, urbanicity status, race/ethnicity status, and estimated grade enrollment. The implicit stratification in this four-fold hierarchical stratification was achieved via a "serpentine sort".

Census region was used as the first level of implicit stratification for the NAEP 2013 private school sample. All four census regions were used as strata.

The next level of stratification was an urbanicity classification based on urban-centric locale, as specified on the PSS. Within a census region-based stratum, urban-centric locale cells that were too small were collapsed. The criterion for adequacy was that the cell had to have an expected school sample size of at least six.

The urbanicity variable was equal to the original urban-centric locale if no collapsing was necessary to cover an inadequate original cell. If collapsing was necessary, the scheme was to first collapse within the four major strata (city, suburbs, town, and rural). For example, if the expected number of large city schools sampled was less than six, large city was collapsed with midsize city. If the collapsed cell was still inadequate, they were further collapsed with small city. If a major stratum cell (all three cells collapsed together) was still deficient, it was collapsed with a neighboring major stratum cell. For example, city would be collapsed with suburbs.

The last stage of stratification was a division of the geographic/urbanicity strata into race/ethnicity strata if the expected number of schools sampled was large enough (i.e., at least equal to 12). This was done by deciding first on the number of race/ethnicity strata and then dividing the geography/urbanicity stratum into that many pieces. The school frame was sorted by the percentage of students in each school who were Black, Hispanic, or American Indian. The three racial/ethnic groups defining the race/ethnicity strata were those that have historically performed substantially lower on NAEP assessments than White students. The sorted list was then divided into pieces, with roughly an equal expected number of sampled schools in each piece.

Finally, schools were sorted within stratification cells by estimated grade enrollment.

NAEP Technical Documentation Sampling of Schools for the 2013 Private School National Assessment

The private school samples were selected with probability proportional to size using systematic sampling from a sorted list. A school's measure of size was a complex function of the school's estimated grade enrollment. For all

three grades, only one "hit" was allowed per school.

Schools were ordered within each school type using a serpentine sort involving the following variables:

census region;

urbanicity classification (based on urban-centric locale); race/ethnicity status; and

estimated grade enrollment.

Computation of Measures of Size

School Sample Sizes: Frame and New School

A systematic sample was then drawn with probability proportional to size using this serpentine sorted list and the measures of size.

Schools with unknown affiliation were treated separately. A sample of about 30 schools with unknown affiliation was selected at each of the three grades.

NAEP Technical Documentation Website

NAEP Technical Documentation School Sample Sizes: List Frame-Based and New School for the 2013 Private School National Assessment

The following table presents the number of schools selected from the private school sampling frame (constructed from the Private School Universe Survey file) and the new-school sampling frame, for grades 4, 8, and 12, by school type.

NAEP private school national assessment frame-based and new school samples, by grade and school type: 2013

Grade and private school type	Total school sample	Frame school sample	New school sample
Grade 4
All private	410	410	#
Catholic	130	130	#
Non-Catholic private	250	250	0
Unknown affiliation	30	30	0
Grade 8
All private	400	390	10
Catholic	130	130	10
Non-Catholic private	240	240	0
Unknown affiliation	30	30	0
Grade 12
All private	160	160	#
Catholic	40	40	#
Non-Catholic private	100	100	0
Unknown affiliation	25	30	0
# Rounds to zero. NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Assessment.

NAEP Technical Documentation Substitute Schools for the 2013 Private School National Assessment

Substitutes were preselected for the private school samples by sorting the school frame file according to the actual order used in the sampling process (the implicit stratification). Each sampled school had its two nearest neighbors on the school frame file identified as potential substitutes. As the last sort ordering was by grade enrollment, the nearest neighbors had grade enrollment values very close to that of the sampled school.

Schools were disqualified as potential substitutes if they were already selected in the private school sample or assigned as a substitute for another private school (earlier in the sort ordering). Schools assigned as substitutes for twelfth-grade schools were disqualified as potential substitutes for fourth- and eighth-grade schools, and schools assigned as substitutes for eighth-grade schools were disqualified as potential substitutes for fourth-grade schools.

If both nearest neighbors were still eligible to be substitutes, the one with the closer grade enrollment was chosen. If both nearest neighbors had the same grade enrollment (an uncommon occurrence), one of the two was randomly selected.

In the process described above, only schools with the same affiliation were selected as substitutes.

NAEP Technical Documentation Ineligible Schools for the 2013 Private School National Assessment

The Private School Universe Survey (PSS) school file, from which most of the sampled schools were drawn, corresponds to the 2009-2010 school year, 3 years prior to the assessment school year. During the intervening period, some of these schools either closed, no longer offered the grade of interest, or were ineligible for other reasons. In such cases, the sampled schools were coded as ineligible.

Eligibility Status of Sampled Schools by Grade and Private School Type

Ineligible Sampled Schools by Ineligibility Type

NAEP Technical Documentation Website

NAEP Technical Documentation Eligibility Status of Sampled Schools for the 2013 Private School National Assessment

The following table presents a breakdown by private school type of ineligible and eligible schools in the fourth-, eighth-, and twelfth-grade private school samples. There are considerable differences across private school types at grades 4, 8, and 12. Schools whose private school type was unknown at the time of sampling subsequently had their affiliation determined during data collection. Therefore, such schools are not broken out separately.

Eligibility status of sampled private schools, national assessment, by grade and private school type: 2013

Private school type	Eligibility status	Fourth grade		Eighth grade		Twelfth grade
Count	Percentage	Count	Percentage	Count	Percentage
All private	Total	410	100.00	400	100.00	160	100.00
	Ineligible	60	14.63	70	17.50	40	25.00
	Eligible	350	85.37	330	82.50	120	75.00
Roman Catholic	Total	130	100.00	130	100.00	40	100.00
	Ineligible	10	7.69	10	7.69	0	0.00
	Eligible	130	100.00	120	92.31	40	100.00
Other private	Total	280	100.00	270	100.00	120	100.00
	Ineligible	60	21.43	70	25.93	40	33.33
	Eligible	220	78.57	200	74.07	80	66.67
NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. Percentages are based on rounded counts. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Assessment.

NAEP Technical Documentation Sampled Schools for the 2013 Private School National Assessment

The table below presents unweighted counts of sampled schools, by grade and eligibility status, for the private school samples.

NAEP sample private schools, national assessment, by grade and eligibility status: 2013

Grade and eligibility status	Unweighted count of schools	Unweighted percentage
All fourth-grade sampled private schools	410	100.00
Eligible	350	85.37
Has sampled grade, but no eligible students	15	3.66
Does not have sampled grade	11	2.68
Closed	22	5.37
Not a regular school	12	2.93
Duplicate on sampling frame	0	0.00
Other ineligible	2	0.49
All eighth-grade sampled private schools	400	100.00
Eligible	330	82.50
Has sampled grade, but no eligible students	11	2.75
Does not have sampled grade	19	4.75
Closed	28	7.00
Not a regular school	13	3.25
Duplicate on sampling frame	0	0.00
Other ineligible	2	0.50
All twelfth-grade sampled private schools	160	100.00
Eligible	120	75.00
Has sampled grade, but no eligible students	4	2.50
Does not have sampled grade	14	8.75
Closed	7	4.38
Not a regular school	12	7.50
Duplicate on sampling frame	0	0.00
Other ineligible	2	1.25
NOTE: Numbers of schools are rounded to nearest ten, except those pertaining to ineligible schools. Detail may not sum to totals due to rounding. Percentages are based on rounded counts. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Assessment.

NAEP Technical Documentation Student Sample Selection for the 2013 Private School National Assessment

The target student sample size within sampled schools varied by grade. For fourth grade, the target was 64 students; and for eighth grade, the target was 65 students. However, schools with 72 or fewer students automatically had all students sampled. In addition, at grade 4 only, a school that had more than 72 students but fewer than 90 students could choose to have all students sampled. For schools sampled for the twelfth grade the target was 60 students. However, schools with 66 or fewer students had all students sampled.

There was only one spiral type for each grade. The percentage of booklets by subject within the spiral for each grade is given below.

Percentage of booklets, private school national assessment, by subject within the spiral and grade: 2013

Grade	Mathematics	Reading	Pilot/Special
4	45.49	46.19	8.33
8	44.94	45.00	10.06
12	48.51	51.49	†
† Not applicable. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Assessment.

The process of student list submission, sampling students from year-round schools, sampling new enrollees, and determining student eligibility and exclusion status was the same as for the state NAEP student sample.

NAEP Technical Documentation School and Student Participation in the 2013 Private School National Assessment

Private school participation in NAEP is not mandatory. The 2013 assessment holds true to the historic pattern of having higher rates of participation in Catholic schools than among non-Catholic schools. Although a portion of the participating school sample consisted of substitute schools, it is preferable to calculate school response rates on the basis of school participation before substitution.

In every NAEP survey, some of the sampled students are not assessed for the following reasons:

withdrawn students;

excluded students with disabilities (SD);

excluded English language learners (ELL) students; or

students absent from both the original session and the makeup session (not excluded but not assessed).

Weighted School Response Rates

Weighted Student Response and Exclusion Rates for Mathematics

Weighted Student Response and Exclusion Rates for Reading

Withdrawn students are those who have left the school before the original assessment. Excluded students were determined by their school to be unable to meaningfully take the NAEP assessment in their assigned subject, even with an accommodation. Excluded students must also be classified as SD and/or ELL. Other students who were absent for the initial session are assessed in the makeup session. The last category includes students who were not excluded (i.e., “were to be assessed”) but were not assessed either due to absence from both sessions or because of a refusal to participate. Assessed students are also classified as assessed without an accommodation or assessed with an accommodation. The latter group can be divided into SD students assessed with an accommodation, ELL students assessed with an accommodation, or students who are both SD and ELL and accommodated. Note that some SD and ELL students are assessed without accommodations, and students who are neither SD nor ELL can only be assessed without an accommodation.

The weighted response rates utilize the student base weights and indicate the weighted percentage of assessed students among all students to be assessed. The exclusion rates, in contrast, provide the weighted percentage of excluded SD or ELL students among all absent, assessed, and excluded students.

NAEP Technical Documentation Weighted School Response Rates for the 2013 Private School National Assessment

The following table presents counts of eligible sampled schools and participating schools, as well as weighted school response rates, for the private school samples in which the mathematics and reading operational assessments were conducted. The weighted school response rates estimate the proportion of the student population that is represented by the participating school sample prior to substitution.

Private school response rates, national assessment, by school type and grade: 2013

Grade	Private school type	Eligible sampled schools	Participating schools, including substitutes	Weighted school response rate prior to substitution (percent)
4	All private	350	280	71.19
	Catholic	130	120	88.65
	Non-Catholic private	220	160	56.94
8	All private	330	260	69.63
	Catholic	120	120	87.18
	Non-Catholic private	200	140	53.51
12	All private	120	90	53.34
	Catholic	40	30	68.06
	Non-Catholic private	80	50	38.52
NOTE: Numbers of schools are rounded to nearest ten. Detail may not sum to totals due to rounding. SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 National Assessment.

Mathematics Assessment

The following table presents the weighted student response and exclusion rates for the mathematics assessment. The exclusion rates give the percentage of students excluded among all eligible students. Excluded students must be either students with disabilities (SD) or English language learners (ELL). The response rates indicate the percentage of students assessed among those who were intended to take the assessment within the participating schools. Thus, students who were excluded are not included in the denominators of the response rates.

Weighted student response and exclusion rates for private schools, national mathematics assessment, by school type and grade: 2013

Grade	Private school type	Weighted student response rate	Weighted percentage of all students who were SD and excluded	Weighted percentage of all students who were ELL and excluded
4	All private	95.61	0.06	0.03
	Catholic	95.60	0.00	0.06
	Non-Catholic private	95.62	0.11	0.00
8	All private	94.74	0.19	0.08
	Catholic	95.73	0.10	0.16
	Non-Catholic private	93.50	0.26	0.00
12	All private	86.51	0.63	0.00
	Catholic	85.53	0.83	0.00
	Non-Catholic private	87.96	0.42	0.00
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Mathematics Assessment.

Reading Assessment

The following table presents the weighted student response and exclusion rates for the reading assessment. The exclusion rates give the percentage of students excluded among all eligible students. Excluded students must be either students with disabilities (SD) or English language learners (ELL). The response rates indicate the percentage of students assessed among those who were intended to take the assessment within the participating schools. Thus, students who were excluded are not included in the denominators of the response rates.

Weighted student response and exclusion rates for private schools, national reading assessment, by school type and grade: 2013

Grade	Private school type	Weighted student response rate	Weighted percentage of all students who were SD and excluded	Weighted percentage of all students who were ELL and excluded
4	All private	95.85	0.46	0.07
	Catholic	95.75	0.17	0.06
	Non-Catholic private	95.96	0.71	0.08
8	All private	95.45	0.18	0.12
	Catholic	96.07	0.21	0.00
	Non-Catholic private	94.67	0.16	0.23
12	All private	85.52	0.78	0.05
	Catholic	84.67	0.81	0.10
	Non-Catholic private	86.75	0.75	0.00
SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2013 Reading Assessment.