ECLS Part B v24 Final

The ECLS-K:2024 national sample of children will be selected using a multi-stage probability design. The precision requirements, sample design, and procedures for computing sampling weights and variance are discussed below. This discussion is followed by a description of the procedures that will be followed to draw the sample that will be used for the K-1 field test.

B.1.1 National Sample Precision Requirements

The objective of the ECLS-K:2024 is to have a sample that can provide precise cross-sectional and longitudinal estimates of children in a particular school year (for the ECLS-K:2024, the 2023-24 school year), similar to the ECLS-K and the ECLS-K:2011. The sample plan was designed to achieve nationally representative and precise estimates for the cohort of kindergarten children in the 2023-24 academic year and the study years that follow. For this reason, the precision requirements that drive the overall sample design are those for the kindergarten children.

The kindergarten sample requirements for the ECLS-K:2024 are similar to those for the previous ECLS studies, to obtain a minimum level of reliability for child-level estimates pertaining to analytical subgroups, such as Asians, Asian and Native Hawaiians and Other Pacific Islanders (APIs), Blacks, Hispanics, private school kindergartners, and language minority students. Four precision requirements form the basis for the kindergarten sample design and plans for the follow-ups in subsequent rounds. These requirements are the ability to do the following:

Measure a relative change of 20 percent in proportions across waves;
Measure a relative change of 5 percent in a mean assessment score across waves;
Estimate a proportion for each wave with a coefficient of variation (CV) of 10 percent or less; and
Estimate a mean assessment score for each wave with a CV of 2.5 percent or less.

These precision requirements involve estimating changes over time and estimating the precision of estimates in the fifth-grade (i.e., final) data collection, and they are similar to those used in the ECLS-K and the ECLS-K:2011 cohorts. For the ECLS-K, it was determined that a fifth-grade sample of about 10,300 students would be adequate to meet the precision requirements overall and for most subgroups. The assumptions underlying the sample size were: a two-tailed test of differences with significance level alpha of 0.05 and power beta of at least 80 percent; estimating proportions of 30 and 36 percent (i.e., 20 percent relative change); a correlation between assessment scores from different waves of 0.6; and a design effect of 2.0. These assumptions were revised during the course of the ECLS-K for several reasons: the addition of subgroups of analytic interest such as language minority students; higher than expected design effects for assessment scores; and, higher than expected correlation of assessment scores over time. The fifth-grade data collection began with 16,143 sample students and resulted in 12,129 eligible fifth-grade students and 11,820 respondents, exceeding the 10,300 students in the initial projections. While the initial assumptions that drove the sample design were not always accurate separately, the overall effect of the losses was very close to expectation. For example, in several rounds of the ECLS-K, the assumed moving rate was lower than the actual moving rate, but this was offset by higher completion rates.

For the ECLS-K:2011, the assumptions were revised based on the previous experience. A fifth-grade sample of about 11,226 students was determined to be adequate to meet the precision requirements overall and for most subgroups. The assumptions underlying the sample size were: a two-tailed test of differences with significance level alpha of 0.05 and power beta of at least 80 percent; estimating proportions of 30 and 36 percent (i.e., 20 percent relative change); a correlation between assessment scores from different waves of 0.75; and a design effect of 4.0. The fifth-grade data collection resulted in 12,346 respondents.

For the ECLS-K:2024, the sample size is determined by first solving for the sample size needed under simple random sampling with 100 percent overlapping samples between waves using the formula:

where n is the sample size per wave, α is the significance level, β is the power term equal, z has the standard normal distribution, ρ is the correlation between two waves, P₁ and P₂ are the two proportions being compared, Q₁=1- P₁, Q₂=1-P_2,, , and . When α=0.05, β=0.80, ρ=0.75, P₁=0.30 and P₂=0.36, the minimum sample size needed per wave for subgroup analysis is 241. Assuming a design effect of 4 (based on the ECLS-K and ECLS-K:2011), this sample size would need to be further increased by a factor of 4 to 964, since the effective sample size is equal to the sample size actually obtained divided by the design effect.

The assumptions used to arrive at the sample size by the end of the study apply the rates at which students move from the base year sampled school to other schools, the rates at which the movers will be subsampled, the rates at which the subsampled movers will be located, and the student completion rates to the base year sample. From the ECLS-K and ECLS-K:2011, the study has good estimates about the movements of students after each year and how successful the study program has been at locating them, and the ECLS-K:2024 has modeled the assumed rates for this study on this experience. In the ECLS-K, students who moved to another school (not residence) were followed at a rate of 50 percent in first grade, slightly higher in third grade so that all language minority students were retained, and slightly lower in fifth grade to accommodate a reduction in the overall sample size. The fifth-grade subsampling rates varied according to student characteristics with the highest rate applied to language minority students. For the ECLS-K:2011, movers were subsampled at a rate of 50 percent in all rounds unless they were in a subgroup of interest (APIs, language minority, or had an IEP/IFSP); those were followed with certainty. As a result, the net rate of following movers in ECLS-K:2011 was 65 percent.

For the ECLS-K:2024, a sample of 1,000 responding schools (800 public and 200 private) with an average sample take of 21 students in each school should yield approximately 21,000 sampled students. Based on cooperation rates observed in the ECLS-K:2011, it is expected that this will yield 18,060 completes¹ at the end of base year and approximately 11,655 completes at the end of fifth grade, as shown in table B-1. This sample size for fifth grade is somewhat smaller than the sample sizes achieved for the ECLS-K (11,820) and ECLS-K:2011 (12,346). However, applying the sampling rates for subgroups of interest described in the next section, the resulting sample size should satisfy the precision requirements.

Using the ECLS-K:2011 to inform the sample design of ECLS-K:2024 follows the model used in the past. ECLS-K estimates were used to guide the sample size decisions for ECLS-K:2011 and even though there were minor decreases in response rates, the ECLS-K:2011 exceeded the sample size target. In the ECLS-K:2024, if rates differ dramatically there are a few approaches that can be used to increase the sample size. In the base year, substitute schools can replace nonresponding districts and schools to enable the study to reach the targets of 1,000 responding schools and 21,000 base year students. The sampling of schools assumes a response rate of approximately 60 percent. If a PSU’s school-level response rate is high enough, then no substitute schools will be selected. For PSUs with a low school response rate but with high district-level cooperation, substitute schools will be selected from the cooperating districts. If both school response rate and district cooperation are low, substitute schools will be selected from new districts. In some cases, for example, if there is only one district in a PSU, these substitute schools may come from a new PSU in the same sampling strata. At the child-level, response rates have traditionally been very high. However, if these are too low causing the number of respondents at the end of a round to fall below expectation, the rate in which movers are followed can be adjusted as a way to increase the sample size for the next round. During data collection study staff will closely monitor response rates and determine any potential modifications to sampling procedures that become necessary.

In table B-1, the starting sample for each subsequent round after the base year is the number of base year completes. A mover is defined as a child who moves out of the original sample school. In each subsequent round after the base year, 50 percent of movers who move out of the original sample schools will be subsampled, unless they are a member of a subgroup of interest (APIs, language minority, or have an IEP/IFSP²); these are followed with certainty. In ECLS-K:2011 this resulted in an overall mover rate of 65 percent.

Table B-1. ECLS-K:2024 expected sample size at each wave - All base year respondents retained for each wave

Study year	Beginning wave sample size	Mover rate	Mover subsampling rate	Locating Rate	Base year response rate	Mover completion rate	Nonmover completion rate	End wave sample size	End wave number of completes
Kindergarten¹	21,000	†	†	†	0.86	†	†	18,060	18,060
Spring first²	18,060	0.18	0.65	0.90	-	0.65	0.90	16,711	14,564
Spring third²	18,060	0.36	0.65	0.90	-	0.65	0.90	15,362	12,875
Spring fifth²	18,060	0.49	0.65	0.90	-	0.65	0.90	14,387	11,655

† Not applicable

¹The base year response rate is from the ECLS-K:2011.

²Mover rate, locating rate, and completion rates are from the ECLS-K:2011.

The four precision requirements are of equal importance for Hispanics, Blacks, and all other races. However, these subgroups do not have an impact on determining the oversampling rates for special groups because their expected sample sizes exceed the required sample size for meeting the requirements. At the end of the fifth-grade data collection for the ECLS-K, the distribution of completed cases among non-API race/ethnicity was 11.0 percent Black, 19.1 percent Hispanic, and 61.4 percent all other non-API races. At the end of the fifth-grade data collection for the ECLS-K:2011, the distribution of completed cases among non-API race/ethnicity was 10.0 percent Black, 26.7 percent Hispanic, and 53.9 percent all other non-API races. For the ECLS-K:2024, it is expected that the study will have approximately 1,535 Blacks, 2,800 Hispanics, and 6,025 children of other non-API races at the end of fifth grade.³ These sample sizes are large enough to satisfy precision requirements.

Language minority children are another subgroup of analytical importance. It is expected that 13.5 percent of the base year sample and 16.5 percent of the fifth-grade sample will be language minority children, based on the data from the ECLS-K:2011. The higher percentage of language minority children in the fifth-grade year is due to language minority movers being retained with certainty. Language minority children will be identified using data from the child assessment and based on the languages spoken at home, as identified by the school and in the parent survey. For the ECLS-K:2024, with a base year sample of approximately 18,060 completes and a fifth-grade sample of approximately 11,655 completes, it is expected that the study will have approximately 2,440 language minority completes in the base year and approximately 1,920 language minority completes in the fifth-grade year.

B.1.2 National Sample Design

The sample of children included in the ECLS-K:2024 will be selected using a multi-stage probability design. The first-stage or primary sampling units (PSUs) will be geographic areas that are counties or groups of counties. In the second stage, samples of public and private schools with kindergarten programs⁴ will selected within the sampled PSUs. Both PSUs and schools will be selected with probability proportional to measures of size that take into account the desired oversampling of APIs. The third stage sampling units for the national sample are children enrolled in kindergarten programs, and children of kindergarten age in ungraded schools, selected within each sampled school.

B.1.2.1 Sampling PSUs

The first sampling stage is the selection of geographic areas or PSUs. Clustering the sample into relatively compact geographic units is necessary to control the cost of data collection. PSUs are counties, or groups of counties, instead of states or school districts. In most cases, state is too large a unit to reduce data collection costs, while school districts do not administratively include private schools and there is no clear mapping of which private schools fall within the geographic boundaries of public school districts. In addition, district-level enrollment would be needed to reflect the enrollment of the corresponding private schools under such a design. Counties, on the other hand, have well-defined boundaries and the use of combined counties has the additional benefit of providing a more heterogeneous area, which may reduce the variance of estimates due to clustering.

Sampling Frame

The sampling frame for the ECLS-K:2024 was created from the 2018 American Community Survey (ACS) conducted by the Census Bureau.⁵ The 2018 ACS produces estimates for the 3,142 counties in the 50 states plus the District of Columbia. Counties or groups of contiguous counties were used to form PSUs that respect state boundaries and have a minimum population size of 15,000.⁶

PSU Measure of Size

The PSU frame was updated to reflect the most recent ACS estimates of kindergartners. The PSUs was further customized by combining units to yield the required minimum number of kindergartners.⁷ A stratified sample of 100 PSUs was selected with probabilities proportional to size.⁸ The PSU measure of size was a function of the number of kindergartners in the PSU. Members of the API minority group for which oversampling is required contributed more to the measure of size, so that the probability of sampling PSUs with a large proportion of APIs was increased. The measure of size for a PSU was the number of 5-year-olds who are not API plus 2.5 times the number of API 5-year-olds.

The oversampling rate for the API group was computed as the target number of completed cases in the API group divided by the expected number of completed cases in the API group without oversampling. Using the ECLS-K:2024 target for a subgroup of 964 (see section 2), and assuming that 4.8 percent of 5-year-olds are APIs (as estimated during the sample design based on the estimate of 0 to 5-year-olds from the 2015 ACS), then the API oversampling rate is 1.7 (the target sample divided by the expected yield). To account for different data collection and measurement issues,⁹ the study will oversample the API using a factor of 2.5 instead of 1.7. This is the same factor used in the ECLS-K and ECLS-K:2011. These collections had 971 and 1,135 completed APIs at the end of the fifth-grade data collections, respectively.

PSU Stratification and Sample Selection

PSUs with large measures of size were included with certainty. In the ECLS-K:2011 the study identified 10 certainty PSUs, each with measures of size large enough so that they had an approximately 75 percent (or higher) selection probability. The ECLS-K:2024 plans to use the same or similar threshold for the ECLS-K:2024 sample. The remaining noncertainty PSUs were then grouped into strata. The number of strata was equal to one-half the number of noncertainty PSUs so that two PSUs could be sampled from each stratum. The grouping of the PSUs into the strata was done using WesStrat. This software identifies the best stratification scheme for minimizing the between-PSU variance, while attempting to make the stratum population sizes approximately equal. Census region, level of urbanization, minority status (e.g., percent of the population in the PSU who are Black, Hispanic, APIs, or American Indian/Alaskan Native), and income level was used as the stratification variables in WesStrat. These stratification variables are the most pertinent to a school-based survey. Other variables on the PSU frame are number of households, area, distance,¹⁰ and state.

A sample of 100 PSUs was selected and the PSUs with large measures of size included in the sample with a probability of one as described above. Selection of two noncertainty PSUs within the noncertainty strata were performed with probabilities proportional to size. Durbin’s Method was used to select the two PSUs, as was done in the ECLS-K and the ECLS-K:2011.

In the ECLS-K, replicate weights were computed taking into account the Durbin method of PSU selection. This method of computing replicate weights for variance estimation caused the variance estimates to be unstable. In the ECLS-K:2011 the study computed replicate weights as if the PSUs were selected with probability proportional to size, discounting the Durbin method. This method produces stable and conservative estimates of variance and is, therefore, preferred because no estimation method is both stable and unbiased for the design. In the ECLS-K:2024 the study plans to compute the variances as done in the ECLS-K:2011.

B.1.2.2 Sampling Schools

The second stage of sampling involves selecting samples of public and private schools that have kindergarten programs or ungraded classrooms that serve kindergarten-aged children, from within the sampled PSUs. The targeted sample is 200 private and 800 public schools, for a total of 1,000 schools. The study will initially sample larger numbers of schools to account for estimated school nonresponse. In the ECLS-K, private schools participated at a rate of 65 percent and public schools at a rate of 70 percent, while in the ECLS-K:2011 these rates were 61 percent for private schools and 63 percent for public schools. Even though all efforts will be made to maintain or raise the school response rate, due to the decreasing nature of response rates across all studies, a modest decline of 4 percent from the ECLS-K:2011 school rates is planned, and so the study will sample 200/0.57 or about 350 private schools, and 800/0.59 or about 1,360 public schools.

School Frame Construction and Measure of Size

Within each sampled PSU, the sampling frame will be the list of all public and private schools offering kindergarten or have ungraded classrooms that serve kindergarten-aged children. For the ECLS-K:2024, the study will use the sampling frame that Westat will have created for the 2023 NAEP.¹¹ The sources for the NAEP school frame are the 2020-21 Common Core of Data (CCD) and 2019-20 Private School Universe Survey (PSS). NAEP supplements this frame with schools with the appropriate NAEP grades that are not found in the CCD and PSS. This sampling frame is expected to be ready around April 2022. It will include all grades from pre-kindergarten to grade 12 (even though NAEP only selects students in grades 4, 8, and 12). School enrollment by grade and race/ethnicity as reported in the CCD and PSS will be included in the NAEP school sampling frame. Charter schools will be included in the public portion of the school frame and it is expected that they will be able to be sample in the same manner as all other schools from this frame. Since charter schools are more likely to be created or dissolved than other public schools, particular attention will be paid to these schools during the frame updating process.

Within each PSU, schools with fewer than 21 kindergarten students will be clustered together before sampling to ensure that the target sample size of about 21,000 kindergarten students in 1,000 responding schools is met. Schools (or combined schools) will be selected with probability proportional to size. As with the PSU sample, a weighted measure of size will be constructed taking into account the oversampling of APIs as follows.

where 2.5 is the oversampling rate for APIs, is the estimated counts of API kindergarten students in the school, and is the estimated count of all other kindergarten students in school j in PSU i.

The target number of sampled schools per PSU will be calculated separately for public and for private schools and adjusted upward to offset anticipated school response and eligibility rates. The number of schools allocated to each PSU will be set proportional to the weighted measure of size of the PSU.

Stratification and Selection of Schools

Public and private schools will constitute distinct sampling strata. The public school sample will be selected using a traditional nested two-stage design. Within each PSU, public schools will be ranked by measure of size and partitioned into classes of roughly equal aggregate measure of size. Within each size class, schools will be sorted in a serpentine manner by the proportion of APIs.¹² For the private school stratum, schools will be grouped within PSU by affiliation (religious vs. nonsectarian) and then sorted in a serpentine manner by the measure of size.

The selection of both public and private schools will be systematic, with probability proportional to the measure of size m described above. This measure of size gives schools with greater numbers of API students a higher probability of selection without inordinately increasing the sampling errors of non-minority estimates.

Supplemental School Sample

Because the school source files come from the 2020-21 CCD and PSS and may not have complete coverage of non-traditional schools such as charter schools, the study undertake a new school search procedure in the spring prior to kindergarten data collection to ensure that coverage is as complete as possible for schools offering kindergarten. This procedure will be based on the experience obtained from the ECLS-K and ECLS-K:2011.

For each public school district and Catholic diocese having one or more schools sampled, study staff will check the relevant website to identify any additional schools expected to offer kindergarten or have ungraded classrooms that serve kindergarten-aged children in academic year 2023-24 that are missing from the list. Since a public-school district or diocese may cut across county or even state lines, each new school will be associated with the correct county, and hence the correct PSU, before checking to see whether it is truly new. Bona fide new schools will be given an appropriate chance of being sampled.

The identification of non-Catholic private schools will be different than what was done for the ECLS-K and ECLS-K:2011. In the ECLS-K, lists of schools from different sources (such as the Yellow Pages) were compiled and then matched against the PSS list frame to remove duplicates. Further screening had to be undertaken by telephone to verify new school status. After all new schools were identified, a supplementary sample of the new schools was drawn. This procedure had a very low yield of eligible, participating schools (only 36% of the sampled schools participated). Given the small return for the effort expended in identifying new non-Catholic private schools in the ECLS-K, this procedure was modified for the ECLS-K:2011. The list of non-Catholic schools on the PSS was compared to those on the Quality Education Data (QED) database and the study sampled from the list of newly found schools. Since the ECLS-K:2011 sample was selected, the creation of the QED file has been transferred from one entity to another. Due to uncertainty as to the quality of the QED and the increased ability to identify Catholic schools over the internet for the ECLS-K:2024, for the ECLS-K:2024 the study will identify new non-Catholic private schools in a similar manner. Web searches will be used to identify any additional private schools in the county that are expected to offer kindergarten in the 2023-24 school year.

New schools identified through web searching, including all new public, Catholic, and non-Catholic private schools, will be selected separately and with equal probabilities. The measure of size used for selecting both the public and Catholic schools will be 21, the minimum desired cluster size of kindergartners. Because there are typically fewer students in non-Catholic private schools, their measure of size will be smaller. The minimum desired cluster size used in these cases will be the median school size among the non-Catholic schools¹³ from the PSS.

B.1.2.3 Sampling Children, Parents, and Teachers

The goal of the sample design is to obtain an approximately self-weighting sample of students, with the exception of APIs who need to be oversampled because they do not naturally occur at a rate necessary to meet the sample size goals. Within each sampled school, the study obtains a complete list of kindergartners enrolled, taking special care to avoid excluding children from the list because of disability or language problems. School staff will be given detailed instructions and prompts to ensure that the list includes children with disabilities or children for whom English is not their native language.

For the students on a school’s student list, two independent sampling strata will be formed, one containing API students and the second containing all other students. API students will be sampled from the API stratum and the sampling rate will be 2.5 times the rate of sampling rate of non-API students. Within each stratum, students will be selected using equal probability systematic sampling. Based on an estimated sample of 1,000 participating kindergarten schools, the target number of children sampled at a school would be 21. If the sampling unit is a cluster of small schools, the school rosters will be kept separate by appending them one after the other. Sampling will be done systematically and with equal probability from the list so that if a school is small, then fewer students will be sampled from this school. If a school has 21 students or more according to the school frame but turns out to have fewer than 21 kindergartners or children of kindergarten age in ungraded classrooms enrolled at the time of student sampling, all students in this school will be sampled. Twins will not be identified prior to sampling, but they may enter the sample through this method of probability sampling. Once the sampled kindergarten children are identified, parents will be contacted to obtain consent to conduct the child assessment and to ask for completion of the parent survey.

It is possible that in some states and school districts the role of transitional kindergarten programs has changed since the ECLS-K:2011. When the initial lists of students are obtained from the sampled schools, large deviations from the expected number of kindergarten students will be investigated and verified with the schools to ensure that the study has complete coverage of all eligible children and classrooms in the school, including those children in transitional kindergarten programs.

Additionally, if a school reports a large number of transitional kindergarten children, further consideration will be given as to how to include the school and students in the study. In past ECLS kindergarten cohort collections, transitional kindergarten students were treated as eligible for sampling and this is currently planned for the ECLS-K:2024. However, if it is observed that that rate of transitional kindergarten students is dramatically different from previous years (e.g., in the prior ECLS collections), the study may consider either subsampling these students at a lower rate or excluding them from the sample.

Teachers who teach the sampled children will also be included in the study and will be asked to fill out various teacher surveys. All teacher data will be linked to their students. In the ECLS-K a census of kindergarten teachers was taken at each school and those teachers that completed the teacher characteristics section were considered to have a completed teacher questionnaire. For the ECLS-K:2011 only the kindergarten teachers of sampled students were surveyed. NCES is currently considering a census of kindergarten teachers in the ECLS-K:2024. If this is included in the study, sampling for ECLS-K:2024 will follow the ECLS-K design; otherwise the ECLS-K:2011 method will be used.

B.1.3 K-1 Field Test Sample Design

The sample of children for the K-1 field test was selected after the sample of PSUs were selected for national data collection. This minimized the possibility of selecting the same school for both the field test and national samples. The K-1 field test sample was then selected from a random sample of schools in five purposely selected areas. These areas came from counties sampled in the ECLS-K:2011 national study that are not included in the ECLS-K:2024 national study. Using areas sampled previously will allow the use of staff with ECLS-K:2011 experience and may result in higher school-level cooperation as some of the school districts with sampled schools will be familiar with the ECLS program.

An initial set of 50 schools in the 5 chosen areas (10 per area) was selected. Another 50 schools in these areas was selected as a reserve sample.

B.1.3.1 Sampling PSUs

One of the primary goals of the field test is to assess survey items on a wide range of students and their parents, their teachers, and their school administrators. To achieve these goals, PSUs for the K-1 field test were chosen from a diverse set of PSUs, based on the stratification variables (such as region, urbanicity, race/ethnicity) used in selecting the ECLS-K:2024 national sample.

As noted earlier, these PSUs came from those counties sampled in the ECLS-K:2011 national study that are not among the ECLS-K:2024 national study primary or reserve PSUs. Preference was given to counties with field staff availability and high child and school response rates in the ECLS-K:2011.

B.1.3.2 Sampling Schools

The second stage of sampling for the K-1 field test involves selecting samples of public and private schools that have kindergarten through second-grade programs, from within the chosen PSUs. The target sample is 10 private and 40 public schools, for a total of 50 participating schools. For the K-1 field test, 2 private schools and 8 public schools will be selected in each of the chosen PSUs. Unlike the national study, where the number of schools contacted will be inflated by the expected school response rate, the initial K-1 field test sample will include just 10 schools per PSU. To account for any school-level nonresponse among these schools, a replacement school will be selected for each original school. These replacement schools will be identified when the original schools are selected. If an original school becomes a final refusal, their replacement school will then be contacted and asked to participate in the field test.

School Frame Construction

Within each sampled PSU, the sampling frame for the field test will be the list of public and private schools that offer kindergarten, first-grade, and second-grade programs. To reach the field test targets for each of these grades, the frame will be restricted to include only those schools with at least 28 kindergartners, 28 first-graders, and 14 second-graders. The sampling frame that will be used will be created for the 2023 NAEP. The sources for the NAEP school frame are the 2020-21 Common Core of Data (CCD) and 2019-20 Private School Universe Survey (PSS). NAEP supplements this frame with schools with the appropriate NAEP grades that are not found in the CCD and PSS. It will include all grades from pre-kindergarten to grade 12 (even though NAEP only selects students in grades 4, 8, and 12). School enrollment by grade and race/ethnicity as reported in the CCD and PSS will be included in the NAEP school sampling frame. Charter schools will be included in the public portion of the school frame and will be sampled the same as all other public schools from this frame. To increase the coverage of schools in the national study, the school frame may be enhanced at that time through web searches.

Stratification and Selection of Schools

Public and private schools will constitute distinct sampling strata. The public school sample will be selected using a traditional nested two-stage design. Within each PSU, public schools will be ranked by measure of size and partitioned into classes of roughly equal aggregate measure of size. For the private school stratum, schools will be grouped within PSU by affiliation (religious vs. nonsectarian) and then sorted in a serpentine manner by the measure of size.

B.1.3.3 Sampling Children, Parents, School Administrators, and Teachers

Within each sampled school a complete list of enrolled kindergartners, first-graders, and second-graders will be obtained. School coordinators, who act as liaisons between the study and the school, will sample eligible students from the list. In the national study, to reach the desired sample size of Asian or Native Hawaiian and Other Pacific Islanders (API) children, that group will be oversampled by a factor of 2.5. For the K-1 field test, API children will be sampled at the same rate as the non-API students.

Once the students are identified, their parents will be contacted to obtain consent¹⁴ to conduct the child assessment and to ask for completion of the parent survey. To reach the targeted number of children, a “take-all” approach will initially be used within schools. Overall, the goal is to complete assessments with approximately 3,000 children (1,200 kindergartners, 1,200 first-graders, and 600 second-graders).¹⁵ Not all eligible students in a school will be assessed, despite using a “take all” approach. Factors such as number of days at the school and the number of field staff will naturally limit the number of assessments that can be completed. Project staff will monitor school production daily and communicate to each school’s field staff which of the eligible students to target in order to meet the necessary number of completes.

The parents of the selected students will be asked to participate in the field test by providing consent for their child’s participation and by completing a web (or telephone, as the back-up mode) survey. The primary and special education teachers of the selected kindergarten and first-grade children will also be identified and asked to complete web (or paper) surveys.¹⁶ Finally, the school administrator at each participating school will be asked to complete a web (or paper) survey.

The goal of the field test is to collect surveys from approximately 50 school administrators, 500 teachers (200 kindergarten, 200 first-grade, and 100 special education), and 3,000 parents.¹⁷ In each school, it is estimated that six children will be linked to each teacher, and two children linked to each special education teacher.

B.2 Procedures for Collection of Information

The previous section addressed the statistical methodology for the study, including stratification and sample selection, estimation, and precision requirements. This section describes the school recruitment and data collection procedures for the K-1 field test data collection, as well as the school recruitment procedures for the national study.

Included in the text are descriptions of the roles of the field staff who will conduct the K-1 field test study activities. Experienced school recruiters will contact district and school staff to discuss participation in the study, schedule dates for the ECLS school visit, discuss parental consent, and collect basic school information. ECLS team leaders manage the ECLS teams in the school and will serve as the main contact for the schools included in their work area during the data collection period. They will take over the cases in the fall of 2022. They will conduct a study activities call with the school and visit the schools, with their team of assessors, to conduct assessments and other study activities. These activities are described in detail in the sections below.

B.2.1 K-1 Field Test District and School Recruitment

Before school, child, parent, and teacher data collection can occur, the study must secure the cooperation of the selected schools and their districts. The goal of the K-1 field test will be to recruit 50 schools to participate.

All of the K-1 field test respondent recruitment and study materials are contained in Attachment A-1.¹⁸ Note that letters included in this attachment will be printed on the field test version of the study letterhead, which is also included in the attachment. Several of the materials reference the MyECLS website, which is the portal respondents will use to provide the study with information and to launch surveys.¹⁹ When referenced in emails or texts, the MyECLS website link will be hyperlinked. Clicking on the link from the email or text will take the respondents to MyECLS.ed.gov. Spanish translations of the parent recruitment materials are included with this submission. The study’s introductory letter and consent forms will also be available in Mandarin and are also included in this submission.²⁰

Attachment A-3 contains the script for a respondent video; A-4 contains study informational PowerPoint slide decks for teachers and parents; and A-5 contains infographics created from findings from previous ECLS, as well as other U.S. Department of Education, studies.²¹ The field test data collection instruments described below appear in full in Attachments B-E. Attachment F contains details about the MyECLS respondent website and Attachment G contains a matrix summarizing the study items and their sources.

District recruitment. In the spring of 2022, public school districts and Catholic dioceses containing schools selected for the field test will be sent a package via FedEx with a signature requirement. A sticker with the U.S. Department of Education seal and the message, “Important information from the U.S. Department of Education.” may be included on the outside of packages. The package will include a letter describing the study and an ECLS fact sheet. A one-page fact sheet acknowledging the coronavirus pandemic and explaining safety precautions to be taken while in the school for data collection will also be included in the mailing as needed.²² Experienced school recruiters will contact the districts via telephone to obtain permission to contact the schools associated with the district. In addition, the recruiters will obtain information on any special approval processes that each district requires, such as research applications or district-specific consent forms. Study staff will complete necessary research applications. Recruiters will follow up with districts to track the status of application submissions. In some cases, in-person visits by field recruiters or teleconferences with Westat home office and/or NCES staff may be necessary to secure district cooperation. If a district chooses not to participate, the recruiters will attempt refusal conversion, including in-person meetings or transferring the case to another recruiter. All concerns and reasons for refusal will be documented.

School recruitment. Once district/diocese approval is secured,²³ principals at selected public and Catholic schools will be sent a package via FedEx with a signature requirement. (Private schools will be sent this package at the beginning of the field period, when public school districts and Catholic dioceses are contacted.) A sticker with the U.S. Department of Education seal and the message, “Important information from the U.S. Department of Education.” may be included on the outside of packages. The package will include a letter describing the study and an ECLS fact sheet, along with the one-page fact sheet on the precautions the study is taking due to the coronavirus pandemic. The letter will display the name of the district-level approver, when available. School recruiters will contact the sampled schools via telephone to obtain principal approval for the school’s participation in the study. Once approval is obtained, recruiters will collect school information to prepare for fall data collection logistics, including identifying a school coordinator to serve as the liaison for study activities in the school and determining the following:

School visit dates. The school recruiters will discuss the schedule for the fall 2022 K-1 field test data collection with the school principal or school coordinator. When arranging for school visit dates, the recruiter will clarify if the kindergarten classes are taught full day or half-day, as well as the start and end time of the school day to determine the number of visit days needed. The dates for the school visit to conduct assessments will be set, making sure to avoid conflicts with any special events in the school’s calendar, such as a field trip or school holiday.
Assessment locations. The locations within the school where the assessments will take place will also be determined. The goal will be to identify locations that provide as little distraction as possible, that protect the privacy of the children, and that are as nondisruptive of the school routine as possible.
Consent type. The recruiter will discuss with the school principal or coordinator the type of parental consent that is required, explicit or implicit. If explicit consent is chosen, parents must actively provide their consent. Implicit consent is assumed unless a parent opts his or her child out of participation. Schools will choose whether to distribute and collect consent forms electronically or on paper. The paper method can be selected for the entire school or for selected parents based on the school’s knowledge of these parents, both in terms of preference and internet access. Additionally, the parent letter will mention the availability of paper consent forms.
Other school logistics and information: in order to facilitate planning for the fall assessments, the recruiter will also collect other school information, for example enrollment counts, grades taught, and any special instructions.

District and school recruitment for the K-1 field test will continue through the summer of 2022 and into the fall of 2022 as needed, until there are enough participating schools to yield a sufficient number of children, parents, and teachers to complete the field test assessments and surveys.

B.2.2 K-1 Field Test Data Collection

The K-1 field test data collection will include direct child assessments, and parent, school administrator, and teacher (both primary classroom and special education) surveys. The field test child assessment will be conducted in a one-on-one setting, with trained field staff administering items that will appear on a paper easel.²⁴ The assessor will read each of the items to the child, as well as any available response options. He or she will then record the child’s response on a paper score sheet. School administrator, parent, and teacher data will be collected via self-administered web surveys. Paper surveys will be provided to school staff upon request. Nonresponding parents will be contacted by field staff to complete the survey via telephone. The goal of the field test will be to conduct assessments with approximately 3,000 children (1,200 kindergartners, 1,200 first-graders, and 600 second-graders) and to collect surveys from approximately 50 school administrators, 500 teachers (200 kindergarten, 200 first-grade, and 100 special education),²⁵ and 3,000 parents. Field test data collection will occur from August through November 2022, although it may continue into December if scheduling assessment dates within the timeframe becomes problematic.

The field test will not only serve as a test of the data collection procedures, but also as an evaluation of the effectiveness of the surveys that are planned for use in the kindergarten and first-grade national rounds. Resulting data will be reviewed for missingness, bias, and interpretation issues. Although some of the survey questions will overlap across the national rounds of data collection, each round of the study will contain unique surveys. Thus, each kindergarten and first-grade national round’s version of the surveys will be deployed in the K-1 field test. These include the following:

Parent surveys: Fall kindergarten, spring kindergarten, and spring first-grade.
Primary teacher teacher-level surveys: Fall kindergarten, spring kindergarten, and spring first-grade.
Primary teacher child-level surveys: Fall kindergarten, spring kindergarten, and spring first-grade.
Special education teacher-level surveys: Spring kindergarten and spring first-grade.
Special education child-level surveys: Spring kindergarten and spring first-grade.
School administrator surveys: Spring kindergarten and spring first-grade.

All surveys that will be deployed in the K-1 field test appear in Attachments B-E. Kindergarten and first-grade child assessment items will also be field tested. The child assessment items will be spiraled across four easels and the easels randomly assigned to the sampled kindergarten, first-grade, and second-grade children. Prior to the national study, the pool of assessment items will be reduced to include only those that demonstrate validity, meet the goals of content coverage to adequately measure the frameworks, and provide the desired distribution of item types. For the national collections, a kindergarten version of the assessment, for use in both the fall and spring kindergarten rounds, and a first-grade version of the assessment, for use in the first-grade round, will be developed.

To test all of the surveys, respondents will be asked to complete the appropriate surveys depending on the grade level they teach or parent. The primary teacher, special education teacher, and school administrator surveys will be available both on the web and on paper, so respondents will have the choice to complete the survey in a variety of school and non-school settings. Initially, respondents will be encouraged to complete the surveys on the web; paper versions will be distributed later in the field period to non-responders. Parent surveys will also be available on the web. Field staff will contact parent non-responders to complete the survey via the phone. In these cases, if the parent has not provided contact information on the MyECLS website, team leaders will work with the school coordinator to obtain parent contact information. The team leader will assign parent cases to himself or herself and to assessors on the team. The assigned field staff member will contact the parent, and if the parent is agreeable, complete the web survey with the parent over the phone. That is, the field staff will access the web survey and ask the questions from the website, entering the responses into the survey directly.

Links to access the assigned survey(s) will appear in MyECLS so as to be transparent for the respondent. Table B-2 displays the assignment scheme for the surveys that will be tested in the K-1 field test. Also included is the expected sample yield for each of the surveys.

Table B-2. Respondents by instrument type, with expected sample yield

Instrument	Kindergarten Participants (expected sample yield of approximately 1,200 children)	First-Grade Participants (expected sample yield of approximately 1,200 children)	Second-Grade Participants (expected sample yield of approximately 600 children)
Parent survey	Half of the parents of kindergartners will be asked to complete the fall kindergarten version of the parent survey (expected sample yield of 600 fall kindergarten parent surveys) Half of the parents of kindergartners will be asked to complete the spring kindergarten version of the parent survey (expected sample yield of 600 spring kindergarten parent surveys)	All of the parents of first-graders will be asked to complete the spring first-grade version of the parent survey (expected sample yield of 1,200 spring first-grade parent surveys)	All of the parents of second-graders will be asked to complete the spring first-grade version of the parent survey (expected sample yield of 600 spring first-grade parent surveys)
Primary teacher teacher-level survey	Half of the teachers that teach kindergartners will be asked to complete the fall kindergarten version of the teacher-level survey (expected sample yield of 100 fall kindergarten teacher-level surveys) Half of the teachers that teach kindergartners will be asked to complete the spring kindergarten version of the teacher-level survey (expected sample yield of 100 spring kindergarten teacher-level surveys)	All of the teachers of first-grade children will be asked to complete the spring first-grade version of the teacher-level survey (expected sample yield of 200 spring first-grade teacher-level surveys)	N/A¹
Primary teacher child-level survey	Half of the teachers that teach kindergartners will be asked to complete the fall kindergarten version of the child-level survey (expected sample yield of 600 fall kindergarten child-level surveys) Half of the teachers that teach kindergartners will be asked to complete the spring kindergarten version of the child-level survey (expected sample yield of 600 spring kindergarten child-level surveys) In the field test, if a teacher is assigned to complete a fall kindergarten teacher-level survey, he/she may complete either a fall or spring kindergarten child-level survey (and similarly teachers completing the spring teacher-level survey will complete either the fall or spring child-level survey).¹	All of the teachers of first-grade children will be asked to complete the spring first-grade version of the child-level survey (expected sample yield of 1,200 spring first-grade child-level surveys)	N/A¹

Table B-2. Respondents by instrument type, with expected sample yield—Continued

Instrument

Kindergarten Participants

(expected sample yield of approximately 1,200 children)

First-Grade Participants

(expected sample yield of approximately 1,200 children)

Second-Grade Participants

(expected sample yield of approximately 600 children)

Special education teacher teacher-level and child-level surveys

Study personnel will establish if a child has an IEP. If a child has an IEP then the child’s special education teacher, service provider, or the person that oversees the IEP will be contacted to complete the spring kindergarten versions of the teacher- and child-level special education surveys (expected sample yield of 50 spring kindergarten special education teacher-level surveys and 100 child-level surveys)

Study personnel will establish if a child has an IEP. If a child has an IEP then the child’s special education teacher, service provider, or the person that oversees the IEP will be contacted to complete the spring first-grade versions of the teacher- and child-level special education surveys (expected sample yield of 50 spring first-grade special education teacher-level surveys and 100 child-level surveys)

N/A²

School administrator survey

Half of the school administrators will receive the spring kindergarten version of the school administrator survey and half will receive the spring first-grade version of the school administrator survey

¹Assignment of the kindergarten fall or spring teacher-level survey will be based on teacher ID, while assignment of the kindergarten fall or spring child-level survey will be based on the child ID. Thus it is possible that kindergarten teachers will complete a fall teacher-level survey but a spring child-level survey, or vice versa. It is also possible that kindergarten teachers may complete both fall and spring child-level surveys.

² Second-grade primary and special education teachers may be asked to participate to meet sample-size targets dependent on response rates during the field period. If needed, they will be asked to complete either kindergarten or first-grade versions of the surveys, depending on for which surveys the response rates are running low.

NOTE: If the child is excluded from the child list and therefore not sampled due to one of the reasons listed above, the child will not be added to the database and the child’s parent and teachers will not be asked to participate. However, if a child is sampled and is later determined to be ineligible for the assessment, the parents and teachers will be asked to complete surveys.

Study activities. As schools agree to participate, school coordinators identified by the approving school administrators will be sent a welcome letter, letting them know about the study and activities they will be asked to complete in the fall. In August 2022, school coordinators in participating K-1 field test schools will be sent a welcome package via FedEx with a signature requirement. A sticker with the U.S. Department of Education seal and the message, “Important information from the U.S. Department of Education.” may be included on the outside of packages. The package will include a letter describing the study activities planned for the fall and the role of the school coordinator and instructions for providing information on the participating children and accessing the MyECLS website to record child, parent, and teacher information. The MyECLS screens that will be used in the K-1 field test appear in Attachment F.²⁶ The school coordinator will be asked to upload a list of kindergartners, first-graders, and second-graders who speak English and are not excluded from standardized assessments. For each child, the list will include the child’s first, middle, and last name; the child’s month and year of birth; the child’s grade; and the child’s sex.²⁷

Study staff will receive notification when the child list has been uploaded. The lists will be checked for formatting and accuracy and then children will be sampled. The K-1 field test child sample will be purposive, meaning that obtaining consent to participate will be attempted for all listed children. However, children will not be selected if they do not speak English well, if they require testing accommodations that the study is unable to provide (e.g., Braille, Sign Language), if they are excluded from standardized testing, and if they are not in the specified grades. Once the child sample has been selected, the school coordinators will be instructed to access the MyECLS website, download the list of participating children, review it for accuracy, and gather child information to provide to the team leader during the study activities call. Schools will receive an incentive of $10 per participating child.

For the study activities call, which will have been scheduled during the spring 2022 school recruitment activities, the team leaders will contact the school coordinators to review the school coordinator role and to address any questions that they may have.²⁸ A checklist of the tasks to be completed, available on the MyECLS website, will guide the study activities call. The team leaders will discuss the logistics of the fall 2022 school visit, confirming or updating school information gathered during the school recruitment round, such as the assessment dates and assessment location. The team leader will record updates in the MyECLS website. After the study activities call, the school coordinator will be sent a study activities package, containing a welcome letter with instructions for notifying school staff and parents about the ECLS, an incentive check, and tips for encouraging respondent participation. Also in the package will be packets for the school administrator, teachers, and parents for distribution by the school coordinator.

Parent consent. The team leaders will confirm the school’s requirements for obtaining parent consent that were identified during the spring recruitment period: that is, whether the school requires explicit consent from each child’s parent, or if the school will notify parents and consider consent to be implicit unless the parent refuses. The team leader will ask whether the school plans to collect consent forms electronically or via hard-copy forms and whether the school coordinator will distribute the parent packets or would like the team leader’s help doing so. They will also discuss if Spanish or Mandarin parent letters and consent forms are needed. Study staff will prepare parent packets containing a letter describing the study and inviting the parents to allow their child’s participation and to complete a parent survey, with instructions for accessing the MyECLS website. The parent packet will also include the ECLS fact sheet and, depending on the school’s assignment in the incentive experiment, a prepaid Mastercard® worth 15 dollars, and a pad of sticky notes displaying the U.S. Department of Education logo. The prepaid card will have text stating “Thank you!” See Section A.9 for details on the incentive experiment. All printed parent communication materials will be provided to all parents in both English and Spanish. The consent materials will also be translated into Mandarin. These translated materials will be included in the mailing alongside the English materials if the school indicates they are needed. If the school prefers that the team leader distribute the parent packets, the team leader will ask the school coordinator to provide parent contact information. If the school prefers to collect hard-copy consent forms, a consent form will be included in the packet. School coordinators will be provided with email and text message templates to send to nonresponding parents (see Attachment A-1).

For schools that require implicit consent only, the children’s consent status will be set to obtained and will remain as such unless the parent actively refuses participation. For schools that require explicit consent, the consent status will initially be set to pending. For schools and parents who choose electronic consent, the MyECLS website will update the consent status to obtained or refused as the parents record their preference on the MyECLS website. For schools and parents who choose hard-copy consent, the school coordinators or team leaders will update the consent status on the MyECLS website as consent forms are received. Regardless of the consent type and delivery method selected, the school coordinator and team leader will monitor the consent status, the goal being to obtain consent for all sampled children prior to the ECLS school visit. The team leader will monitor the consent status and offer to help follow up with parents; again, if follow-up is needed, the team leader will ask the school coordinator to provide contact information for the parents. If the school coordinator handles the parent contact, the team leader will ask him or her to update the MyECLS website with children’s consent status. Otherwise, as necessary, the team leader will update the consent status. For schools requiring explicit consent, to encourage the return of consent forms, schools will be offered a food event (e.g., pizza party, bagels, ice cream party) if all consent forms are returned, regardless of whether consent was obtained for all children.

Child and teacher information. The team leaders will also ask the school coordinators to provide information for the selected children, including the presence of an Individualized Education Plan (IEP), required testing accommodations, and language spoken at home. The team leaders will ask the school coordinators to provide information on the children’s primary classroom teachers, and to identify the special education teachers or related service providers for children with IEPs. Team leaders will record this information in the MyECLS website, where the school coordinator can view the information and communicate any changes to the team leader via telephone or via secure message on the MyECLS website. The secure messaging system allows the school and study staff to communicate information about the participating children. In use for the last three rounds of the ECLS‑K:2011, the Secure Messaging system allows field staff and schools to communicate PII electronically via a secure system, instead of through non-secure email messages or over the telephone.

Throughout these study activities, positive and cooperative working relationships with school personnel and the school community will be maintained.

Direct child assessment. Unlike the direct cognitive assessments in the national data collections, but as with the ECLS-K:2011 field tests, data from the direct cognitive assessments for the K-1 field test will be conducted using paper and pencil. Assessment items will appear on printed easels. The front of the easel will display the question text, any response options, and any associated images. The back of the easel will display the questions text, any response options, and any appropriate gesturing instructions for the assessor to use when administering the item. Assessors will read each item to the child and record children’s responses in a paper-and-pencil format using a separate score sheet.

In order to test many different items without overburdening the children, the four K-1 field test assessment easels will be developed from items divided into two reading forms and two mathematics forms. The reading assessments in the national data collections typically include more items than the math assessments. In addition, the reading forms generally require more time than math forms because many reading items require children to read passages. Although the passages are short, they are time-consuming. Consequently, more time is usually allotted to the reading forms than to the math forms.

These forms will be spiraled such that each child in the field test will receive one of two versions of the reading and one of two versions of the mathematics. The ECLS data collection management system will assign easel types to children, regardless of grade, to ensure that a sufficient number of each form is administered in each school. Assessors will be trained to record observations about children’s behaviors and response to the assessments and will also keep a more general diary of field test experiences. These field test observations will be used to prepare the field test report.

Accommodations will be provided to children who need them to the greatest extent possible. As previously mentioned, the team leader will work with the school coordinator to determine any accommodations that may need to be provided. Possible accommodations include, but are not limited to, scheduling the assessment at a particular time of day, conducting the assessment in a particular setting, using an assistive device, and having an aide present during the child’s assessment session. Children who do not speak English or are excluded from standardized assessments will be excluded from the K-1 field test assessment. The school coordinator will be asked to exclude these children from the child list, and the children’s language spoken will be confirmed during the study activities call.

Typically, the ECLS school visit will take between 3 and 5 days in each school. The number of days for the visit will depend on several factors, such as the number of participating children at the school, any restrictions on the assessment schedule (e.g., if assessments can be conducted only in the morning), and the amount of space available for simultaneous one-on-one assessments. The length of the school visit will be determined with the school coordinator during the study activities call. Generally, the ECLS team that visits the school will include the team leader and three assessors, though sometimes an additional assessor or two will participate in an school visit when a larger number of children needs to be assessed in a shorter time frame due to the school calendar. There will be one team per work area (which loosely corresponds to a selected county). The ECLS team will arrive at the school on the appointed first day of assessments and, following any of the school’s required check-in procedures, immediately contact the school coordinator. The team leader will introduce the assessors to the school coordinator. The procedures to be used during the on-site data collection period will be discussed with the school coordinator to ensure there is a common understanding of those procedures. The team leader also will confirm that all sampled children are still enrolled in the school as of the school visit and determine which children are present at school that day.

On each day of the school visit, the team leader and assessors will be taken by school personnel to the assessment area(s), where they will remove potential distractions as much as possible and establish a comfortable environment for conducting the assessments. They will set up the assessment materials: easels, score sheets, and any manipulatives. The team leader will print the school visit materials (Child Information Sheet and Sign-out Sheets) they will use to select the children for assessment. The Child Information Sheet will contain a list of the sampled children, their teacher information, any details on needed accommodations, and the easel type that they have each been assigned. Each assessor will have a copy of the form and will refer to it for each child he or she assesses. There will also be a central copy for assessors to check off children they plan to assess so that the child is not taken by another assessor. If the school wishes to use them, Sign-Out Sheets will be given to teachers of sampled children. Assessors will sign in and out the children they assess so that teachers and other school staff can find the children in the assessment location as needed. These forms will be stored at the school in the SC’s care at the end of each day and will be destroyed by the SC at the conclusion of the field period.

Once the assessment areas have been set up and assessors are ready to begin work, the school coordinator will introduce the ECLS-K:2024 team members to the teacher(s) whose children will be assessed. The teacher, in turn, will introduce the assessors to the class. Assessors will then escort the sampled children to the assessment areas, one-by-one, and conduct each direct child assessment, using the assigned easel and scoresheets.

When the assessor and study child arrive at the assessment space, the assessor will introduce the child to the task and begin the cognitive assessments. After completing the cognitive assessments, the child will be returned to the classroom and the next sampled child will be taken to the assessment area and assessed.²⁹ At the end of each day, the team leader will update the children’s assessment status in the MyECLS website and the team will return the assessment locations to their original state.

It is expected that some children will be absent from school when the assessments are scheduled. Certain days throughout the field period will be designated as days on which some field staff can conduct make-up assessments. Depending on assessment production, attempts will be made to conduct make-up assessments at some point during the field period for children absent on the day of the ECLS school visit.

Teacher and school administrator surveys. As mentioned previously, as part of the study activities, the school coordinators are asked to include the names and email addresses for the primary classroom teachers of sampled kindergarten and first-grade children³⁰ on the child list submitted on the MyECLS website. During the study activities calls, the team leaders will ask the school coordinators to confirm the teacher information and the team leaders will ensure that all teacher names are recorded on the MyECLS website. If a child has more than one primary classroom teacher, the team leader will ask the school coordinator to identify the teacher with whom the child spends the most time. The team leader will also ask the school coordinator to identify the special education teachers or related services provider for kindergarten and first-grade sampled children with an IEP. The team leader will then record the teacher information in the MyECLS website and link the sampled children to his or her teachers. This linking system was first developed and used successfully for the ECLS-K and ECLS-K:2011 data collections. Once teacher information has been recorded, the team leader will assemble teacher packets and send them to the school coordinator to distribute to the appropriate kindergarten and first-grade primary teachers and special education teachers. The teacher packets will contain a letter describing the ECLS-K:2024 and instructions for accessing the MyECLS website to complete the surveys, the ECLS fact sheet, and an incentive check. Each teacher will be offered $20 for the teacher-level survey plus $7 for each child-level survey. The average number of children per teacher is expected to be about six, so each teacher will receive an average of $62.

Primary classroom teachers of kindergarten and first-grade children will be asked to complete two types of self-administered web surveys. Half of the kindergarten teachers will complete the fall kindergarten surveys and the other half will complete the spring kindergarten surveys. All first-grade teachers will complete the spring first-grade surveys. Second-grade teachers will be asked to complete surveys for the field test only if necessary to meet production goals; information will be collected for all primary second-grade teachers to aid in school visit logistics for the second-grade children.³¹

The teacher-level survey includes questions about the teachers’ background and education, curriculum, and instructional practices, including their views on the school climate and their evaluation methods used for reporting to parents.

The second teacher survey is a child-level survey with questions that ask the teacher to rate the child identified in the survey on academic and social skills, school engagement, and classroom behaviors. The survey also contains questions on instruction and grading practices, behavioral issues, and homework assignments.

The special education teachers or related service providers of kindergarten and first-grade sampled children who are receiving special education services will also be asked to complete web surveys about their background and qualifications. They also will be asked to answer questions about the types of services the sampled children who have an IEP receive in a separate child-level survey. Special education teachers of kindergartners will complete the spring kindergarten surveys and special education teachers of first-graders will complete the spring first-grade surveys. Special education teachers of second-graders will be asked to complete spring first-grade surveys only if necessary to meet production goals. In order to have a consistent incentive structure for all teachers participating in a school, special education teachers will also be offered an incentive of $20 plus $7 per child-level survey, and the expectation is that each special education teacher will complete two child-level surveys, on average, for an average incentive of $34 per teacher. The incentives will be included in the package of materials the special education teachers receive from the school coordinator.

During the study activities call, the team leaders will ask the school coordinators to confirm the school administrator information that was collected during the spring recruitment phase. Once the information recorded on the MyECLS website is confirmed, the school administrator will be sent a package with materials describing the ECLS-K:2024 and requesting that he or she complete a self-administered web survey. The package will contain a letter with instructions for accessing the MyECLS website to complete the survey, an ECLS-K:2024 fact sheet, and an incentive check for $25.

Half of the school administrators will complete the spring kindergarten survey, while the other half will complete the spring first-grade survey. Information about the school administrator, the school staff, and the school building will be collected through this survey. All school administrator survey sections can be launched from the school administrator home screen on the MyECLS website. The school administrator will be encouraged to review a description of the sections and determine whether to complete the sections him or herself, or to ask a knowledgeable school staff member to complete select sections. The school administrator will be provided with a secondary login to provide to the designee in this situation. Most of the sections may be completed by a designee, but the study requests that the principal or head administrator complete the section about his/her personal characteristics. These questions appear in a separate section that can be accessed using only the school administrator login.

The school coordinator and team leader for each school will monitor the completion of the school administrator and teacher web surveys on the MyECLS website and prompt staff to complete them. Paper surveys will be provided to school administrators and teachers upon request, as well as non-responders. Team leaders will prompt for the school staff surveys, with assistance from the school coordinator, during the on-site school visit. On the first day of the school visit, the team leader will remind the school coordinator of the need for school staff to complete the surveys, provide paper surveys as necessary, and will collect paper surveys as they are completed. If any surveys still are not completed by the end of the school visit, the team leader will ask for a specific date from the school coordinator and/or school administrator by which the surveys will be completed. The team leader will then visit the school at that later date to collect the remaining surveys and return them to the home office via FedEx. The number of follow-up attempts conducted will depend on field test response rates. Nonresponse materials, including reminder email templates, and reminder text messages, will be sent to teachers who do not complete the teacher surveys online. These materials appear in Attachment A-1.

Parent survey. Parents of all sampled children (kindergarten, first grade, and second grade) will be asked to complete a self-administered web survey on the MyECLS website. Half of the parents of kindergartners will complete the fall kindergarten parent survey, while the other half will complete the spring kindergarten parent survey. Parents of first- and second-grade children will complete the spring first-grade survey. The parents of second-graders are included in the data collection to ensure there is a large enough sample for the incentive experiment. While they will be asked to complete the spring first-grade version of the parent survey, most if not all of the questions will still be applicable and relevant to these parents. Instructions for accessing the MyECLS website to complete the web parent survey will be included with the parent packets distributed by the school coordinator. The school coordinators and team leaders will monitor the completion of web surveys and follow-up with parents as necessary.

For parents who do not complete the survey on their own, the ECLS-K:2024 field staff who conduct the child assessments will also be trained to conduct surveys with parents using the web survey instrument on the MyECLS website. The team leader will assign the nonresponding parent survey cases to assessors to complete via telephone. In these cases, procedures for conducting telephone surveys at times that are most convenient for parents and that allow sufficient flexibility will be used. To establish initial contact with a parent of a sampled child, field staff will be trained to place calls during weekdays, evenings, and weekends. If the initial assessor is unable to make contact and complete the survey, the cases will be assigned to different assessors to attempt to complete the survey. Sometimes having a different assessor call from a different phone number has a positive effect on completion. Depending on response rates, during the last few weeks of data collection, cases that have not yet been contacted or completed may be attempted as in-person surveys by local assessors. Parents will receive a pad of white sticky notes displaying the U.S. Department of Education logo in the parent packet. Some parents will be offered a monetary incentive of $15, pursuant to the incentive experiment (see Section A.9 for details on the incentive experiment).

Nonresponse materials, including reminder email templates, and reminder text messages, will be sent to parents who do not provide parent consent or who do not complete the web parent survey. These materials appear in Attachment A-1.

B.2.3 National State, District, and School Recruitment

This section discusses the district and school recruitment for the ECLS-K:2024 national data collection, beginning in August 2022. For the most part, the recruitment procedures used in the K-1 field test will be followed for the national recruitment. All of the national respondent recruitment materials are contained in Attachment A-2. Note that letters included in this attachment will be printed on the national version of the study letterhead, which is also included in the attachment.

State notification. In the summer of 2022, state school superintendents will be sent a package via FedEx with signature required. A sticker with the U.S. Department of Education seal and the message, “Important information from the U.S. Department of Education.” may be included on the outside of packages. The package will include a letter describing the study and an ECLS-K:2024 fact sheet. To encourage district and school participation in the study, their respective state education agencies will be contacted to inform them about the study. The state testing coordinator and, where applicable, the early childhood education coordinator at the state level will be copied on the state letter. Within 3 days of sending the letter to the state, the state superintendent, state testing coordinator, and, where applicable, the early childhood education coordinator will be contacted to discuss and secure support for the study. Any letters supporting the study received from the state will be included in all mailings to districts and schools within the state.

District recruitment. Beginning in August of 2022, public school districts and Catholic dioceses containing schools selected for the ECLS-K:2024 national study will be contacted, using the same procedures outlined in section B.2.1 for the K-1 field test.

School recruitment. Beginning in August 2022, once district/diocese approval is secured,³² principals at the sampled public and Catholic schools will be contacted using the same procedures outlined in section B.2.1 for the K-1 field test. Private non-Catholic schools will be contacted in August 2022 at the beginning of the field period since district/ diocese approval is not required. In some cases, in-person visits by field recruiters may be necessary to secure school cooperation.

District and school recruitment will continue through the spring of 2023 and into the fall data collection period as needed. Beginning in spring 2023, depending on response rates, the statisticians will select replacement schools that will be recruited using the same methods as the original school sample.

The national study child sampling and data collection activities are not part of this package and will be in the K-1 data collection package, planned for submission in early 2023.

B.3 Methods to Secure Cooperation, Maximize Response Rates, and Deal with Nonresponse

This section describes methods for securing cooperation and maximizing completion rates for the K-1 field test data collection in fall 2022, for which recruitment of schools and districts will be conducted in spring 2022, as well as the national district and school recruitment beginning in fall 2022. A major challenge in any survey today is obtaining high response rates. The main problem associated with nonresponse is the potential for nonresponse bias in the estimates produced using data collected from those people who do respond. Bias can occur when the people who respond are systematically different from the people who do not. The approach that will be used in both the K-1 field test and future national rounds to reduce the potential for bias is to design the data collection procedures and methods so as to reduce nonresponse (e.g., generate engaging targeted respondent materials, contact respondents using preferred methods of communication, provide frequent study updates and preliminary findings, and display materials and study information on an easily accessible respondent website. Responsive design models may also be considered.). While the statistical approaches are important in controlling biases and costs, the recruitment and data collection procedures and materials are at the heart of a successful survey.

B.3.1 Gaining Cooperation from a Variety of Sources

Cooperation issues loom large in any major school-based survey today. The demands of required testing, which have increased since the enactment of No Child Left Behind (NCLB), may reduce time for and willingness to participate in voluntary studies like the ECLS-K:2024, so districts and schools may be increasingly less likely to cooperate. Parents are increasingly skeptical about the value of surveys and non-required tests for their children. Teachers are heavily burdened and often reluctant to spend time on non-teaching activities. Incentives have proven to be effective tools in achieving high response rates, and the study plans to offer monetary incentives to schools, teachers, and parents. The study also plans to offer non-monetary incentives to schools and parents. See Section A.9 of this submission for a full discussion of incentives.

The K-1 field test and national recruitment plan approaches the school as a community. The study aims to establish rapport with the whole community—principals, teachers, parents, and children. The school community must be approached with respect and sensitivity to achieve high initial response rates and, for the national study, maintain cooperation for future rounds of data collection. Moreover, to convey the study’s legitimacy, various respondent materials will include a list of organizations that have endorsed the ECLS project.³³ The ECLS-K:2024 is endorsed by:

Alliance for Excellent Education
American Federation of Teachers
American Montessori Society
Association for Middle Level Education
Association for Montessori International/USA
Association of Christian Schools International
Association of Christian Teachers and Schools
Christian Schools International
Council for Exceptional Children
Council of Chief State and School Officers
Council of Islamic Schools in North America
International Literary Association
Islamic Schools League of America
Lutheran Church-Missouri Synod
National Alliance of Christian Schools
National Association of Elementary School Principals
National Association of Independent Schools
National Catholic Educational Association
National Christian School Association
National Council of Teachers and Mathematics
National Institute for Early Education Research
National Parent Teacher Association
National School Boards Association
National Science Teaching Association
Texas State Teachers Association
The School Superintendents Association
U.S. Conference of Catholic Bishops
WELS Commission on Lutheran Schools

In order to better understand the decision-making process when considering participation in the study, multiple focus groups were held with school administrators (in 2019 and 2020), parents (in 2019 and 2020), and teachers (in 2020) (OMB #1850-0803 v.246 and #1850-0803 v.264). Participants were asked to discuss their reservations about participating in studies such as the ECLS-K:2024. They were also asked to consider the types of information that would be needed before participation and to review and comment on draft recruitment materials. The results were used to revise draft respondent recruitment materials, and draft new materials for use in mailings and on the MyECLS website.

Secure District and School Cooperation for the Field Test

The process by which cooperation is sought for the K-1 field test will be customized based on conditions in the local school systems. For example, for many districts an informational package and a telephone call may be sufficient to secure cooperation, but in some districts, there are other required materials that must be submitted prior to approval. Experiences from recent studies with district contacts and decision-making processes will be used to develop a highly efficient, tailored approach to gaining cooperation from districts. District requirements that district personnel contact the school’s first, that they disseminate the school information packets, or that they receive copies of all communications with schools will be followed and documented. The district and school recruitment process for the K‑1 field test will begin in spring 2022 with recruitment letters being sent to districts requesting their cooperation.

Successful school recruitment for the K-1 field test depends on an experienced field staff to carry out the recruitment contacts. Recruiters will be trained to address a variety of participation concerns, while simultaneously communicating the value of the study. Once district cooperation is secured, schools will receive a package of materials to convey the study’s legitimacy and importance. A sticker with the U.S. Department of Education seal and the message, “Important information from the U.S. Department of Education.” may be included on the outside of packages. Included in this package is a cover letter describing the study and stating the name of the person at the district who has approved (whenever possible) so that the recipient will immediately understand the importance and the widespread support of the study. Providing this information up front will encourage school participation and perhaps speed the schools’ decision-making process. The information in the packet will be presented in a way that is clear and specific, yet concise, and will emphasize that the study team will work closely with schools to accomplish the study with the least burden and disruption possible. The package will also include fact sheets, and various materials for the school administrator. Special letters designed to address specific reasons for refusal at the district and school levels will also be developed. During the recruitment phase, the recruiter will collect school information, identify the school coordinator, ascertain the school’s consent requirements, and schedule the fall assessment date. Collecting these data in advance will allow the school coordinator to focus on the data collection activities in the fall. In-person recruitment visits to large districts to encourage their participation and explain the value of the study will be made as needed.

School recruiters will be provided welcome letters, contact information, nonresponse materials, email templates, and completed packages for special handling districts. These materials form a toolkit that allows the recruiters to customize their contact with schools as needed. During the K-1 field test spring 2022 recruitment and communication with school coordinators the following fall, recruiters will work diligently with schools to facilitate study participation. As school staff increasingly move to digital solutions for their everyday needs, the MyECLS website will allow school coordinators to provide school information, will allow parents to provide consent for their children’s participation, and will allow all respondents to complete their respective web surveys. Hard-copy parent consent materials and school and teacher surveys will be available by request. Principals and school coordinators will be provided materials, including checklists and tip sheets, to promote and encourage participation of school staff and parents, to gather information from relevant staff as necessary, and to easily enter this information on the website.

The data collection task leader will review cases that initially refuse to participate and decide on appropriate strategies to try and convert the refusal, and possibly enlisting the assistance of district or diocesan staff to encourage their participation. The school coordinator for data collection will be provided with all the necessary materials before the study activities call. Best practices from the K-1 field test will be carried over to the national data collection.

Securing Cooperation of Children’s Parents and Teachers for the K-1 Field Test

In the fall of 2022, the K-1 field test focus will shift to securing child, parent, and teacher cooperation. Parent respondent materials will include welcome letters, fact sheets designed specifically for parents, and reminders about survey completion. Consent forms will be available on the MyECLS website, but will be provided on paper for districts and schools that request paper consent forms. Teacher respondent materials include a welcome letter, a welcome email, and a fact sheet tailored specifically for teachers. All respondents will also be provided a thank you email in December 2022. These respondent materials are available in Attachment A-1.

Field staff will discuss the preferred method for distributing respondent materials and contacting parents and teachers about the study. The recommended method will be to push information electronically to parents via the MyECLS website, which can either be done directly by the school coordinator or by the field staff. However, schools can be very reluctant to share parental contact information with study staff. In addition, parents reported in the focus groups that materials sent home from the school have more legitimacy than study materials mailed to their home from Westat, the data collection contractor. Thus, a variety of communications in print and electronic format were developed for school coordinators to use when reaching out to participants in their school. This enables the school coordinator to simply distribute materials rather than take time to draft them as well. Furthermore, in instances where school coordinators agree to share parental contact information, nonresponding parents will be contacted via telephone by local field staff assigned to their case. Field staff can complete the parent survey with the parent via telephone. Additionally, school administrator and teacher surveys are available on paper upon request.

Secure State, District, and School Cooperation for the National Study

The process by which states, districts, and schools are contacted for the national study will be largely the same as during the K-1 field test, with the exception of communication with the state superintendents. For the national recruitment, state superintendents will be mailed study materials and their endorsement for the study will be sought. Due to the much smaller scope of the K-1 field test, this step will not be taken prior to contacting field test districts.

A proactive and intensive approach will be used to secure the cooperation of the states, districts, dioceses to which sampled Catholic schools belong, and schools. Experienced field staff will make in-person visits to schools, present at board meetings, and work with schools to facilitate successful completion of the ECLS-K:2024 study tasks.

As previously noted, the process by which cooperation is sought will be customized based on conditions in the local school systems. The study’s procedures will be flexible enough to address the concerns and needs of states, districts/dioceses, and as schools to the greatest extent possible without compromising the systematic procedures that are essential to high-quality data collection. The additional burden of a longitudinal survey (and the need to communicate clearly to parents and schools the expected burden of participation in a longitudinal survey) makes securing cooperation in the base year even more challenging. The base year must pave the way for concerted follow-up efforts in later rounds by collecting high quality data to help maintain cooperation and track families who move and children who change schools. Securing the cooperation of states and districts/dioceses in the ECLS-K:2024 will be handled by a small group of experienced field recruiters and in-house staff with extensive experience in this capacity. School recruitment for the ECLS-K:2024 national data collection will begin in August 2022.

B.3.2 Methods to Maximize Response Rates

District Recruitment

For both the field test and the national data collections, a successful district recruitment effort is essential to securing adequate school response rates. The main areas of focus will be timing of contact, research proposals, obtaining formal approval, and refusal conversion.

Timing. All districts will be contacted early in the recruitment phase, as securing district cooperation early will allow sufficient time for recruiting schools. Accordingly, early contacts will allow the study to identify barriers to cooperation, such as required research proposals and districts requiring refusal conversion. For the ECLS-K:2011 base year kindergarten, 2010-11 school year), a 77 percent district-level school cooperation rate was obtained. However, it is expected that district recruitment to be more challenging in the current environment of the ECLS-K:2024.

Research Proposals. Field recruiters will first contact districts requiring special research proposals to ensure that the study is able to meet deadlines and that study staff preparing the proposals are using the required applications and procedures. Once submitted, applications will be closely tracked, with regular field follow-up with appropriate district personnel on the status. Once the study is approved, recruiters will review any special requirements within the district research proposal and response to ensure the study is compliant. For example, some districts will not allow certain schools to be contacted or will not allow for school and teacher incentive payments. These requirements will be documented so that school recruiters and, eventually, team leaders, are aware. Whenever possible, the recruiter who worked with the district will also recruit the schools within that district.

Formal Approval. For all districts, every effort will be made to receive formal approval from a district staff member, such as the superintendent, deputy superintendent, testing director, or early childhood education director. Experience has shown that, when contacted, schools usually ask who at the district-level has approved the study. Being able to respond affirmatively with a specific name and title will facilitate the school recruitment process.

Refusal Conversion. District refusal conversion efforts will begin early, using all available methods, including in-person visits and presentations. In addition to allowing field recruiters to contact schools, cooperative districts will be encouraged to support the study and encourage their schools to participate.

Parent Instruments

There are five areas that can be focused on during the K-1 field test in order to maximize completion rates for the various respondent instruments in this field test: (1) easily accessible study materials and surveys, (2) instrument length, (3) design of questions, (4) non-English instruments, and (5) avoiding refusals and converting initial refusals to completed interviews. For purposes of historical comparisons, in the ECLS-K:2011’s base year (kindergarten, 2010-11 school year), response rates for parents were 80 percent for fall or spring participation, or 50 percent for the overall base year response rate taking into account the schools’ response rate.

Easily accessible study materials. As previously noted, the use of the MyECLS website provides respondents with an easily accessible format to access materials and complete their respective surveys. Unlike previous rounds of the ECLS-K and ECLS-K:2011 where interviews with various respondents needed to be scheduled, the current cohort’s focus on digital solutions minimizes the restrictions respondents face with completing voluntary tasks (work, classes, recreational activities, vacations, etc.). Completion rates may improve when respondents have the flexibility to access and complete materials as needed, with guidance and oversight from their school coordinator and/or ECLS-K:2024 field staff.

Instrument length. Overall, instrument design occurs with the knowledge that completion rates typically are higher when the survey is shorter in length. The average length of time to complete items in the ECLS-K:2011 was carefully considered during the design phase of the ECLS-K:2024 to determine if questions should be cut entirely or simplified to keep surveys to the desired time. At the time of this submission, some study instruments have been determined to be too lengthy, so in the interest of reducing respondent burden in the field test, not every section of every instrument will be fielded with every sample member. Additionally, one purpose of the longitudinal study is to build understanding of change and/or stability within families in part by asking the same questions over time. However, the K-1 field test is a cross-section of students in kindergarten, first, and second grade. Therefore, it is not necessary to field the same questions across all students’ parents. Attachments B, C, D, and E provide the full study instruments, though some of these items may not be administered to particular respondents in the interest of minimizing respondent burden. Programming instructions to skip sections of items were added to the instruments in the Attachments so that only a subset of items from any given instrument may be administered to any one respondent.

Design of questions. As much as possible, the survey is streamlined so the questions included are straightforward and easy for respondents to answer. In addition, skip patterns based on answers to questions that appear earlier in the survey are built into the survey so that not all questions need to be asked of all respondents. The planned instrument usability testing may also inform the type and format of questions that are included in the K-1 field test. If any revisions are made to survey instruments to take into account issues found during programming, the revised instruments will be included in the change request submitted to OMB in June of 2022.

Non-English materials. To achieve high response rates, it is important that study procedures work to include non-English speaking parents to the greatest extent possible. As described in the data collection procedures, web and mailed versions of all parent recruitment materials (including the reminder messages) will be available in English and in Spanish. The study’s introductory letter and consent forms will also be available in Mandarin. The field staff and school coordinator will discuss the home language of the participating parents; if warranted, the Spanish or Mandarin materials will be included in the mailing, alongside the English materials.

Refusal avoidance and conversion procedures. Achieving an acceptable parent response rate will require active and effective refusal conversion efforts. This activity must begin as soon as the parents receive access to their consent materials, whether they be digital or physical. A key factor in converting refusals is the ability of the team leaders and assessors to clearly and confidently convey the purpose and importance of the study and the benefits that will be derived from it. This will be a focus of the field staff training. The training materials for averting refusals include information about becoming thoroughly familiar with the study, including answers to frequently asked questions (FAQs) and respondent objections, drafting responses in the respondent’s own words to FAQs, practicing saying these responses, and diagnosing respondent objections and quickly responding with a response tailored to the objection. The training includes self-analysis by recording responses and listening to them, preparing answers for different situations, using the voice effectively, and role-plays between trainers and field staff and between paired field staff, with one member of each pair acting as the respondent. Averting refusal training will focus specifically on addressing reasons for refusals on the parent survey component of the ECLS-K:2024. If a refusal occurs, the field staff will be instructed to record key demographic information about the refusing respondent (e.g., sex, approximate age) and the respondent’s reason(s) (if given) for refusing to participate. This information will be evaluated by the field supervisor to determine the best strategy for converting refusals. Cases identified for refusal conversion will be assigned to a select group of field staff identified as possessing the necessary skills to act as refusal converters. Field managers will hold telephone conferences with the identified field staff to review the refusal conversion procedures and discuss strategies for converting refusals.

Child Assessments

The areas that can be focused on in order to maximize completion rates for the child assessments in the K-1 field test involve working with schools and teachers to schedule children for assessment at convenient times, keeping the assessment length as short as possible so that children do not miss too much class time, and conducting make-up assessments with consented children who are absent on the scheduled ECLS school visit. Although untimed, the field test assessment is expected to last approximately 45 minutes. Field staff will work with teachers to determine if the child is able to take the assessment during one session, or if the assessment should be broken up to better accommodate the child’s or teacher’s schedule. For example, the reading assessment could be administered in the morning, while the math assessment administered in the afternoon. The goal is to be as flexible as possible to minimize classroom and learning disruptions. ECLS response rates at the child level have generally been high when enacting such refusal conversion techniques. For example, in the ECLS-K:2011 base year (kindergarten, 2010-11 school year), weighted student unit response rate for fall or spring participation was 89 percent. The overall base year response rate (i.e., the school rate times the child rate) was 56 percent.

Absent children. It is expected that some consented children will be absent from school during the scheduled ECLS school visit. Days will be set aside throughout the field period in which some field staff have no assessments scheduled, so that make-up assessments can be more easily conducted. Make-up assessments will be conducted as necessary, depending on whether or not sufficient cases were assessed during the scheduled school visit.

School and Teacher Instruments

In addition to the easily accessible study materials and surveys, instrument length, and consideration of questions detailed in the Parent Instrument section, the efforts made by supervisory staff to avoid refusals and to convert initial refusals to cooperating respondents is needed in order to maximize completion rates for the school and teacher instruments in the K-1 field test. In the ECLS-K:2011’s base year (kindergarten, 2010-11 school year), the school-level response rate was 63 percent. Teacher response rates varied by instrument (i.e., child-level and teacher-level instruments for both fall and spring kindergarten). In the base year, the weighed unit response rates varied from 80 to 85 percent and the overall response rates (i.e., school rate times teacher rate) varied from 50 to 53 percent.

Refusal avoidance and conversion procedures. Much like the averting refusal training described above for ECLS-K:2024 parent survey, the training session for refusal aversion will include analyzing the reasons for refusal, preparing answers for different situations, using the voice effectively, and role-play situations.

Special considerations in obtaining cooperation. District and school personnel have stated that they face increasing demands upon their schools for a variety of non-instructional activities, including requirements for state and district assessment. Sensitivity to these concerns is essential to gaining cooperation for the ECLS-K:2024, and it must be made clear to school system personnel at all levels that the ECLS-K:2024 staff is more than willing to work with them to facilitate their participation with the least burden and disruption possible.

B.4 Tests of Methods and Procedures

The previous kindergarten cohorts of the ECLS, the classes of 1998-99 and 2010-11, have informed the design of the ECLS-K:2024. By design, the ECLS-K:2024 data collection instruments are in large part a collection of items used in the ECLS-K and ECLS-K:2011, to allow comparisons between the various cohorts of kindergartners. However, because there are significant changes to the ECLS-K:2024 instrumentation and study procedures, namely the move from paper instruments or phone interviews to data collections on web, a field test for the kindergarten and first-grade rounds of ECLS-K:2024 is included in this request for OMB approval. The results of this K-1 field test will inform instrument design as well as the sampling and operational procedures employed for the national kindergarten and first-grade data collections, planned to take place in the fall of 2023, the spring of 2024, and the spring of 2025. Districts and schools will be contacted in spring 2022 to recruit them to participate in the fall 2022 field test data collection. Instruments, sampling, and operational procedures will be tested to ensure successful data collection in the national study.

A request to conduct usability testing of the kindergarten and first-grade field test survey instruments was approved by OMB (OMB #1850-0803 v280). Results of the usability testing were used to revise the kindergarten and first-grade field test instruments. The K-1 field test findings will be summarized in the request to conduct the fall 2023, spring 2024, and spring 2025 national kindergarten and first-grade rounds.

B.5 Individuals Responsible for Study Design and Performance

The following individuals are responsible for the study design and the collection and analysis of the data for the ECLS‑K:2024.

Jill McCarroll, NCES	(202) 304-2920	Liz Bissett, Westat	(301) 294-4414
Gail Mulligan, NCES	(202) 245-8413	Nancy Vaden-Kiernan, Westat	(512) 351-8270
Chris Chapman, NCES	(202) 245-7103	Michael Brick, Westat	(301) 294-2004

1 To be counted as a base year complete, the study must obtain child- or parent-level data during kindergarten.

2 In the ECLS-K:2024, there will be data from both parents and schools about IFSPs and IEPs. If a child has an IFSP/IEP based on either school or parent report in any round, the child will be followed with certainty. In the ECLS-K:2011, parents were asked about IFSPs, but not IEPs.

3 Estimated using Census race/ethnicity estimates of the population of children under the age of five, after accounting for the oversample of APIs.

4 Includes transitional kindergartens and ungraded classrooms that educate children of kindergarten age.

5 The frame of PSUs will be created prior to the release of county-level data from the 2020 Census. The 2018American Community Survey contained the most recent national estimates and was therefore used to define the PSUs for the ECLS-K:2024. Although in 2021 it was determined that the study’s data collection activities would be delayed by 1 year (i.e., the national base year was originally 2022-23 but was delayed 1 year due to the COVID-19 pandemic), the sampling of the national and field test PSUs occurred in 2020 before the delay was established, and will not be repeated.

6 This population size was selected to potentially yield a large enough sample size of kindergarten-aged children. If upon creation a PSU is still too small it will be combined with an adjacent PSU.

7 For the ECLS-K:2024 this number is expected to be 369. This is based on the calculation (10 * 21) / .57, where 10 is the average number of responding schools in a PSU, 21 is the average number of sampled kindergartners in a school, and .57 is the estimated private school response rate. The private school response rate is used for this estimate because it is lower than the expected response rate for public schools and is, therefore, more conservative.

8 This is the same number of PSUs as in the ECLS-K design. The ECLS-K:2011 design had fewer PSUs, however when designing the ECLS-K:2024, given the changes in the design and potential changes in response rates, using the larger number of PSU’s provides extra flexibility.

9 These issues include schools having missing or erroneous child race/ethnicity data or child race/ethnicity data that do not comply with OMB definitions.

10 Maximum within PSU distance of 100 miles.

11 If NAEP does not create the 2023 frame (e.g., due to changes in their data collection schedule as a result of the coronavirus pandemic), the ECLS-K:2024 will likely use the 2022 NAEP private school frame. Other resources may also be investigated.

12 Schools will be sorted by percent API but not always from smallest to largest within the size class. For example, in size class A, schools will be sorted from smallest to largest percent API. In size class B, schools will be sorted from largest to smallest percent API. In size class C, schools will be sorted again from smallest to largest percent API, etc. The list of sorted schools will then be put together before selection.

13 In the ECLS-K:2011 this value was equal to 13.

14 Westat staff will discuss the type of parental consent that is required, explicit or implicit, with the participating schools’ principals or coordinators.

15 Children will be randomly assigned to one of the reading assessment forms and one of the math assessment forms. This model will provide the needed yield for each item required for the analysis to select the final set of items for the national assessments. Although a second-grade data collection is not planned for the national study, second-graders are included in the K-1 field test to allow the study to develop a broad enough assessment in terms of the spectrum of ability and guard against a ceiling effect. Additionally, their parents will be included in the field test to allow for a large enough sample for the parent incentive experiment.

16 Primary and special education teachers of second-graders will not complete surveys because the K-1 field test does not include second-grade non-assessment instruments. However, if the target sample sizes of teachers is low, primary and/or special education teachers of the sampled second-graders may also be asked to participate in some schools.

17 Parents of sampled kindergarteners, first-, and second-graders will be asked to complete a parent survey. While the teachers of the second-graders are not asked to participate in the K-1 field test unless they are needed to supplement the sample size, parents of second-graders are included in the field test to provide enough sample for the parent incentive experiment.

18 Included in Attachment A-1 are letters and emails for nonresponding districts, schools, parents, and school staff. Note that there is text included in the district and school recruitment materials acknowledging the coronavirus pandemic. This text will be used only if needed, dependent on the state of the pandemic when the letters are to be used. Additional revisions to respondent materials regarding the coronavirus pandemic may be needed at a later date, dependent on the progress of the pandemic. If additional revisions are needed, these will be included in a future change memo. Several infographics displaying results from previous ECLS cohorts, as well as from other applicable NCES studies, have been developed for these materials. In order to provide the recruiting or field staff the discretion to choose an appropriate infographic based on the respondent’s specific concern, each of these materials indicates the possible infographics that may be included on the letters. All infographics are provided in Attachment A5.

19 Text regarding the coronavirus pandemic is included for the log-in screen of the MyECLS website (see Attachment F). This text will be used only if and when needed, dependent on the state of the pandemic.

20 Schools that indicate a need for the parent introductory letter and consent forms in Mandarin will have the electronic consent function on MyECLS turned off and only paper consent forms, including those in Mandarin, will be used.

21 Placeholders for the infographics are contained in the relevant respondent materials in Attachments A-1, A-2 (the national recruitment materials), and F (the MyECLS website screens). Infographics are available for review in Attachment A-5.

22 The one-page fact sheet will be included in the packages if the impact of the coronavirus pandemic is still a concern for the study, the districts, and/or the schools at the time of recruitment. If it is determined that there is no longer a need for the fact sheet it will not be included in the mailings.

23 Given the time constraints for the K-1 field test, there may be districts that are notified about the study, but for which formal district approval is not received during the allotted recruitment period. In these cases, the schools will be contacted directly and informed that the district has been notified.

24 If needed due to the coronavirus pandemic, personal protective equipment (PPE), such as face masks, will be used during the administration of the assessment.

25 It is not anticipated that primary and special education teachers of second-graders will complete surveys because the K-1 field test does not include second-grade instruments. Second-grade teachers may be asked to participate to meet sample-size targets dependent on response rates during the field period. Kindergarten and first-grade teachers who do not teach participating children may also be asked to participate to meet sample-size targets dependent on response rates during the field period.

26 Note that screenshots for the MyECLS website are included in this submission. The website is currently in programming and some small changes may be necessary as further review and testing occur. If needed, a change memo that includes screenshots of the updated MyECLS website is planned for June 2022.

27 IEP information might be deemed too sensitive for the school to be willing to provide it on the child list used for sampling. Therefore, IEP information for sampled children will be collected during the study activities call.

28 If needed due to the coronavirus pandemic, the team leader will also discuss safety precautions to be implemented for the school visit. The study staff will be trained to follow the latest safety measures in place by the study as well as additional safety precautions required by the district and/or school.

29 After returning a child to the classroom, assessors will record their observations about the child’s behaviors and response to the assessments and any general notes for their diary of field test experiences before retrieving the next child on the list for assessment.

30 If teacher sample size or survey response rates are low, other teachers in the schools (including special education teachers), who are not linked to sampled children, may be asked to complete surveys. Additionally, teachers of the sampled second-grade students may also be asked to complete surveys.

31 If teacher sample size or survey response rates are low, second-grade teachers may be asked to complete surveys.

32 Every effort will be made to obtain district permission to contact the sampled schools. This may mean delays in contacting schools as the decision is weighed at the district or applications for special handling districts are completed and reviewed. However, if after a set amount of time (e.g., several months of nonresponse after repeated contact attempts), a district remains nonresponsive, contact with the relevant school(s) in the district will be initiated.

33 If additional study endorsements are received prior to the field test, the list of endorsers will be updated in the June 2022 change request/change memo.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	2021-12-30