Part B School Pulse Panel v4

The U.S. Census Bureau will collect the School Pulse Panel (SPP) data on behalf of NCES. Data collection will be a self-administered, online survey. The survey is estimated to require, on average, 30 minutes of school staff time each month. The sampled school will be offered a reimbursement of $400 each month for their participation in the study over the course of 12 months. The reimbursement will be paid out monthly in the form of debit cards. If a school district does not permit its schools to receive any form of incentive, the reimbursement will be sent to a point of contact in the district, or the reimbursement will be withheld. Principals and other school staff most knowledgeable about COVID-19 impacts on the school environment and instructional offerings can help complete the survey. No classroom time is involved in the completion of this survey.

The resulting data will provide aggregate estimates for public schools across the nation. A stratified sample design was used to select approximately 1,200 U.S. public schools. In addition, a reserve sample of replacement schools was selected to boost the number of responses if any schools from the initial sample do not respond. The sample is designed to provide national estimates of primary, middle, and high schools taking into account the type of locale (urbanicity) and racial/ethnic student enrollment.

B1. Respondent Universe and Sample Design and Estimation

The sampling frame for the School Pulse Panel is derived from the National Teacher and Principal Survey (NTPS) 2020-21 frame, which itself is largely derived from the 2018-19 Common Core of Data (CCD), the file of public schools supplied annually by State educational agencies to NCES. Only public schools in the 50 states and the District of Columbia will be included in the School Pulse Panel sampling frame, though the School Pulse Panel may sample Puerto Rico separately at a later date. Certain types of schools are excluded, including newly closed schools, home schools, virtual schools, ungraded schools, private schools, and schools with a high grade of kindergarten or lower. Regular public schools, charter schools, alternative schools, special education schools, vocational schools, correctional facilities/juvenile justice facilities, and schools that have partial or total magnet programs are included in the frame. For sample allocation purposes, strata are defined by instructional level, and the sample is sorted primarily on the type of locale (urbanicity), percent minority enrollment, and geographic region. Beginning in August 2022, data will also be collected universally (i.e., all schools) from the outlying areas (American Samoa, Guam, Commonwealth of Northern Mariana Islands, and U.S. Virgin Islands).

Tables 1 and 2 show the estimated expected distribution of the public-school sampling universe for the School Pulse Panel, by school level and urbanicity and by school level and percent minority enrollment, respectively.

Table 1. Expected respondent universe for the SPP sample, by school level and region, based on the 2020-21 NTPS Frame

Region	Elementary	Middle/Combined/Other	High/Grade 9-11	Total
Northeast	8,450	2,969	3,443	14,862
Midwest	12,468	4,461	6,051	22,980
South	18,075	7,480	8,157	33,712
West	13,306	4,805	5,813	23,924
Total	52,299	19,715	23,464	95,478

Table 2. Expected respondent universe for the SPP sample, by school level and urbanicity, based on the 2020-21 NTPS Frame

Urbanicity	Elementary	Middle/Combined/Other	High/Grade 9-11	Total
City	15,650	5,031	6,332	27,013
Suburb	18,007	6,471	6,355	30,833
Town	5,823	2,942	3,687	12,452
Rural	12,819	5,271	7,090	25,180
Total	52,299	19,715	23,464	95,478

Table 3. Expected respondent universe for the SPP school sample, by school level and percent minority enrollment, based on the 2020-21 NTPS Frame

Percent Minority	Elementary	Middle/Combined/Other	High/Grade 9-11	Total
0 to less than 25	15,487	5,975	7,423	28,885
25 to less than 50	11,220	4,508	4,435	20,163
50 to less than 75	8,884	3,529	3,576	15,989
75+	16,708	5,703	8,030	30,441
Total	52,299	19,715	23,464	94,578

Sample Selection and Response Rates

A stratified sample design was used to select approximately 1,200 U.S. public schools. The sample was designed to provide national estimates of primary, middle/combined, and high schools taking into account the type of locale (urbanicity), racial/ethnic student enrollment, and region. Note that combined schools will be grouped with middle schools for the purposes of measurement and estimation.

There are two stages of sample selection. A base sample of 1,200 schools was drawn as an initial stage as well as another reserve sample (second stage) of 1,200 schools with similar characteristics to our base sample. In case we do not get the necessary number of schools to participate out of the base sample, we plan to reach out to the reserve sample schools to complete the panel.

Roughly 120 Outlying Areas schools are to be added to the SPP data collection starting in August 2022. It is planned for the collection of these schools to be a universe collection, so they are separated from the sample of stateside schools. All study operations will be identical for the collection from these schools.

Recruitment for the panel study began in July and August 2021. The initial goal was to obtain commitments from1,000 schools to participate in the study throughout the duration of the year-long monthly collections (roughly a 70 to 80 percent response rates). However, initial recruitment efforts were not as successful as anticipated. Therefore, the reserve sample schools were included in recruitment efforts in order to try to get enough responding schools to be able to report out accurate and reliable national estimates. This will likely result in an overall response rate of around 20-30 percent.

Note that there are no previous administrations of the School Pulse Panel, so we cannot derive estimates of response from those.

Sample Design for the School Pulse Panel

The main objective of the School Pulse Panel sampling design is to obtain overall subgroup estimates broken out by various school characteristics. For sample allocation and sample selection purposes, strata were defined by instructional level. In addition, region, locale, percent minority enrollment, enrollment size, and charter status were used as implicit stratification variables by sorting schools by these variables within each stratum before sample selection. The explicit stratification and the first three implicit stratification variables (region, locale, and percent minority enrollment) are priorities for evaluation for this panel. The method determined to allocating schools to the different sampling strata is allocate them proportionally to the U.S. public school population.

Note that there are no experiments planned within the sample, in which different schools in sample may be given different content or pathways.

Calculation of Weights

Weights will be attached to each surveyed school so that the weighted data will represent population levels. The final weight for completed cases will be composed of a sampling base weight and an adjustment for nonresponse. Nonresponse weighting adjustment cells for the SPP data will be determined using a categorical search algorithm called Chi-Square Automatic Interaction Detection (CHAID). CHAID begins by identifying the school-level characteristics of interest that are the best predictors of response. It divides the dataset into groups so that the unit response rate within cells is as constant as possible and the unit response rate between cells is as different as possible. The characteristics of interest as predictors of response must be available for both respondents and nonrespondents in order to conduct a CHAID analysis, and, in the case of SPP, will be available through the CCD sampling frame. The final, adjusted weights will be raked so that the sum of the weights matches the number of schools derived from the School Pulse frame.

Methods for Variance Estimation

Standard errors of the estimates will be estimated using jackknife replication. Replicate codes that indicate the computing strata and the half-sample to which each sample unit belongs will be developed, as will the weights for all replicates that were formed in order to calculate variances.

B2. Procedures for the Collection of Information

The School Pulse Panel data collection initially began in September of 2021. Due to low response at that time, the SPP was paused until January 2022 to focus on recruitment and to build a more robust panel with committed schools. Over the course of November and December, NCES restarted recruiting sampled schools by informing them of the reduced burden for the survey and asking them to complete a screener that captures pertinent information regarding contact materials and a mailing address for the debit cards.

The U.S. Census Bureau, acting as a contractor for NCES, is responsible for the data collection operation. For each month, each sampled school will receive a letter and an email notifying them of the survey, which will include information necessary to complete the online questionnaire. Respondents will have a two-week window to respond to the survey. Reminder emails will be sent during the data collection window.

Due to the expedited development of this study and release of data, NCES considers the SPP data collections as experimental. Therefore, NCES data quality standards may not be met and estimates will be released on the SPP dashboard (https://ies.ed.gov/schoolsurvey/) within 2-4 weeks from the end of data collection. This will be repeated monthly through December of 2022. This experimental approach will continue for as long as necessary to fulfill EO 14000.

Drawing the Sample

The samples of schools were drawn in the summer of 2021. Many districts (known as “special contact districts”) require research applications to be submitted and reviewed for approval before contact can be made with schools within their districts. Special contact district outreach was cleared in July (OMB# 1850-0963 v.3) and recruitment has continued to date.

School Communication

The School Pulse Panel is conducted via a self-administered web-based survey instrument. A clerical operation prior to data collection was conducted to obtain e-mail addresses for all the sampled school principals, and these e-mail addresses are used to contact the school principals throughout the data collection. For each month, an initial letter and initial email are sent requesting their participation in the study and to inform schools about reimbursements of $400 a month offered to schools over the course of 12 months for their continued participation in the study. Reminder e-mails are sent throughout the two-week data collection window.

A copy of the letters and e-mails to be sent to school principals throughout the SPP data collection are included in appendix A.

Protocol for Follow-up Calls

During the monthly data collections, Census Bureau staff will initiate phone calls with nonrespondents, reminding them to complete their questionnaire.

Refusal Conversion for Schools That Will Not Participate

If a school expresses strong concerns about confidentiality at any time during data collection, these concerns will be directed to the Census Project Director (and possibly to NCES) for formal assurance. All materials will include the project’s toll-free number. In addition, initial emails will include information about why the participation of each sampled school is important and how respondent information will be protected.

B3. Methods to Maximize Response Rates

NCES is committed to obtaining a response rate that allows for reporting national-level estimates on a monthly basis. In general, a key to achieving this is to track the response status of each sampled school, with telephone follow-up, as well as follow-up by e-mail, of those schools that do not respond promptly. To help track response status, survey responses will be monitored through an automated receipt control system. Telephone interviews will be conducted only by Census Bureau interviewers who have received training in general telephone interview techniques as well as specific training for SPP.

It is estimated that the survey will require, on average, about 30 minutes of school staff time each month. To encourage study participation, the sampled school will be offered a reimbursement of $400 a month for their participation in the study over the course of 12 months.

B4. Tests of Procedures

Cognitive Testing

As part of the development of the SPP, the monthly instrument starting in September underwent 20 cognitive testing with school and district administrators during the summer of 2021. The cognitive testing focused on items pertaining to the COVID-19 pandemic that caused widespread school closures, and significant changes to school policies and disruptions to their delivery of instruction to students in 2020, 2021, 2022. Iterative modifications were made throughout the testing to improve clarity and comprehension. Items in the September instrument were used as part of the January, February, and March survey instruments. Additional items and modification to items were cleared through change requests. Final instruments for January, February, and March, as well as proposed items to be included on the April, May, and June instruments are included in Appendix B.

B5. Individuals Responsible for Study Design and Performance

Several key staff responsible for the study design and performance of the School Pulse Panel. They are:

Rachel Hansen, Project Director, National Center for Education Statistics
Rebecca Bielamowicz, National Center for Education Statistics
Ryan Iaconelli, National Center for Education Statistics
Chris Chapman, National Center for Education Statistics
Andrew Zukerberg, National Center for Education Statistics
Cassandra Logan, U.S. Census Bureau
Elke McLaren, U.S. Census Bureau
Aaron Gilary, U.S. Census Bureau
Alfred Meier, U.S. Census Bureau
Kathleen Kephart, U.S. Census Bureau
Jessica Holzberg, U.S. Census Bureau

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Kathryn.Chandler
File Modified	0000-00-00
File Created	2022-08-04