Part B School Pulse Panel

The U.S. Census Bureau will collect the School Pulse Panel (SPP) data on behalf of NCES. Data collection will be a self-administered, online survey. It is estimated for the survey to require up to 2-3 hours of staff time each month. The sampled school will be offered up to a total reimbursement of $5,000 for their participation in the study over the course of 12 months (including September 2021 through August 2022). The reimbursement will be paid out quarterly in the form of a debit card and will be prorated to include the dollar amount that reflects the number of months of participation for that quarter. Additionally, a non-monetary incentive (tote bag) will be included in the November communications. If a school district does not permit its schools to receive any form of incentive, the reimbursement will be sent to a point of contact in the district or the reimbursement and any non-monetary incentives will be withheld. Principals, other school staff most knowledgeable about COVID-19 impacts on the school environment and instructional offerings, and district staff can help complete the survey. No classroom time is involved in the completion of this survey.

The resulting data will provide aggregate estimates for public schools across the nation. A stratified sample design was used to select approximately 1,200 U.S. public schools. In addition, a reserve sample of replacement schools was selected to boost the number of responses if any schools from the initial sample do not respond. The sample is designed to provide national estimates of primary, middle, and high schools taking into account the type of locale (urbanicity) and racial/ethnic student enrollment.

B1. Respondent Universe and Sample Design and Estimation

The sampling frame for the School Pulse Panel is derived from the National Teacher and Principal Survey (NTPS) 2020-21 frame, which itself is largely derived from the 2018-19 Common Core of Data (CCD), the file of public schools supplied annually by State educational agencies to NCES. Only public schools in the 50 states and the District of Columbia will be included in the School Pulse Panel sampling frame, though the School Pulse Panel may sample Puerto Rico separately at a later date. Data may also be collected from the outlying areas (American Samoa, Guam, Commonwealth of Northern Mariana Islands, and U.S. Virgin Islands) at the district-level. Certain types of schools are excluded, including newly closed schools, home schools, virtual schools, ungraded schools, private schools, and schools with a high grade of kindergarten or lower. Regular public schools, charter schools, alternative schools, special education schools, vocational schools, correctional facilities/juvenile justice facilities, and schools that have partial or total magnet programs are included in the frame. For sample allocation purposes, strata are defined by instructional level, and the sample is sorted primarily on the type of locale (urbanicity), percent minority enrollment, and geographic region.

Tables 1 and 2 show the estimated expected distribution of the public-school sampling universe for the School Pulse Panel, by school level and urbanicity and by school level and percent minority enrollment, respectively.

Table 1. Expected respondent universe for the SPP sample, by school level and region, based on the 2020-21 NTPS Frame

Region	Elementary	Middle/Combined/Other	High/Grade 9-11	Total
Northeast	8,450	2,969	3,443	14,862
Midwest	12,468	4,461	6,051	22,980
South	18,075	7,480	8,157	33,712
West	13,306	4,805	5,813	23,924
Total	52,299	19,715	23,464	95,478

Table 2. Expected respondent universe for the SPP sample, by school level and urbanicity, based on the 2020-21 NTPS Frame

Urbanicity	Elementary	Middle/Combined/Other	High/Grade 9-11	Total
City	15,650	5,031	6,332	27,013
Suburb	18,007	6,471	6,355	30,833
Town	5,823	2,942	3,687	12,452
Rural	12,819	5,271	7,090	25,180
Total	52,299	19,715	23,464	95,478

Table 3. Expected respondent universe for the SPP school sample, by school level and percent minority enrollment, based on the 2020-21 NTPS Frame

Percent Minority	Elementary	Middle/Combined/Other	High/Grade 9-11	Total
0 to less than 25	15,487	5,975	7,423	28,885
25 to less than 50	11,220	4,508	4,435	20,163
50 to less than 75	8,884	3,529	3,576	15,989
75+	16,708	5,703	8,030	30,441
Total	52,299	19,715	23,464	94,578

Sample Selection and Response Rates

A stratified sample design was used to select approximately 1,200 U.S. public schools. The sample was designed to provide national estimates of primary, middle/combined, and high schools taking into account the type of locale (urbanicity), racial/ethnic student enrollment, and region. Note that combined schools will be grouped with middle schools for the purposes of measurement and estimation.

There are two stages of sample selection. A base sample of 1,200 schools was drawn as an initial stage as well as another reserve sample (second stage) of 1,200 schools with similar characteristics to our base sample. In case we do not get the necessary number of schools to participate out of the base sample, we plan to reach out to the reserve sample schools to complete the panel.

Recruitment for the panel study began in July and August, 2021. An initial goal was set to obtain 1,000 schools to commit to participate in the study throughout the duration of the year-long monthly collections (roughly a 70 to 80 percent response rates). However, current recruitment has not been as successful as anticipated. Therefore, it has been determined to begin recruiting the reserve sample schools in order to try to get enough responding schools to be able to report out accurate and reliable national estimates. This will likely result in an overall response rate of around 20-30 percent.

Note that there are no previous administrations of the School Pulse Panel, so we cannot derive estimates of response from those.

Sample Design for the School Pulse Panel

The main objective of the School Pulse Panel sampling design is to obtain overall subgroup estimates broken out by various school characteristics. For sample allocation and sample selection purposes, strata were defined by instructional level. In addition, region, locale, percent minority enrollment, enrollment size, and charter status were used as implicit stratification variables by sorting schools by these variables within each stratum before sample selection. The explicit stratification and the first three implicit stratification variables (region, locale, and percent minority enrollment) are priorities for evaluation for this panel. The method determined to allocating schools to the different sampling strata is allocate them proportionally to the U.S. public school population.

Note that there are no experiments planned within the sample, in which different schools in sample may be given different content or pathways.

Calculation of Weights

Weights will be attached to each surveyed school so that the weighted data will represent population levels. The final weight for completed cases will be composed of a sampling base weight and an adjustment for nonresponse. Nonresponse weighting adjustment cells for the SPP data will be determined using a categorical search algorithm called Chi-Square Automatic Interaction Detection (CHAID). CHAID begins by identifying the school-level characteristics of interest that are the best predictors of response. It divides the dataset into groups so that the unit response rate within cells is as constant as possible and the unit response rate between cells is as different as possible. The characteristics of interest as predictors of response must be available for both respondents and nonrespondents in order to conduct a CHAID analysis, and, in the case of SPP, will be available through the CCD sampling frame. The final, adjusted weights will be raked so that the sum of the weights matches the number of schools derived from the School Pulse frame.

Methods for Variance Estimation

Standard errors of the estimates will be estimated using jackknife replication. Replicate codes that indicate the computing strata and the half-sample to which each sample unit belongs will be provided, as will the weights for all replicates that were formed in order to calculate variances.

B2. Procedures for the Collection of Information

The School Pulse Panel data collection will begin in September of 2021¹. The U.S. Census Bureau, acting as a contractor for NCES, will handle the data collection. Starting in September, each sampled school will receive an email notifying them of the full survey, which will include information necessary to complete the online questionnaire. Respondents will have a two-week window to respond to the survey. Reminder emails will be sent during the data collection window. Data will be formally reviewed for disclosure prevention and released a several weeks after data collection ends for that month. This will be repeated monthly through August of 2022.

Drawing the Sample

The samples of schools were drawn in the summer preceding data collection. Many districts (known as “special contact districts”) require research applications to be submitted and reviewed for approval before contact can be made with schools within their districts. Special contact district outreach was cleared in July (latest change request in August, OMB# 1850-0963 v.3) and recruitment has continued through August, 2021.

School Communication

The School Pulse Panel will be conducted via a self-administered web-based survey instrument. A clerical operation prior to data collection was conducted to obtain e-mail addresses for all the sampled school principals, and these e-mails will be used to contact the school principals throughout the data collection. During the recruitment process (cleared in July 2021; last change request approved in August OMB# 1850-0963 v.3), an initial letter was sent via e-mail in July 2021 to notify sampled schools of their selection for the survey, to verify contact information, to inform schools about reimbursements of up to $5,000 offered to schools over the course of 12 months for their continued participation in the study, and also included a link to a short preliminary summer questionnaire. Reminder e-mails have continued to be sent throughout August 2021. A new invitation e-mail will be sent in September, and each month thereafter, to distribute instructions on how to complete the monthly web questionnaire, including the survey URL to access the survey online (Attachment A). Physical letters will be sent in October and November in advance of the monthly collections to assist with legitimizing the study and bring attention to the study, in general.

The sampled school will be offered up to a total reimbursement of $5,000 for their participation in the study over the course of 12 months (including September 2021 through August 2022). The reimbursement will be paid out quarterly in the form of a debit card and will be prorated to include the dollar amount that reflects the number of months of participation for that quarter. Additionally, a non-monetary incentive (tote bag) will be included in the November communications. If a school district does not permit its schools to receive any form of incentive, the reimbursement will be sent to a point of contact in the district or the reimbursement and any non-monetary incentives will be withheld.

A copy of the letters and e-mails to be sent to school principals throughout the SPP data collection are included in appendices A.

Protocol for Follow-up Calls

During the monthly data collections, Census Bureau staff will initiate phone calls with nonrespondents, reminding them to complete their questionnaire.

Refusal Conversion for Schools That Will Not Participate

If a school expresses strong concerns about confidentiality at any time during data collection, these concerns will be directed to the Census Project Director (and possibly to NCES) for formal assurance. All materials will include the project’s toll-free number. In addition, initial emails will include information about why the participation of each sampled school is important and how respondent information will be protected.

B3. Methods to Maximize Response Rates

NCES is committed to obtaining a high response rate in the SPP. In general, a key to achieving a high response rate is to track the response status of each sampled school, with telephone follow-up, as well as follow-up by e-mail, of those schools that do not respond promptly. To help track response status, survey responses will be monitored through an automated receipt control system. Telephone interviews will be conducted only by Census Bureau interviewers who have received training in general telephone interview techniques as well as specific training for SPP.

It is estimated that the survey will require a total of about 2-3 hours of school staff time each month. To encourage study participation, the sampled school will be offered up to a total reimbursement of $5,000 for their participation in the study over the course of 12 months (including September 2021 through August 2022) and will be sent a tote bag as a token of appreciation as part of the November mailout.

B4. Tests of Procedures

Cognitive Testing

As part of the development of the SPP, the monthly instrument starting in September underwent 20 cognitive testing with school and district administrators during the summer of 2021. The cognitive testing focused on items pertaining to the COVID-19 pandemic that caused widespread school closures, and significant changes to school policies and disruptions to their delivery of instruction to students in 2020, 2021, 2022. Iterative modifications were made throughout the testing to improve clarity and comprehension. Final recommendations to items are reflected in the final items in the instrument (Appendix B).

B5. Individuals Responsible for Study Design and Performance

Several key staff responsible for the study design and performance of the School Pulse Panel. They are:

Rachel Hansen, Project Director, National Center for Education Statistics
Michelle McNamara, National Center for Education Statistics
Rebecca Bielamowicz, National Center for Education Statistics
Cassandra Logan, U.S. Census Bureau
Elke McLaren, U.S. Census Bureau
Aaron Gilary, U.S. Census Bureau
Alfred Meier, U.S. Census Bureau
Kathleen Kephart, U.S. Census Bureau
Jessica Holzberg, U.S. Census Bureau

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Kathryn.Chandler
File Modified	0000-00-00
File Created	2021-09-10