Part B School Pulse Panel

The U.S. Census Bureau will collect the School Pulse Panel (SPP) data on behalf of NCES. Data collection will be a self-administered, online survey. It is estimated for the survey to require, on average, 30 minutes of school staff time each month. The sampled school will be offered a reimbursement of $400 for their participation in the study over the course of 12 months. The reimbursement will be paid out quarterly in the form of debit cards. Additionally, a non-monetary incentive (tote bag) may be included in one of the recruitment communications. If a school district does not permit its schools to receive any form of incentive, the reimbursement will be sent to a point of contact in the district, or the reimbursement and any non-monetary incentives will be withheld. Principals, other school staff most knowledgeable about COVID-19 impacts on the school environment and instructional offerings, and district staff can help complete the survey. No classroom time is involved in the completion of this survey.

The resulting data will provide aggregate estimates for public schools across the nation. A stratified sample design was used to select approximately 1,200 U.S. public schools. In addition, a reserve sample of replacement schools was selected to boost the number of responses if any schools from the initial sample do not respond. The sample is designed to provide national estimates of primary, middle, and high schools taking into account the type of locale (urbanicity) and racial/ethnic student enrollment.

B1. Respondent Universe and Sample Design and Estimation

The sampling frame for the School Pulse Panel is derived from the National Teacher and Principal Survey (NTPS) 2020-21 frame, which itself is largely derived from the 2018-19 Common Core of Data (CCD), the file of public schools supplied annually by State educational agencies to NCES. Only public schools in the 50 states and the District of Columbia will be included in the School Pulse Panel sampling frame, though the School Pulse Panel may sample Puerto Rico separately at a later date. Data may also be collected from the outlying areas (American Samoa, Guam, Commonwealth of Northern Mariana Islands, and U.S. Virgin Islands) at the district-level. Certain types of schools are excluded, including newly closed schools, home schools, virtual schools, ungraded schools, private schools, and schools with a high grade of kindergarten or lower. Regular public schools, charter schools, alternative schools, special education schools, vocational schools, correctional facilities/juvenile justice facilities, and schools that have partial or total magnet programs are included in the frame. For sample allocation purposes, strata are defined by instructional level, and the sample is sorted primarily on the type of locale (urbanicity), percent minority enrollment, and geographic region.

Tables 1 and 2 show the estimated expected distribution of the public-school sampling universe for the School Pulse Panel, by school level and urbanicity and by school level and percent minority enrollment, respectively.

Table 1. Expected respondent universe for the SPP sample, by school level and region, based on the 2020-21 NTPS Frame

Region	Elementary	Middle/Combined/Other	High/Grade 9-11	Total
Northeast	8,450	2,969	3,443	14,862
Midwest	12,468	4,461	6,051	22,980
South	18,075	7,480	8,157	33,712
West	13,306	4,805	5,813	23,924
Total	52,299	19,715	23,464	95,478

Table 2. Expected respondent universe for the SPP sample, by school level and urbanicity, based on the 2020-21 NTPS Frame

Urbanicity	Elementary	Middle/Combined/Other	High/Grade 9-11	Total
City	15,650	5,031	6,332	27,013
Suburb	18,007	6,471	6,355	30,833
Town	5,823	2,942	3,687	12,452
Rural	12,819	5,271	7,090	25,180
Total	52,299	19,715	23,464	95,478

Table 3. Expected respondent universe for the SPP school sample, by school level and percent minority enrollment, based on the 2020-21 NTPS Frame

Percent Minority	Elementary	Middle/Combined/Other	High/Grade 9-11	Total
0 to less than 25	15,487	5,975	7,423	28,885
25 to less than 50	11,220	4,508	4,435	20,163
50 to less than 75	8,884	3,529	3,576	15,989
75+	16,708	5,703	8,030	30,441
Total	52,299	19,715	23,464	94,578

Sample Selection and Response Rates

A stratified sample design was used to select approximately 1,200 U.S. public schools. The sample was designed to provide national estimates of primary, middle/combined, and high schools taking into account the type of locale (urbanicity), racial/ethnic student enrollment, and region. Note that combined schools will be grouped with middle schools for the purposes of measurement and estimation.

There are two stages of sample selection. A base sample of 1,200 schools was drawn as an initial stage as well as another reserve sample (second stage) of 1,200 schools with similar characteristics to our base sample. In case we do not get the necessary number of schools to participate out of the base sample, we plan to reach out to the reserve sample schools to complete the panel.

Recruitment for the panel study began in July and August, 2021. An initial goal was set to obtain 1,000 schools to commit to participate in the study throughout the duration of the year-long monthly collections (roughly a 70 to 80 percent response rates). However, current recruitment has not been as successful as anticipated. Therefore, it has been determined to begin recruiting the reserve sample schools in order to try to get enough responding schools to be able to report out accurate and reliable national estimates. This will likely result in an overall response rate of around 20-30 percent.

Note that there are no previous administrations of the School Pulse Panel, so we cannot derive estimates of response from those.

Sample Design for the School Pulse Panel

The main objective of the School Pulse Panel sampling design is to obtain overall subgroup estimates broken out by various school characteristics. For sample allocation and sample selection purposes, strata were defined by instructional level. In addition, region, locale, percent minority enrollment, enrollment size, and charter status were used as implicit stratification variables by sorting schools by these variables within each stratum before sample selection. The explicit stratification and the first three implicit stratification variables (region, locale, and percent minority enrollment) are priorities for evaluation for this panel. The method determined to allocating schools to the different sampling strata is allocate them proportionally to the U.S. public school population.

Note that there are no experiments planned within the sample, in which different schools in sample may be given different content or pathways.

Calculation of Weights

Weights will be attached to each surveyed school so that the weighted data will represent population levels. The final weight for completed cases will be composed of a sampling base weight and an adjustment for nonresponse. Nonresponse weighting adjustment cells for the SPP data will be determined using a categorical search algorithm called Chi-Square Automatic Interaction Detection (CHAID). CHAID begins by identifying the school-level characteristics of interest that are the best predictors of response. It divides the dataset into groups so that the unit response rate within cells is as constant as possible and the unit response rate between cells is as different as possible. The characteristics of interest as predictors of response must be available for both respondents and nonrespondents in order to conduct a CHAID analysis, and, in the case of SPP, will be available through the CCD sampling frame. The final, adjusted weights will be raked so that the sum of the weights matches the number of schools derived from the School Pulse frame.

Methods for Variance Estimation

Standard errors of the estimates will be estimated using jackknife replication. Replicate codes that indicate the computing strata and the half-sample to which each sample unit belongs will be provided, as will the weights for all replicates that were formed in order to calculate variances.

B2. Procedures for the Collection of Information

The School Pulse Panel data collection will begin in September of 2021. The U.S. Census Bureau, acting as a contractor for NCES, will handle the data collection. Starting in September, each sampled school will receive an email notifying them of the full survey, which will include information necessary to complete the online questionnaire. Respondents will have a two-week window to respond to the survey. Reminder emails will be sent during the data collection window. Data will be formally reviewed for disclosure prevention and released a several weeks after data collection ends for that month. This will be repeated monthly through August of 2022.

Due to a low response rate for the September collection, NCES plans to put the monthly collection on hold until January to focus on recruitment and building a more robust panel with committed schools.

Over the course of November and December, NCES restarted recruiting sampled schools by informing them of the reduced burden for the survey and requesting for them to complete a screener that captures pertinent information regarding contact materials and a mailing address for the debit cards. The response to this screener will be a way of capturing committed schools for the upcoming monthly collections that will resume in January, 2022. For January and subsequent months, respondents will have a two-week window to respond to the survey. An initial invitation email and reminder emails will be sent during this timeframe. After data collection has ended, the data will go through minimal data quality checks and will be released on a dashboard within 3-4 weeks of the end of collection.

Drawing the Sample

The samples of schools were drawn in the summer preceding data collection. Many districts (known as “special contact districts”) require research applications to be submitted and reviewed for approval before contact can be made with schools within their districts. Special contact district outreach was cleared in July (OMB# 1850-0963 v.3) and recruitment has continued through August, 2021.

School Communication

The School Pulse Panel will be conducted via a self-administered web-based survey instrument. A clerical operation prior to data collection was conducted to obtain e-mail addresses for all the sampled school principals, and these e-mails will be used to contact the school principals throughout the data collection. During the recruitment process (cleared in July 2021; last change request approved in August OMB# 1850-0963 v.3), an initial letter was sent via e-mail in July 2021 to notify sampled schools of their selection for the survey, to verify contact information, to inform schools about reimbursements of $400 a month (up to $5000 total) offered to schools over the course of 12 months for their continued participation in the study, and also included a link to a short preliminary summer questionnaire. Reminder e-mails have continued to be sent throughout August 2021. An initial invitation e-mail was sent in September to distribute instructions on how to complete the monthly web questionnaire, including the survey URL to access the survey online. Due to the pause in the monthly collection to focus on recruitment, new emails and letters have been developed and will be sent in November and December (Attachment A). As part of these emails and letters, sampled schools are being asked to complete a short screener capturing updated point of contact information that will be used for communications about upcoming collections, which we believe will also aid in successfully reaching the appropriate staff at schools (Attachment B). Physical letters will be sent in October and November in advance of the monthly collections to assist with legitimizing the study and to bring attention to the study in general.

The sampled school will be offered a reimbursement of $400 for their monthly participation in the study over the course of 12 months. The reimbursement will be paid out quarterly in the form of a debit card and will be prorated to include the dollar amount that reflects the number of months of participation for that quarter. Additionally, a non-monetary incentive (tote bag) will be included in the January or February communications. If a school district does not permit its schools to receive any form of incentive, the reimbursement will be sent to a point of contact in the district or the reimbursement and any non-monetary incentives will be withheld.

A copy of the letters and e-mails to be sent to school principals throughout the SPP data collection are included in appendices A.

Protocol for Follow-up Calls

During the monthly data collections, Census Bureau staff will initiate phone calls with nonrespondents, reminding them to complete their questionnaire.

Refusal Conversion for Schools That Will Not Participate

If a school expresses strong concerns about confidentiality at any time during data collection, these concerns will be directed to the Census Project Director (and possibly to NCES) for formal assurance. All materials will include the project’s toll-free number. In addition, initial emails will include information about why the participation of each sampled school is important and how respondent information will be protected.

B3. Methods to Maximize Response Rates

NCES is committed to obtaining a high response rate in the SPP. In general, a key to achieving a high response rate is to track the response status of each sampled school, with telephone follow-up, as well as follow-up by e-mail, of those schools that do not respond promptly. To help track response status, survey responses will be monitored through an automated receipt control system. Telephone interviews will be conducted only by Census Bureau interviewers who have received training in general telephone interview techniques as well as specific training for SPP.

It is estimated that the survey will require, on average, about 30 minutes of school staff time each month. To encourage study participation, the sampled school will be offered a reimbursement of $400 a month for their participation in the study over the course of 12 months and may be sent a tote bag as a token of appreciation as part of a monthly mailout.

B4. Tests of Procedures

Cognitive Testing

As part of the development of the SPP, the monthly instrument starting in September underwent 20 cognitive testing with school and district administrators during the summer of 2021. The cognitive testing focused on items pertaining to the COVID-19 pandemic that caused widespread school closures, and significant changes to school policies and disruptions to their delivery of instruction to students in 2020, 2021, 2022. Iterative modifications were made throughout the testing to improve clarity and comprehension. Final recommendations to items are reflected in the final items in the instrument (Appendix B).

B5. Individuals Responsible for Study Design and Performance

Several key staff responsible for the study design and performance of the School Pulse Panel. They are:

Rachel Hansen, Project Director, National Center for Education Statistics
Rebecca Bielamowicz, National Center for Education Statistics
Ryan Iaconelli, National Center for Education Statistics
Cassandra Logan, U.S. Census Bureau
Elke McLaren, U.S. Census Bureau
Aaron Gilary, U.S. Census Bureau
Alfred Meier, U.S. Census Bureau
Kathleen Kephart, U.S. Census Bureau
Jessica Holzberg, U.S. Census Bureau

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Kathryn.Chandler
File Modified	0000-00-00
File Created	2022-01-21