Change Memo

Memorandum United States Department of Education

Institute of Education Sciences

National Center for Education Statistics

DATE: December 18, 2023

TO: Beverly Pratt, OMB

THROUGH: Carrie Clarady, NCES

FROM: Maura Spiegelman, NCES

SUBJECT: 2023-24 National Teacher and Principal Survey (NTPS 2023-24) School Conditional Incentive Threshold Change Request (OMB# 1850-0598 v.44)

The National Teacher and Principal Survey (NTPS), conducted every two to three years by the National Center for Education Statistics (NCES), is a system of related questionnaires that provides descriptive data on the context of elementary and secondary education. Redesigned from the Schools and Staffing Survey (SASS) with a focus on flexibility, timeliness, and integration with other ED data, the NTPS system allows for school, principal, and teacher characteristics to be analyzed in relation to one another. NTPS is an in-depth, nationally representative survey of first through twelfth grade public and private school teachers, principals, and schools. Kindergarten teachers in schools with at least a first grade are also surveyed. NTPS utilizes core content and a series of rotating modules to allow timely collection of important education trends as well as trend analysis. Topics covered include characteristics of teachers, principals, schools, teacher training opportunities, retention, retirement, hiring, and shortages.

The NTPS 2023-24 preliminary activities were approved in December 2022 (OMB# 1850-0598 v.41-42). OMB approved the main study for 2023-24 in June 2023 (OMB# 1850-0598 v.43). This change request modifies an experiment that was specified for the full collection, simply by lowering the threshold eligible for incentivizing schools from 90% to 75%.

During the 2017-18 NTPS, we found that small, pre-paid cash incentives increased teacher response rates and the representativeness of responding teachers. We planned additional experiments during the 2020-21 NTPS, which were cancelled in the fall of 2020 shortly after the start of data collection due to operational challenges during the COVID-19 pandemic. For this collection, our data collection plan builds upon our earlier experimental findings and will inform future collections of the NTPS and longitudinal follow-up surveys. In particular, we will test promised incentives with sampled teachers in two different ways.

First, we will compare a $10 prepaid incentive (which we know to be effective from past testing) to a $5 prepaid incentive and promised $20 incentive to see whether it can increase response rates and representativeness, reduce operational complexity, and whether other types of promised incentives (e.g., digital rather than cash) can be used in future collections. We also will compare the use of a group incentive to see whether promising an incentive to their school, rather than an individual teacher, will have the same effect. In our approved package, we promised a threshold of 90%, that is, a school would receive a $200 incentive if 90% of sampled teachers returned a questionnaire.

This threshold was established by looking at the distribution of teachers’ response rates within schools during the 2017-18 and 2020-21 NTPS collections. NTPS teachers are sampled on a rolling basis as we obtain teacher roster from sampled schools, and response rates among late-sampled teachers are notably lower than among early-sampled teachers, so these experiments are limited to late-sampled teachers. However, the 90% threshold was based on response rates for all teachers, rather than limited to late-sampled teachers. In the 2020-21 NTPS, approximately 15 percent of schools with late-sampled teachers obtained a within-school teacher response rate of 90% or more, compared to about 45 percent of schools with early-sampled teachers. While a 90% threshold may be appropriate for teachers in early-sampled schools, while a 75% threshold sets an ambitious, yet attainable, goal for teachers in late-sampled schools. Therefore, we propose that a school with late-sampled teachers can receive a $200 incentive with 75% of sampled teachers, rather than 90%, submit an NTPS teacher questionnaire.

All changes in the attached materials are shown in full in the pages to follow. Black font demarks the text that remains unchanged from the last approved package, while text in red font shows the revisions that were made. This requested change does not affect the approved total cost to the federal government for conducting this study nor the estimated respondent burden.

Changes to Part A

A.9 Provision of Payments or Gifts to Respondents

Beginning with the wave where teachers are sampled from the publicly available teacher data, and all teacher waves thereafter (referred to as “late sampled teacher waves” from this point forward), all teachers will receive a prepaid, cash monetary incentive at the first contact by mail. Half will receive $10 at the first contact by mail (treatments A and B) while the other half (treatments C and D) will receive $5 at the first contact by mail. A portion of the of the public-school teachers and all of the private school teachers in the $5 group will be offered an additional promised monetary incentive ($20) upon questionnaire completion (treatment C). The remaining public-school teachers in the $5 group (treatment D) and a portion of the public-school teachers in the $10 group (treatment B) will be offered a promised school-level incentive if they complete their questionnaire and help their school achieve an overall response rate of at least 9075% in their school. Treatments will be assigned randomly at the time of school sampling at the school-level – that is, all teachers within a sampled school will receive the same treatment.

In summary, teachers sampled from publicly available teacher data TLFs and all teachers sampled thereafter will be eligible for inclusion in a teacher incentive experiment with the following four treatments:

• Treatment A: Teachers receive a prepaid cash ($10) incentive at the first contact by mail.

• Treatment B: Teachers receive a prepaid cash ($10) incentive, and their school will receive a school-level promised incentive if their school reaches 9075% in overall teacher response. This treatment is restricted to public school teachers only.

• Treatment C: Teachers receive a prepaid cash ($5) incentive and will receive an additional $20 promised incentive upon completion of their Teacher Questionnaire.

• Treatment D: Teachers receive a prepaid cash ($5) incentive, and their school will receive a school-level promised incentive if their school reaches 9075% in overall teacher response. This treatment is restricted to public school teachers only.

In addition, for the schools whose teachers are sampled in late waves, survey coordinators identified in the screener interview from vendor-sampled schools and a subsample of principals (as proxy coordinators) from non-screened schools will receive a cash incentive in the first mailed package of teacher materials for distribution. The teachers in the remaining non-screened schools will be mailed their materials directly throughout data collection rather than through a proxy coordinator.

Further details about the use of incentives are provided in sections B.2.3 and B.3.2 of Supporting Statement Part B.

Changes to Part B

B.2.3 Teacher Data Collection

Teachers sampled from publicly available teacher data and all teachers sampled thereafter will be included in an experiment aimed at boosting teacher response by offering different levels and types of incentivizes. Teachers will be randomly assigned to a treatment group in which they receive either a $5 or $10 prepaid cash incentive. Teachers in both the $5 and $10 individual incentive groups may be randomly assigned to an experimental school-level incentive treatment in which their school will be offered a $200 promised school-level incentive if they achieve an overall teacher questionnaire response rate of at least 9075 percent in their school. The remaining teachers in the $5 individual incentive treatment who are not assigned to the school-level incentive treatment will receive an additional promised $20 upon questionnaire completion. Note that only public schools are eligible for inclusion in the school-level incentive experiment.

B.3.2.1 Data Collection Strategies to Minimize Non-Response

8. Offer additional incentives for late-sampled teachers. Results of previous administrations of the NTPS indicated that offering a prepaid teacher incentive significantly increased the final response rates for both public and private school teacher questionnaires. Due to these favorable results, NTPS 2023-24 will offer all teachers prepaid incentives at the first contact by mail.

Previous NTPS cycles showed that response rates for late-sample wave teachers in NTPS appeared to be lower than for earlier waves of teachers. This may have been a product of the timing of school testing and late-school year activities because late-sample wave teachers received an invitation to complete the survey during a period with a heavy school workload, while teachers sampled earlier may have completed their questionnaire in the fall. It may also have been because the late sampled teachers were in schools that were either late responders to the TLF or TLF non-respondents (in instances where teachers were sampled from a teacher roster obtained from clerical research or vendor data) and, therefore, may have had less support and encouragement from their principal and/or survey coordinator to complete their questionnaire.

Starting with the teacher sampling wave during which teachers are selected from publicly available teacher lists and all teacher sampling waves thereafter, teachers will be included in an experiment aimed at boosting teacher response for these historically low-responding teachers by offering different levels and types of incentivization.

All teachers will receive a prepaid cash incentive at the first contact by mail. The cash incentive amount will vary between $5 or $10 depending on the sampled teacher’s assigned treatment group. A portion of the teachers in the $5 group will be offered an additional promised monetary incentive ($20) upon completing their questionnaire. The remaining teachers in the $5 group and a portion of teachers in the $10 group will be offered a promised $200 school-level incentive if they complete their questionnaire and help their school achieve an overall teacher response rate of ninety 75 percent or higher within their school.

B.4.1.2 Testing at the Teacher-level

2. Testing Various Incentive Strategies for Teachers sampled from publicly available teacher data (“Coordinator and Promised Incentive Experiment”). NTPS 2023-24 will once again include the use of monetary incentives for teachers. Early and mid-sampled teachers will all receive a prepaid, $5 cash incentive at the first contact by mail. Beginning with the teacher wave selected from publicly available teacher data, and all teacher waves thereafter, all teachers will receive a prepaid cash incentive at the first contact by mail.

Half will receive $5 cash at the first contact by mail while the other half will receive $10 cash at the first contact by mail. A portion of the of the public-school teachers and all of the private school teachers in the $5 group will be offered an additional promised cash monetary incentive ($20) upon questionnaire completion. The remaining public-school teachers in the $5 group and a portion of the public-school teachers in the $10 group will be offered a promised school-level incentive if they complete their questionnaire and help their school achieve an overall response rate of at least 9075 percent in their school. In summary, teachers sampled from vendor TLFs and all teachers sampled thereafter will be eligible for inclusion in a teacher incentive experiment with the following four treatments:

• Treatment A: Teachers receive a prepaid cash ($10) incentive at the first contact by mail.

• Treatment B: Teachers receive a prepaid cash ($10) incentive, and their school will receive a school-level promised incentive if their school reaches at least 9075 percent in overall teacher response. This treatment is restricted to public school teachers only.

• Treatment C: Teachers receive a prepaid cash ($5) incentive and will receive an additional $20 promised incentive upon completion of their teacher listing form.

• Treatment D: Teachers receive a prepaid cash ($5) incentive, and their school will receive a school-level promised incentive if their school reaches at least 9075 percent in overall teacher response. This treatment is restricted to public school teachers only.

In addition, for the teacher wave selected from publicly available teacher data and beyond, survey coordinators identified in the screener interview (treatment 1) and a subsample of principals (as proxy coordinators) from non-screened schools (treatment 2) will receive a prepaid cash incentive of the same amount as the teachers’ prepaid cash incentive ($10 for the A and B treatments and $5 for the C and D treatments) in the first mailed package of teacher materials for distribution. This nonexperimental prepaid incentive for school coordinators is based on research from the 2017-18 NTPS. The teachers in the remaining non-screened schools will be mailed their materials directly throughout data collection rather than through a proxy coordinator (Treatment 3). As a result, there are ten treatment groups for the Coordinator and Promised Incentive Experiment:

• Treatment 1: Schools with a survey coordinator – the coordinator will be sent a package of individually-sealed teacher survey invitations and asked to distribute them to the teachers.

  • Treatment 1a: Each teacher will receive a half-sheet-sized envelope containing a letter and a prepaid cash ($10) incentive. Survey coordinators will receive a prepaid cash ($10) incentive.

  • Treatment 1b: Each teacher will receive a half-sheet-sized envelope containing a letter and a prepaid cash ($10) incentive. Survey coordinators will receive a prepaid cash ($10) incentive. The school will receive a school-level promised incentive if the school reaches at least 9075% in overall teacher response, and this will be communicated to sampled teachers and the survey coordinator. This treatment is restricted to public school teachers only.

  • Treatment 1c: Each teacher will receive a half-sheet-sized envelope containing a letter and a prepaid cash ($5) incentive. The letter will offer a promised cash incentive ($20) upon the teacher’s survey completion. Survey coordinators will receive a prepaid cash ($5) incentive.

  • Treatment 1d: Each teacher will receive a half-sheet-sized envelope containing a letter and a prepaid cash ($5) incentive. Survey coordinators will receive a prepaid cash ($5) incentive. The school will receive a school-level promised incentive if the school reaches at least 9075% in overall teacher response, and this will be communicated to sampled teachers and the survey coordinator. This treatment is restricted to public school teachers only.

• Treatment 2: Schools without a survey coordinator, principal coordinator treatment – the principal will be treated as a proxy coordinator and will be sent a package of individually-sealed teacher survey invitations and asked to distribute them to the teachers.

  • Treatment 2a: Each teacher will receive a half-sheet-sized envelope containing a letter and a prepaid cash ($10) incentive. Principals will receive a prepaid cash ($10) incentive.

  • Treatment 2b: Each teacher will receive a half-sheet-sized envelope containing a letter and a prepaid cash ($10) incentive. Principals will receive a prepaid cash ($10) incentive. The school will receive a school-level promised incentive if the school reaches at least 9075% in overall teacher response, and this will be communicated to sampled teachers and the principal. This treatment is restricted to public school teachers only.

  • Treatment 2c: Each teacher will receive a half-sheet-sized envelope containing a letter and a prepaid cash ($5) incentive. The letter will offer a promised cash incentive ($20) upon the teacher’s survey completion. Principals will receive a prepaid cash ($5) incentive.

  • Treatment 2d: Each teacher will receive a half-sheet-sized envelope containing a letter and a prepaid cash ($5) incentive. Principals will receive a prepaid cash ($5) incentive. The school will receive a school-level promised incentive if the school reaches at least 9075% in overall teacher response, and this will be communicated to sampled teachers and the principal. This treatment is restricted to public school teachers only.

• Treatment 3: Schools without a survey coordinator, no coordinator treatment - The teacher packages will be mailed directly to the teachers.

  • Treatment 3a: Each teacher will receive a half-sheet-sized envelope containing a letter and a prepaid cash ($10) incentive.

  • Treatment 3c: Each teacher will receive a half-sheet-sized envelope containing a letter and a prepaid cash ($5) incentive. The letter will offer a promised cash incentive ($20) upon the teacher’s survey completion.

3