OMB83C Change Memo v22

U NITED STATES DEPARTMENT OF EDUCATION

National Center for Education Statistics

MEMORANDUM

Date: February 12, 2018

To: Robert Sivinski, OMB

From: Andrew Zukerberg, NCES

Through: Kashka Kubzdela, NCES

Re: 2017–18 National Teacher and Principal Survey (NTPS 2017-18) Contingency Plan and Email Change Request (OMB# 1850-0598 v.22)

The National Teacher and Principal Survey (NTPS), conducted biennially by the National Center for Education Statistics (NCES), is a system of related questionnaires that provides descriptive data on the context of elementary and secondary education. Redesigned from the Schools and Staffing Survey (SASS) with a focus on flexibility, timeliness, and integration with other ED data, the NTPS system allows for school, principal, and teacher characteristics to be analyzed in relation to one another. NTPS is an in-depth, nationally representative survey of first through twelfth grade public and private school teachers, principals, and schools. Kindergarten teachers in schools with at least a first grade are also surveyed. NTPS utilizes core content and a series of rotating modules to allow timely collection of important education trends as well as trend analysis. Topics covered include characteristics of teachers, principals, schools, teacher training opportunities, retention, retirement, hiring, and shortages. NTPS 2017-18 was approved in June 2017 with the latest change request approved in September 2017 (OMB# 1850-0598 v.18-21). This request updates the approved for NTPS 2017-18: (a) contingency plan for an incentive experiment and (b) procedures and materials with an addition of a reminder email to schools, principals, and teachers urging completion of the questionnaire(s).

NTPS is currently in the field and, per the approved plan, we have mailed three packages to schools asking them to complete the questionnaires. A fourth package is scheduled for February 12, 2018. Our current response rates for the school and principal questionnaires are approximately 53 percent and 49 percent respectively. We propose an additional email contact in late February or early March to boost response rates for the school and principal questionnaires in an effort to reduce the number of cases included in the in-person follow-up operation scheduled to begin in mid-March. We also propose sending an additional email to teachers that are not eligible for the contingency plan incentive experiment. Lastly, we have included updated information on the domains that we intend to target for the teacher contingency plan incentive.

This request does not introduce changes to respondent burden or the cost to the federal government. The following changes were made to the approved package materials as part of this change request.

Part A

In addition, NCES seeks approval to will provide monetary boosts as a contingency plan to combat potential low response rates from teachers in the later teacher mailing waves. If activated, t The contingency plan would will be executed as needed based on monitoring data collection status.

Part B

In section B.2.2 (School-level Data Collection Procedures), the following paragraph was added to the text (p.8) and the following text box to Exhibit 1 (p.9):

Reminder email to Principals/Schools

Late February-Early March

Reminder email to Teachers

Late February- Early June

Between late February and early March 2018, an additional reminder email will be sent to Principals and/or Survey Coordinators of nonresponding schools, and between late February and early June 2018, an email reminder will be sent to teachers who are not eligible for the contingency plan incentive experiment. Each of these emails will include a link to an informational NTPS video and to the relevant survey, the respondent’s User ID, and selected findings from the 2015-16 NTPS.

In Section B.4.2, table 9 and its accompanying text (p.19-20) were revised to show the correct allocation to treatment groups (in the “Initial School Sample Sizes” column of Table 9, “1,298 Public Schools” were replaced with “1,325 Public Schools” in each row – the revised portions are reflect in red font below):

Table 9. Experimental Groups and initial sample sizes.

Experimental Group	Phase One (Waves 1-12)	Phase Two (Waves 13-20)	Initial School Sample Sizes	Initial Teacher Sample Sizes1
1	Teacher Incentive	Teacher Incentive School Coordinator (SC) or Principal Incentive	1,325 Public Schools 500 Private Schools	5,784 Public School Teachers 875 Private School Teachers
2	Teacher Incentive	Teacher Incentive No SC or Principal Incentive	1,325 Public Schools 500 Private Schools	5,784 Public School Teachers 875 Private School Teachers
3	Teacher Incentive	No Teacher Incentive SC or Principal Incentive	1,325 Public Schools 500 Private Schools	5,784 Public School Teachers 875 Private School Teachers
4	Teacher Incentive	No Teacher Incentive No SC or Principal Incentive	1,325 Public Schools 500 Private Schools	5,784 Public School Teachers 875 Private School Teachers
5	No Incentive	Teacher Incentive SC or Principal Incentive	1,325 Public Schools 500 Private Schools	5,784 Public School Teachers 875 Private School Teachers
6	No Incentive	Teacher Incentive No SC or Principal Incentive	1,325 Public Schools 500 Private Schools	5,784 Public School Teachers 875 Private School Teachers
7	No Incentive	No Teacher Incentive SC or Principal Incentive	1,325 Public Schools 500 Private Schools	5,784 Public School Teachers 875 Private School Teachers
8	No Incentive	No Teacher Incentive No SC or Principal Incentive	1,325 Public Schools 500 Private Schools	5,784 Public School Teachers 875 Private School Teachers

The schools will be assigned into one of eight experimental groups prior to the beginning of data collection. As such, the random assignment should result in similar TLF response rates across all groups by the beginning of Phase Two of the experiment. Approximately 10,385 10,600 public schools and 4,000 private schools will be sampled for NTPS 2017-18. Because Phase One of the incentive experiment is independent of Phase Two of the data collection period, the random assignment of schools into the eight groups should result in similar TLF response rates across all groups at the start of Phase Two. Before the start of data collection, each experimental group will be assigned 1,298 1,325 public schools and 500 private schools. Therefore, approximately 5,192 5,300 public schools and 2,000 private schools will be assigned to the treatment groups that send teacher incentives during Phase One of the experiment.

In Section B.4.2, in subsection “Contingency Plan” (p.22-23) text was revised as follows:

NCES will monitored actual and expected response in each of the key domains on a weekly basis and identified domains that are If a domain is determined to be ‘at-risk’ (at risk for meeting NCES publishability standards) by as of February 12, 2018, NCES will activate the contingency plan. Since this is the first year utilizing a contingency incentive, it is being implemented will be done as an experiment with a control group that does not receive the incentive. (…)

The earliest the contingency plan may go into effect will be at during wave 13 (2/26/2018). We would like to meet with OMB during early in the week of 2/12/2018 to brief OMB on the status of data collection and any domains that have been identified for the contingency plans, and to agree on the next steps. Subsequent to this meeting, we would submit a change request to update the NTPS 2017-18 active record with further details of how the contingency plan will be implemented. The NTPS 2017-18 Contingency Plan and Email Change Request Memo (OMB# 1850-0598 v.22) provides the final details about the contingency plan.

Appendix A – Communication Materials

We added the “Targeted Reminder E-mail with NTPS Video” on pages 81-82 and information about this email and video to the summary table on page 29, as follows:

	Data Collection Activity/Operation	Correspondence Identifier(s)	Description
Targeted Reminder E-mail with NTPS Video		NTPS-23E(P1//P2// S1//S2//T1//T2)	Email is targeted by school level, state, and urbanicity and reminds respondent to complete their questionnaire. It includes a link to a new NTPS informational video. Six versions: NTPS-23E(P1): Sent to public school principals, includes data from the 2015-16 NTPS principal survey. NTPS-23E(P2): Sent to private school principals. NTPS-23E(S1): Sent to public school survey coordinators or school principals. Includes data from the 2015-16 NTPS school survey. NTPS-23E(S2): Sent to private school survey coordinators or school principals. NTPS-23E(T1): Sent to public school principals, includes data from the 2015-16 NTPS teacher survey. NTPS-23E(T2): Sent to private school teachers.

Targeted Reminder E-mail with NTPS Video

NTPS-23E(P1//P2//S1//S2//T1//T2) (P=Principal Questionnaire; S=School Questionnaire; T=Teacher Questionnaire; 1=Public School Version; 2=Private School Version)

Subject: Please help us produce representative data for [P1, P2: principals in//T1, T2: teachers in] schools like yours.

Dear <P1, P2: PRIN_NAME//S1, S2: COOR_NAME if available, else PRIN_NAME//T1,T2: TCH_NAME>:

We’ve contacted [S1, S2: your school//P1, P2, T1, T2: you] throughout the school year regarding the 2017–18 National Teacher and Principal Survey (NTPS). We wanted to take an opportunity to share a brief video with you that highlights the importance of this survey, and provides selected findings from previous surveys. You can view the video at: https: _{//www.youtube.com/watch?v=YaO87bhBOXc}

[P1, S1, T1: The 2015-16 NTPS included findings such as:]

[P1:

• Public school principals spent an average of 58.6 hours per week on all school-related activities.

• Public school principals had an average of 6.6 years of experience as a principal and an average of 4 years as a principal of their current school.//

S:

• Nationwide, about 21 percent of public schools offered at least one course entirely online.

• Overall, 59 percent of public schools had instruction beyond the normal school day for students who need academic assistance.//

T:

• On average, regular full-time teachers in public schools spent 53 hours per week on all school-related activities, including 27 hours that they were paid to deliver instruction to students during a typical full week.

• Among teachers in self-contained classrooms, the average class size was 21 students in primary schools, 14 students in middle schools, 15 students for high schools, and 16 students for combined-grade schools. Among departmentalized teachers, the average class size was 27 for primary and middle schools, 26 for high schools, and 22 for combined-grade schools. ]

We need your help to produce representative data for <primary/middle/high/combined grade level> [P: school principals//S: schools//T: school teachers] in <state’s> <cities/suburbs/towns/rural areas> this year. Policy makers at the state, federal, and local levels use this data to set education policy and improve teacher and principal working conditions. This is your chance to let your voice be heard.

Please complete the <P: Principal//S: School//T: Teacher> Survey using the login information provided below. Alternatively, you may complete and return the paper questionnaire we sent you [P, S: in mid-February//T: a few weeks ago].

To complete the survey, please go to: https://respond.census.gov/ntpsxx

Log in using this User ID: <USERID >

The National Center for Education Statistics (NCES), within the U.S. Department of Education, is authorized to conduct this survey by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). If you have any questions about the survey or need assistance, please contact the U.S. Census Bureau at 1-888-595-1338 between 8:00 a.m. and 8:00 p.m. (Eastern Time) Monday through Friday. The U.S. Census Bureau is also available to answer your questions via e-mail at [email protected].

Thank you in advance for your help to ensure your school is represented in this survey.

Sincerely,

National Teacher and Principal Survey Team

U.S. Census Bureau, on behalf of the

National Center for Education Statistics (NCES)

-------

Voiceover Script for the NTPS Information Video at the url Included in the Email

Audio Text for the National Teacher and Principal Survey (NTPS) Video

NOTE FOR NARRATOR - Pronunciation instructions:

• Please pronounce “2015-16”: two-thousand fifteen sixteen

• Please pronounce “SASS” as “sass,” like the word.

• In the web address, pronounce the letters “ED”. Do not say “Ed” like the name.

• In the web address, pronounce “guv”. Do not spell out “GOV”.

Teachers and principals form the foundation of the educational process, and it is important to have nationally representative data on the characteristics and experiences of these key staff. The National Teacher and Principal Survey, or NTPS, provides descriptive data on the context of elementary and secondary education in America. As the country’s primary source of information on teachers and administrators, the survey gives policymakers vital statistics on the condition of education in the United States.

Historically, NCES collected data from school staff using the Schools and Staffing Survey, or SASS. Since 1987, SASS has provided data that made it possible to answer critical questions about schools, teachers, principals, and students. This includes questions about teacher preparation, principals’ goals, and how the workforce has changed over time.

NCES updated the SASS, to reflect changes to the structure of teaching and new opportunities to integrate data from a variety of different sources.

NCES launched NTPS, the new survey, in the 2015-16 school year. NTPS includes updated content that addresses pressing topics like online courses and teacher and principal evaluations.

NTPS includes several different but related questionnaires that collect data from multiple points within a school. The Teacher Questionnaire includes questions about teacher satisfaction, use of instructional software in the classroom, and first year teaching experiences, along with questions on the characteristics of teachers.

Here’s a first look at some of the data from the survey.

In 2015-16, over 40 percent of public school teachers had 15 or more years of teaching experience.

By contrast, 15 percent of teachers had less than 4 years of experience.

The average base salary for regular full-time teachers in 2015-16 was 55,100 dollars.

Salaries differed by school level and ranged from 54,300 dollars for primary school teachers to 57,400 dollars for high school teachers.

The Principal Questionnaire asks questions on parent or guardian involvement, frequency of problems such as bullying, how teachers and principals are evaluated, and principals’ top goals, along with questions on the characteristics of principals.

In 2015-16, a majority of public school principals were White.

About 11 percent of principals were Black, 8 percent were Hispanic, and 3 percent were of other races and ethnicities.

The School Questionnaire asks about topics like the length of the school day, how difficult it is to fill vacancies at the school, online course offerings, and school start times.

In 2015-16, high schools had the earliest start times, with an average start time of 7:59am. Primary schools had the latest start times, with an average start time of 8:17am.

Visit nces.ed.gov to learn more about the NTPS, and to find out more about teachers, principals, and schools in the United States.

NTPS 2017-18 – Updates² for Implementing the Contingency Plan Teacher Incentives

1. Purpose

The purpose of this document is to expand on the information contained in the last approved 2017-18 National Teacher and Principal Survey (NTPS) clearance request (OMB# 1850-0598 v.21) regarding a contingency incentive plan and confirm that the contingency plan teacher incentives will be implemented during the 2017-18 NTPS. The NTPS is a system of surveys, and in its current design, has only been fielded once before, in 2015-16. During that fielding, response rates for some key sub groups (e.g., city teachers) did not meet NCES reporting standards (e.g., a response rate of at least 50% within the subgroup). To prepare for this possibility occurring during the 2017-18 collection, we included in the approved clearance documents (OMB# 1850-0598 v.18-21) a contingency incentive experiment that could be implemented if needed during data collection. A description of this plan was included in section B.4.2 of the Supporting Statement Part B (pp.21-22). Below, we outline the current 2017-18 NTPS response rates, impact of incentives to date, and groups where we believe additional (contingency) incentives are needed.

2. Overview of the Contingency Incentive Plan

The contingency plan teacher incentive is a secondary incentive (boost incentive), and will be implemented on an experimental basis at the school level (all NTPS sampled teachers in the school are in the same condition group (treatment or control)). The incentive will be sent with the third survey mailout (this will be the first time many teachers receive a paper questionnaire rather than an invitation to take the survey by internet). Eligible non-priority school teachers will receive a pre-paid $10 cash incentive and eligible priority school teachers will receive a pre-paid cash $20 incentive. If the teacher has already completed their interview by the time of the third mail-out, they will receive the incentive with a ‘thank-you’ letter as opposed to the paper questionnaire. This will prevent incentive conditioning (e.g., if teachers observe other teachers receiving larger incentives by delaying response during this round, they may choose to delay response in subsequent NTPS data collection cycles, should they be sampled again, in hopes of receiving larger incentives).

Prior to data collection, schools were selected to be eligible to receive the contingency plan teacher incentives at random across eight experimental groups that were created for the main incentive experiment (described below). Half of the schools in sample are eligible to receive the contingency plan teacher incentives and the other half are in the control group. This will allow comparisons between the teachers that receive the contingency plan incentives and teachers that do not receive the contingency plan incentives, within the selected school domains.

3. Background on the NTPS 2017-18 Teacher Incentive Experiment

For the 2017-18 data collection period, NTPS is conducting a teacher incentive experiment. The goal of the experiment is to determine whether or not providing teachers with incentives increases the overall teacher response rates. Because teachers are sampled on a flow basis, the teacher incentive experiment is comprised of two phases. Phase one of the teacher incentive experiment is currently active. This phase runs during waves 1 through 12 of the teacher sampling period. Phase two of the teacher incentive experiment began with wave 13 on February 6^th, 2018, and runs through wave 20. During phase one of the experiment, teachers in the experimental group are receiving a $5 incentive with their first mail-out invitation to complete the survey. During phase two of the experiment, teachers in experimental group are receiving $5 or $10, depending on whether their school is considered a priority school. Schools in the phase two of the teacher incentive experiment tend have lower response rates, which impacts results publishability. As such, an incentive is being provided to the school coordinator or principal in the first teacher mailout package in waves 13-20. The school coordinator or principal is responsible for distributing the teacher questionnaires and acts as a gatekeeper to the teachers. Therefore, an incentive is provided to the school’s coordinator or principal as a token of appreciation for their assistance with distributing the teacher questionnaires.

Table 1 shows the approved incentive types and amounts for each phase. Priority schools receive a higher incentive during phase two of the incentives experiment because these school were identified prior to data collection as having low propensity to respond and larger impacts on weighting, and thus require extra intervention, based on previous cycles of NTPS or SASS data collection.

Table 1. Incentive Types and Amounts

Waves	Incentive Type	Incentive Amount
		Non-Priority School	Priority Schools
1-12	Teacher Incentive	$5	$5
13-20	School Coordinator/Principal Incentive Teacher Incentive	$5 $5	$10 $10

The sampling design for the teacher incentive experiment is comprised of eight experimental groups, with schools sampled into the groups at random. All schools were assigned to the experimental groups prior to the beginning of data collection. To ensure a similar distribution of schools for each of the eight experimental groups, the sample was sorted by an indicator for whether or not the school is covered on a vendor list, an indicator for priority schools, the Census region, the urban/rural locale code, the response propensity for each school, and a random number (for the purpose of breaking ties).

There are 10,600 public schools and 4,000 private schools in sample for the 2017-18 NTPS. Before the start of data collection, 1,325 public schools and 500 private schools were assigned to each of the eight experimental groups. Therefore, 5,300 public schools and 2,000 private schools are included treatment groups that receive teacher incentives in the experiment. Table 2 shows which experimental groups will receive incentive and the initial school and teacher sample sizes within each group.

Table 2. Experimental Group Descriptions and Initial Sample Sizes

Experimental Group	Phase One (Waves 1-12)	Phase Two (Waves 13-20)		Initial School Sample Sizes	Initial Teacher Sample Sizes3
	Teacher Incentive?	Teacher Incentive?	School Coordinator or Principal Incentive?
1	Yes	Yes	Yes	1,325 Public Schools 500 Private Schools	5,784 Public Teachers 875 Private Teachers
2	Yes	Yes	No	1,325 Public Schools 500 Private Schools	5,784 Public Teachers 875 Private Teachers
3	Yes	No	Yes	1,325 Public Schools 500 Private Schools	5,784 Public Teachers 875 Private Teachers
4	Yes	No	No	1,325 Public Schools 500 Private Schools	5,784 Public Teachers 875 Private Teachers
5	No	Yes	Yes	1,325 Public Schools 500 Private Schools	5,784 Public Teachers 875 Private Teachers
6	No	Yes	No	1,325 Public Schools 500 Private Schools	5,784 Public Teachers 875 Private Teachers
7	No	No	Yes	1,325 Public Schools 500 Private Schools	5,784 Public Teachers 875 Private Teachers
8	No	No	No	1,325 Public Schools 500 Private Schools	5,784 Public Teachers 875 Private Teachers

For analysis purposes, the experimental groups are collapsed based on the phase of the experiment (breakdown shown in Table 3 below). For teachers that are sampled during phase one, the analysis groups collapse into treatment (1, 2, 3, and 4) and control (5, 6, 7, and 8). For phase two, the analysis groups collapse into treatment one (1,5), treatment two (2,6), treatment three (3,7), and control (4,8).

Table 3. Breakdown of Experimental Groups into Analysis Groups

Description of Two Analysis Groups for Phase One	Description of Four Analysis Groups for Phase Two
Experimental Groups 1, 2, 3, and 4 – Teacher Incentive	Experimental Groups 1 and 5 – Teacher Incentive and SC/Principal Incentive
	Experimental Groups 2 and 6 – Teacher Incentive and No SC/Principal Incentive
Experimental Groups 5, 6, 7, and 8 – No Teacher Incentive	Experimental Groups 3 and 7 – No Teacher Incentive and SC/Principal Incentive
	Experimental Groups 4 and 8 – No Teacher Incentive and No SC/Principal Incentive

4. Current Status of the Teacher Questionnaire Response Rates by Domain

At the time of this memo, data for waves 1-8 from NTPS 2017-18 was available for preliminary analysis to determine the effectiveness of the incentive experiment on response rates. Although other waves have received initial mailings, waves 1-8 were the only groups for which a data file was available and enough time had passed to see the effectiveness of the incentives. Data was analyzed separately for the public school domains and the private school domains

5. Current Results for the NTPS 2017-18 Teacher Incentive Experiment

The current teacher response rates for the first eight waves of the 2017-18 NTPS teacher sample data collection are shown for the private school domains in Table 4 and for the public school domains in Table 5. The response rates were calculated overall and for the incentive and non-incentive groups. The table shows that for “all schools” and within a majority of the domains, the teachers receiving an incentive are responding at a significantly higher rate than the teachers that are not receiving an incentive. Cells in blue indicate the domains with response rates below 50%, which may make good candidates for receiving contingency incentives, while cells highlighted in red indicate that the response rate for the teacher incentive group is significantly higher than the response rate for the non-incentive group with 95% confidence.

Table 4. Current TQ Response Rates across and within the Private School Domains

Private School Domain	TQ Response Rates
	Overall	Incentive	Non-Incentive
All	51.63%	55.50%1	47.76%
3-Category Affiliation
Catholic	53.08%	58.32%	48.14%
Other Religious (non-Catholic)	51.91%	55.02%	48.65%
Nonsectarian	50.10%	53.83%	46.41%
School Level
Elementary	50.67%	54.42%	46.88%
Secondary	49.69%	54.25%	45.28%
Combined	54.00%	57.37%	50.58%
School Region
Northeast	48.71%	50.23%	47.13%
Midwest	58.02%	63.15%	53.23%
South	52.35%	58.73%	46.18%
West	47.76%	50.22%	45.14%
11-Category Affiliation
Catholic-parochial	51.51%	59.86%	43.42%
Catholic-diocesan	53.23%	54.37%	52.29%
Catholic-private	54.06%	62.02%	45.16%
Baptist	54.10%	60.47%	50.63%
Jewish	43.38%	40.28%	49.33%
Lutheran	67.70%	68.14%	67.26%
Seventh-Day Adventist	45.00%	41.18%	47.83%
Other Religious	50.53%	56.06%	44.85%
Nonsectarian - Regular	51.15%	56.84%	45.57%
Nonsectarian - Special Emphasis	41.49%	41.72%	41.28%
Nonsectarian - Special Education	57.44%	56.82%	58.18%
School Locale
City	48.98%	54.75%	43.41%
Suburban	54.20%	56.69%	51.73%
Town	57.14%	62.90%	50.32%
Rural	49.51%	50.69%	48.30%
School Size (Enrollment)
Enrollment: less than 100	45.25%	49.34%	40.92%
Enrollment: 100-199	52.57%	53.73%	51.43%
Enrollment: 200-499	53.34%	57.59%	48.64%
Enrollment: 500-749	52.53%	56.86%	48.70%
Enrollment: 750 or more	56.50%	64.14%	50.17%

¹ Cells in blue indicate the domains below 50%. Cells highlighted in red indicate that the response rate for the teacher incentive group is significantly higher than the response rate for the non-incentive group with 95% confidence.

Table 5. Current TQ Response Rates across and within the Public School Domains

Public School Domain	TQ Response Rates
	Overall	Incentive	Non-Incentive
All	48.30%	52.61%1	44.67%
Charter, Non-Charter
Charter	45.48%	46.00%	45.02%
Non-Charter	48.58%	53.29%	44.63%
School Level
Primary	48.09%	52.87%	43.92%
Middle (school level)	50.18%	55.85%	45.58%
High (school level)	46.70%	49.80%	44.12%
Combined (school level)	50.59%	54.20%	47.65%
School Locale
City	39.18%	42.94%	36.55%
Suburban	49.39%	53.91%	44.87%
Town	56.69%	60.95%	52.91%
Rural	55.70%	58.68%	53.12%
School Size (Enrollment)
Enrollment: less than 100	48.99%	48.99%	48.99%
Enrollment: 100-199	53.21%	59.59%	47.82%
Enrollment: 200-499	51.10%	55.91%	46.88%
Enrollment: 500-749	47.18%	51.75%	43.30%
Enrollment: 750-999	49.71%	52.42%	47.34%
Enrollment: 1000 or more	43.26%	47.08%	40.30%
Free/Reduce Price Lunch Participation
Free/Reduced Price Lunch: 0-34%	52.38%	53.74%	51.07%
Free/Reduced Price Lunch: 35-49%	52.07%	57.44%	47.59%
Free/Reduced Price Lunch: 50-74%	48.07%	52.59%	44.50%
Free/Reduced Price Lunch: 75% or more	40.19%	46.39%	35.26%
Free/Reduced Price Lunch: DNP	51.71%	61.81%	44.66%

¹ Cells in blue indicate the domains with a response rate below 50%. Cells highlighted in red indicate that the response rate for the teacher incentive group is significantly higher than the response rate for the non-incentive group with 95% confidence.

The response rates reported in tables 4 and 5 do not take into account the multiplicative effect of response rates from the Teacher Listing Form (TLF), which affects the final teacher response rates. The TLF response rate is approximately at 59% for most domains. While the highlighted cells show that the incentives are effective at increasing response rates, the final teacher response rates are still well below NCES’s publishability standards (a minimum response rate of 50% or higher within each domain must be reached in order to publish results for that domain).

The results in tables 4 and 5 show that overall, for both private and public schools, the incentive group has a significantly higher response rate than the control group (teachers receiving no incentive) by approximately 8 percentage points. It should be noted that the private school 11-category affiliation breakout resulted in small cell sizes, thus a significant difference would be harder to detect, while the 3-category affiliation breakout showed significant difference between the incentive and non-incentive group, with the incentive group having higher response rates.

6. Historically Under-Performing Domains

Groups for which historically NCES has not been able to report results at the end of data collection activities are as follows:

1. Public School Domains - NTPS 2015-16 – upon review of the NTPS 2015-16 Teacher First Look document⁴, the following domains had one or more unpublishable estimates:

• City

• Charter

• Enrollment: Less than 100

• Free/Reduced-Price Lunch: 75% or more

2. Private School Domains – SASS 2011-12 – upon review of the SASS 2011-12 Teacher First Look document⁵, the following domains had one or more unpublishable estimates:

• Other Religious

• Nonsectarian

• Suburban

• Rural

• Combined

• Enrollment: Less than 100

• Enrollment: 100-199

• Enrollment: 750 or more

Note: Due to the SASS 2011-12 publishability issues, private schools were not part of NTPS 2015-16. They are included in NTPS 2017-18 to test collection strategies to increase response rates before bringing private school back in future collections.

These unpublishable domains from NTPS 2015-16 public school reports and 2011-12 SASS private school reports were considered as potential candidates for targeting as part of the contingency plan teacher incentive experiment. Teachers eligible for the contingency plan incentive experiment are teachers in waves 13 through 20. Teachers in these waves have historically been from schools in the domains listed above. Thus, targeting the contingency plan teacher incentive experiment to teachers in these school domains is expected to increase response rates in these domains.

7. Recommendation

The contingency incentive is being implemented as an experiment, and we will track the effectiveness among the incentivized groups to make a determination about implementing this type of incentive sample-wide in future NTPS administrations. Based on the previous NTPS and SASS school domains that historically did not meet NCES publishability standards, the current status of the teacher response rates, and the initial results of the teacher incentives experiment, domains recommended to be targeted for contingency plan teacher incentive experiment are based on the following criteria:

1. The domain was previously an unpublished domain

2. The domain currently has a lower response rate than domains of the same type

3. The domain has shown a significant increase in response due to the initial incentive

Based on these criteria, and our estimation of the group having the best chance of reaching publishablity with the contingency incentive while maintaining the cost effectiveness of the operation, the following domains are recommended to be targeted in the contingency plan teacher incentive experiment:

Private School Domain: Public School Domain:

Secondary High Schools

Enrollment: Less than 100 Combined Schools

City Enrollment: 1000 or more

Rural City

Other religious Free/Reduced Price Lunch: 75% or more

Nonsectarian Charter

The number of teachers eligible for the contingency incentive will not exceed the totals provided in table 11 of the approved Supporting Statement Part B (p.23): 2,860 private school teachers and 15,230 public school teachers.

1 The teacher samples sizes will not be exactly equal across experimental groups, as the number of teachers sampled from each school is not equal. However, each group should contain roughly the same number of teachers.

2 All NTPS 2017-18 analyses included in this section reflect data collection results as of January 25^th, 2018

3 The teacher samples sizes will not be exactly equal across experimental groups, as the number of teachers sampled from each school is not equal. However, each group should contain roughly the same number of teachers.

4 For more information, please reference the “Characteristics of Public Elementary and Secondary School Teachers in the United States: Results from the 2015-16 NTPS” First Look document (https://nces.ed.gov/pubs2017/2017072.pdf)

5 For more information, please reference the “Characteristics of Public and Private Elementary and Secondary School Teachers in the United States: Results from the 2011-12 Schools and Staffing Survey” First Look document (https://nces.ed.gov/pubs2013/2013314.pdf)

Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation.