Appendix D - NTPS 2015-2016 Adaptive Design Report

Appendix D - NTPS 2015-2016 Adaptive Design Report.pdf

2017–18 National Teacher and Principal Survey (NTPS 2017-18)

Appendix D - NTPS 2015-2016 Adaptive Design Report

OMB: 1850-0598

Document [pdf]
Download: pdf | pdf
f

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Authors
Minsun Riddles
Leslie Wallace

Lou Rizzo
David Marker

August 22, 2017

Prepared for:
National Center of Education Statistics
550 12th Street SW
Washington, DC 20006

Prepared by:
Westat
An Employee-Owned Research Corporation®
1600 Research Boulevard
Rockville, Maryland 20850-3129
(301) 251-1500

Table of Contents
Chapter

Page

1

Introduction ........................................................................................................

1

2

School Questionnaire Response Experience in NTPS 2015-16 ..................

4

2.1
2.2

5
9

Response Rate .......................................................................................
Follow Up ..............................................................................................

3

Principal Questionnaire Response Experience in NTPS 2015-16 ..............

16

4

Teacher Listing Form Response Experience in NTPS 2015-16..................

18

5

Teacher Questionnaire Response Rate in NTPS 2015-16 ............................

27

6

Teacher Questionnaire Response Experience in NTPS 2015-16 ................

38

7

Experimental Study in NTPS 2015-16 ............................................................

43

8

Summary and Conclusions ................................................................................

46

A

School Control Data File Analysis: Details and Assumptions
Made .....................................................................................................................

A-1

B

Teacher Control Data File Analysis: Details and Assumptions
Made .....................................................................................................................

B-1

C

School Questionnaire Experience: Detailed Tables ......................................

C-1

D

Teacher Questionnaire Response Rates: Detailed Tables ...........................

D-1

E

Teacher Questionnaire Response Experience: Detailed Tables ..................

E-1

F

Experimental Study Detailed Tables................................................................

F-1

2-1.

SASS 2011-12 school response and eligibility rates .......................................

4

2-2.

NTPS 2015-16 school response and eligibility rates .....................................

4

Appendixes

Tables

NTPS 2015-2016 Analysis of Response Rates
and Field Collection Experience

iii

Contents (continued)

Tables (continued)

Page

2-3.

NTPS 2015-16 school eligibility and response rates, major
domains ................................................................................................................

6

2-4.

SASS 2011-12 Linear regression parameter estimates, standard
errors, t-statistics, p-values for school completion rates...............................

8

2-5.

NTPS 2015-16 Linear regression parameter estimates, standard
errors, t-statistics, p-values for school completion rates...............................

9

2-6.

Number of schools by priority status and by survey coordinator
status, NTPS 2015-16 ........................................................................................

11

2-7.

Response follow-up experience with school interview final
respondents, among non-priority schools with a survey
coordinator, NTPS 2015-16 ..............................................................................

12

Response follow-up experience with school interview final
respondents, among non-priority schools without a survey
coordinator ..........................................................................................................

13

2-9.

Response follow-up experience with school interview final
respondents, among priority schools ...............................................................

13

2-10.

Response follow-up experience with school interview final
respondents, among priority schools with a survey coordinator .................

13

2-11.

Response follow-up experience with school interview final
respondents, among priority schools without a survey
coordinator ..........................................................................................................

14

3-1.

NTPS 2015-16 principal eligibility and response rates, major
domains ................................................................................................................

17

4-1.

NTPS 2015-16 TLF eligibility and response rates, major domains.............

19

4-2.

Response follow-up experience for final TLF respondents among
non-priority schools with a survey coordinator, NTPS 2015-16.................

20

4-3.

Response follow-up experience for final TLF respondents among
non-priority schools without a survey coordinator, NTPS 201516 ...........................................................................................................................

20

2-8.

NTPS 2015-2016 Analysis of Response Rates
and Field Collection Experience

iv

Contents (continued)

Tables (continued)

Page

4-4.

Response follow-up experience for final TLF respondents among
priority schools, NTPS 2015-16 .......................................................................

21

4-5.

Response follow-up experience for final TLF respondents among
priority schools with a survey coordinator, NTPS 2015-16 .........................

21

4-6.

Response follow-up experience for final TLF respondents among
priority schools without a survey coordinator, NTPS 2015-16 ...................

21

4-7.

Weighted distribution of number of days after initial school mailout for initially fielding teacher questionnaires for SASS 2011-12
and NTPS 2015-16. ............................................................................................

23

4-8.

Regression coefficients for number of eligible teachers ...............................

25

4-9.

Regression coefficients for teacher eligibility rate..........................................

26

5-1.

SASS 2011-12 public school teacher response and eligibility rates .............

27

5-2.

NTPS 2015-16 public school teacher response and eligibility rates............

27

5-3.

NTPS 2015-16 teacher weighted response rates, with and without
alternative TLFs included ..................................................................................

28

5-4.

NTPS 2015-16 teacher eligibility and response rates, major school
domains ................................................................................................................

30

5-5.

NTPS 2015-16 public school teacher wave and initial mailout
date........................................................................................................................

31

5-6.

NTPS 2015-16 teacher eligibility and response rates for early
teacher wave group, major school domains....................................................

33

5-7.

NTPS 2015-16 teacher eligibility and response rates for middle
teacher wave group, major school domains....................................................

34

5-8.

NTPS 2015-16 teacher eligibility and response rates for late
teacher wave group, major school domains....................................................

35

NTPS 2015-2016 Analysis of Response Rates
and Field Collection Experience

v

Contents (continued)

Tables (continued)
5-9.

Page

NTPS 2015-16 teacher eligibility and response rates by
nonresponse follow-up experimental group, major school
domains ................................................................................................................

37

6-1.

Response follow-up experience with teacher interview final
respondents .........................................................................................................

38

6-2.

Response follow-up experience with teacher interview final
respondents in early teacher wave group ........................................................

38

6-3.

Response follow-up experience with teacher interview final
respondents in middle teacher wave group ....................................................

39

6-4.

Response follow-up experience with teacher interview final
respondents in late teacher wave group ..........................................................

39

6-5.

Response follow-up experience with teacher interview final
respondents from non-priority schools without a survey
coordinator ..........................................................................................................

40

6-6.

Response follow-up experience with teacher interview final
respondents from non-priority schools with a survey coordinator.............

40

6-7.

Response follow-up experience with teacher interview final
respondents from priority schools without a survey coordinator ...............

40

6-8.

Response follow-up experience with teacher interview final
respondents from priority schools with a survey coordinator .....................

41

6-9.

Response follow-up experience with teacher-interview final
respondents by nonresponse experimental group, NTPS 2015-16.............

42

7-1.

Response rate comparison by experimental group and by
questionnaire, NTPS 2015-16 ...........................................................................

44

A-1.

Relevant stages of data collection by type of case .........................................

A-4

A-2.

Setting the “in” flags for each process.............................................................

A-4

A-3.

Rules for assigning follow-up experience flags by type of case ...................

A-6

NTPS 2015-2016 Analysis of Response Rates
and Field Collection Experience

vi

Contents (continued)

Tables (continued)

Page

B-1.

Relevant stages of data collection for teachers ...............................................

B-2

B-2.

Setting the “in” flags for each process.............................................................

B-3

B-3.

Rules for assigning follow-up experience flag by flag value, all
teachers .................................................................................................................

B-4

C-1.

Percentage of school respondents that received no/limited
follow-up by school domain, among non-priority schools with a
survey coordinator ..............................................................................................

C-3

Percentage of school respondents that received survey
coordinator follow-up (telephone reminder) and mailout by
school domain, among non-priority schools with a survey
coordinator ..........................................................................................................

C-4

Percentage of school respondents that received telephone and
mail follow-up by school domain, among non-priority schools
with a survey coordinator ..................................................................................

C-5

Percentage of school respondents that received field follow-up by
school domain, among non-priority schools with a survey
coordinator ..........................................................................................................

C-6

C-5.

Percentage of school respondents that received no/limited
follow-up by school domain, among priority schools ...................................

C-8

C-6.

Percentage of school respondents that received phase 1 follow-up
by school domain, among priority schools .....................................................

C-9

C-7.

Percentage of school respondents that received further follow-up
after phase 1 by school domain, among priority schools .............................

C-10

C-8.

Percentage of school respondents that received no/limited
follow-up by school domain, among non-priority schools without
a survey coordinator ...........................................................................................

C-12

Percentage of school respondents that received phase 1 follow-up
by school domain, among non-priority schools without a survey
coordinator ..........................................................................................................

C-13

C-2.

C-3.

C-4.

C-9.

NTPS 2015-2016 Analysis of Response Rates
and Field Collection Experience

vii

Contents (continued)

Tables (continued)
C-10.

Page

Percentage of school respondents that received further follow-up
after phase 1 by school domain, among non-priority schools
without a survey coordinator ............................................................................

C-14

D-1.

NTPS 2015-16 teacher eligibility and response rates, teacher
domains ................................................................................................................

D-3

D-2.

NTPS 2015-16 teacher eligibility and response rates for teacher
early wave group, teacher domains ..................................................................

D-5

D-3.

NTPS 2015-16 teacher eligibility and response rates for middle
teacher wave group, teacher domains ..............................................................

D-7

D-4.

NTPS 2015-16 teacher eligibility and response rates for late
teacher wave group, teacher domains ..............................................................

D-9

D-5A.

NTPS 2015-16 teacher eligibility and response rates by
nonresponse follow-up experimental group, major school
domains ................................................................................................................

D-12

D-5B.

NTPS 2015-16 teacher eligibility and response rates by
nonresponse follow-up experimental group, teacher domains ....................

D-14

E-1A.

Percentage of teacher respondents with no/limited follow-up by
school domain by teacher wave group, NTPS 2015-16 ................................

E-3

E-1B.

Percentage of teacher respondents with no/limited follow-up by
teacher domain by teacher wave group, NTPS 2015-16...............................

E-4

E-2A.

Percentage of teacher respondents with telephone/mail follow-up
by school domain by teacher wave group, NTPS 2015-16 ..........................

E-6

E-2B.

Percentage of teacher respondents with telephone/mail follow-up
by teacher domain by teacher wave group, NTPS 2015-16 ........................

E-7

E-3A.

Percentage of teacher respondents with field follow-up by school
domain by teacher wave group, NTPS 2015-16 ............................................

E-9

E-3B.

Percentage of teacher respondents with telephone/mail follow-up
by teacher domain by teacher wave group, NTPS 2015-16 ........................

E-10

NTPS 2015-2016 Analysis of Response Rates
and Field Collection Experience

viii

Contents (continued)

Tables (continued)
E-4.

Page

Percentage of teacher respondents with telephone or mail followup by nonresponse follow-up experimental group by
school/teacher domain, NTPS 2015-16..........................................................

E-13

F-1.

School response rate comparison by experimental group and by
school domain, NTPS 2015-16 ........................................................................

F-3

F-2.

Principal response rate comparison by experimental group and by
school domain, NTPS 2015-16 ........................................................................

F-4

F-3.

Teacher listing form response rate comparison by experimental
group and by school domain, NTPS 2015-16 ................................................

F-5

1-1.

2015-16 NTPS production data collection operations..................................

3

2-1.

Follow-up required prior to response with school-interview final
respondents by survey coordinator status and by priority status,
NTPS 2015-16.....................................................................................................

14

Follow-up required prior to response with TLF final respondents
by survey coordinator status and by priority status, NTPS 201516 ...........................................................................................................................

22

5-1.

NTPS 2015-16 public school teacher response rate by teacher
wave group...........................................................................................................

32

6-1.

Follow-up required prior to response with teacher-interview final
respondents by teacher wave group, NTPS 2015-16 ....................................

39

6-2.

Follow-up required prior to response with teacher-interview final
respondents by survey coordinator status and by priority status,
NTPS 2015-16.....................................................................................................

41

Figures

4-1.

NTPS 2015-2016 Analysis of Response Rates
and Field Collection Experience

ix

Introduction

1

The National Teachers and Principals Survey (NTPS) has an active program in adaptive design.
Response rates to federal surveys have been dropping almost universally, and the Schools and
Staffing Survey (SASS) has not been immune from this long-term secular decline in response rates.
The National Center of Education Statistics has been committed to overcoming this decline and
ameliorating the effects of this decline (where it can’t be overcome) through aggressive, systematic
adjustments in its field data collection. This effort includes changes in fielding procedures and
experimentation with new methods. For the 2015-16 NTPS, the following changes were made:


School survey coordinators were recruited within as many schools as possible, given the
good experience with this collection approach in SASS 2011-12.



Schools were defined as priority schools and non-priority schools, based on their
relative importance and our understanding of how difficult it would be to recruit them
for NTPS 2015-16. Among the factors which lead to a definition of a school as a
priority school is that it is in a ‘special district’ (a district which requires specialized
recruitment), it is a school with a large weight (making it important in estimates), and/or
it has a low propensity to respond based on its characteristics (e.g., it is a city school, a
high poverty school, a high school).

Priority schools received more aggressive data collection efforts in NTPS 2015-16, including a field
visit fairly early in the data collection process, bypassing telephone and mail reminder phases. Nonpriority schools with no school coordinator recruited also received these more aggressive data
collection efforts. Figure 1-1 below presents graphically the 2015-16 NTPS production data
collection operations. Results are presented in Sections 2 through 6, with details in Appendices A
through E.
Another change from SASS 2011-12 was the use of alternative methods for collecting Teacher
Listing Forms (TLFs). The only purpose of collecting TLFs is to provide a frame for teacher
sampling, with the teacher samples being selected for receiving teacher questionnaires. Any delay in
receiving TLFs from the schools leads to late teacher samples being drawn for the school, which
compresses the time period for teacher data collection. This in turn reduces teacher response rates.
Receiving late TLFs may be no better ultimately than receiving none at all if little time is left for
follow-up teacher questionnaire data collection. In NPTS 2015-16, two alternative methods were put
into place for collecting TLFs from non-cooperative schools: the collection of TLFs from
NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

1

Introduction

1

commercial vendors, and clerical operations utilizing school websites. In the former case, a
commercial vendor of teacher lists was utilized to provide teacher lists to NCES from schools that
did not return TLFs quickly. This was found to be successful in the Pilot Test in the 2014-15 school
year (the lists were found to compare well to TLFs also collected from the school itself, and the
whole process was found to be much less expensive than direct TLF collection from the schools). In
the latter case, Census personnel went onto publicly accessible school or district websites to collect
teacher lists, where those were available online. Results of this experiment are presented in Sections
5 and 6, and Appendices D and E.
A field collection experiment was carried out for teacher questionnaire data collection. In this field
experiment teachers who were assigned to telephone nonresponse follow-up (excluding those who
sent in their questionnaires before this phase) were randomly assigned to either receive the regular
series of reminder telephone calls, or to receive a special series of late afternoon telephone calls. The
regular series of telephone cells are done throughout the school day before 2:00 pm, with further
later afternoon telephone calls between 2:00 pm and 5:00 pm. This was done on the theory that
teachers might be easier to reach at this point in the school day than at other times. Results of this
are presented in Section 6, with details in Appendix E.
In SASS 2011-12, an experiment was carried out on doing much of the data collection through the
internet. Internet data collection, if successful, can considerably reduce costs, so that even if
response rate outcomes and the characteristics of respondents through the internet mode are
equivalent to direct data collection, there would be a strong argument for this data collection regime.
Unfortunately, the experience in SASS 2011-12 was not positive: while the respondent
characteristics were similar regardless of the data collection mode, response rates from the internet
data collection branch were significantly lower. Internet data collection was dropped from the main
data collection all together, but NCES decided to at least try this branch with some modifications in
the data collection as a stand-alone experiment with 1,000 schools (beyond the 8,300 schools in the
main 2015-16 NTPS sample). The results of this experiment are provided in Section 7 and Appendix
F.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

2

Introduction

Figure 1-1.

2015-16 NTPS production data collection operations

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

3

1

School Questionnaire
Response Experience in NTPS 2015-16

2

We begin with a comparison of SASS 2011-12 and NTPS 2015-16 in the overall school
questionnaire response experience, and by domain. Some of this difference is the difference between
years (changes in response “environment”), and some will be due to shifts in data collection between
the two cycles. Differences in sample design (differing oversampling rates between SASS 2011-12
and NTPS 2015-16) can be eliminated by utilizing weighted response rates.
Table 2-1 below presents unweighted and weighted counts from the SASS 2011-12 school sample.
The weighted counts utilize the school base weight (the sum of the weighted counts is an estimate of
the total number of schools).
Table 2-1.

SASS 2011-12 school response and eligibility rates1

SASS 2011-12
School outcomes
Completes
Nonrespondents
Ineligible
Total sample

Survey count
7,481
2,874
645
11,000

Survey unweighted
percent
68.01%
26.13%
5.86%
100.00%

Weighted survey
count
64,960
24,825
5,595
95,380

Survey weighted
percent
68.11%
26.03%
5.87%
100.00%

Table 2-2 presents the same information from the NTPS 2015-16 school sample.
Table 2-2.

NTPS 2015-16 school response and eligibility rates2

NTPS 2015-16
School outcomes
Completes
Nonrespondents
Ineligible
Total sample

Survey count
5,765
2,262
273
8,300

Survey unweighted
percent
69.46%
27.25%
3.29%
100.00%

Weighted survey
count
66,058
25,036
3,738
94,832

Survey weighted
percent
69.66%
26.40%
3.94%
100.00%

1

This is the same as Table E-1A in the report “NTPS 2017-18 Public School Sample Design Including State-Level
Estimates,” dated March 3, 2017.

2

This is the same as Table E-1B in the report “NTPS 2017-18 Public School Sample Design Including State-Level
Estimates,” dated March 3, 2017.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

4

School Questionnaire
Response Experience in NTPS 2015-16

2.1

2

Response Rate

The percentages of nonrespondents for SASS 2011-12 and NTPS 2015-16 are broadly comparable,
though the percentage of ineligible schools has dropped in NTPS 2015-16 as compared to SASS
2011-12. One can say that NTPS 2015-16 school questionnaire response is about the same as SASS
2011-12 school questionnaire response overall.
Eligibility and response rates for NTPS 2015-16 school questionnaires are presented in Table 2-3.
Both rates were computed unweighted and weighted. Table 2-3 also includes the standard errors and
95% confidence intervals for the weighted response rates. The school base full sample and replicate
weights are used for any statistics for weighted rates. The response rates for domains with a
statistically significant difference in response rate are in bold and italic. Rao-Scott Chi-Square tests
are conducted to detect any differences. Response rates for school questionnaires differ by special
district flag, school span, urbanicity, Census region, poverty level, and school size. Response rates
are lower for schools in special districts, city and suburban schools, Northeastern schools, low
poverty schools (percentage of students eligible for free or reduced-price lunch less than 34%), and
high poverty schools (percentage of students eligible for free or reduced-price lunch equal to or
greater than 75%). Response rates are higher for combined schools, Midwestern schools, and those
with an enrollment of 100-199 students in NTPS 2015-16.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

5

Special district
Not special district

1,449
6,851

1,421
6,608

822
4,952

98.1%
96.5%

57.8%
74.9%

97.2%
95.9%

58.3%
75.6%

1.2%
0.5%

56.0%
74.5%

60.6%
76.7%

Charter
Non-charter

1,173
7,127

1,094
6,935

783
4,991

93.3%
97.3%

71.6%
72.0%

92.5%
96.4%

73.1%
72.6%

1.4%
0.5%

70.3%
71.5%

75.9%
73.6%

Primary
Middle
High
Combined

3,708
1,441
2,054
1,097

3,626
1,420
1,981
1,002

2,607
1,003
1,377
787

97.8%
98.5%
96.4%
91.3%

71.9%
70.6%
69.5%
78.5%

97.8%
98.3%
93.8%
87.5%

72.7%
72.1%
70.6%
78.8%

0.7%
1.2%
1.3%
1.4%

71.2%
69.9%
68.0%
76.0%

74.1%
74.4%
73.1%
81.6%

City
Suburban
Town
Rural

2,507
2,585
1,201
2,007

2,395
2,520
1,161
1,953

1,564
1,715
930
1,565

95.5%
97.5%
96.7%
97.3%

65.3%
68.1%
80.1%
80.1%

94.9%
96.7%
95.7%
96.9%

65.3%
69.3%
79.5%
80.7%

1.1%
0.9%
1.4%
0.9%

63.2%
67.5%
76.8%
78.8%

67.4%
71.1%
82.1%
82.5%

6

NTPS 2015-16 school eligibility and response rates, major domains

Northeast
Midwest
South
West

1,352
1,924
3,200
1,824

1,332
1,849
3,087
1,761

890
1,434
2,181
1,269

98.5%
96.1%
96.5%
96.5%

66.8%
77.6%
70.7%
72.1%

98.1%
94.8%
96.0%
96.2%

67.3%
77.9%
71.3%
72.7%

1.3%
1.0%
0.9%
1.2%

64.8%
75.9%
69.6%
70.4%

69.8%
79.9%
73.0%
75.1%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

2,416
1,395
2,331
2,158

2,323
1,357
2,270
2,079

1,621
1,023
1,703
1,427

96.2%
97.3%
97.4%
96.3%

69.8%
75.4%
75.0%
68.6%

95.9%
96.7%
95.9%
96.2%

69.7%
75.8%
76.1%
70.0%

1.1%
1.2%
1.0%
1.2%

67.5%
73.4%
74.1%
67.7%

71.8%
78.2%
78.1%
72.4%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1,000+

404
551
2,820
2,151
1,035
1,339

342
512
2,733
2,098
1,022
1,322

262
428
2,027
1,519
692
846

84.7%
92.9%
96.9%
97.5%
98.7%
98.7%

76.6%
83.6%
74.2%
72.4%
67.7%
64.0%

80.5%
94.7%
97.5%
97.9%
98.9%
99.0%

73.6%
84.0%
74.2%
71.9%
67.8%
64.3%

3.0%
1.8%
0.9%
1.0%
1.5%
1.3%

67.6%
80.4%
72.3%
69.9%
64.7%
61.6%

79.6%
87.5%
76.0%
73.9%
70.8%
67.0%

Regular
Special education
Vocational
Alternative

7,641
158
111
390

7,485
125
99
320

5,392
89
74
219

98.0%
79.1%
89.2%
82.1%

72.0%
71.2%
74.7%
68.4%

97.9%
74.8%
92.1%
78.8%

73.0%
72.2%
74.7%
66.3%

0.5%
5.4%
5.9%
3.5%

72.0%
61.5%
63.1%
59.4%

73.9%
82.9%
86.4%
73.2%

Domain
All

Sample
size
8,300

Number of
Eligible
schools
Completes
8,029
5,774*

Unweighted
Eligibility
Response
rate
rate
96.7%
71.9%

Eligibility
rate
96.1%

Response
rate
72.6%

Weighted
Std.
error
0.5%

Lower
bound CI
71.6%

Upper
bound CI
73.6%

2

* The numbers of eligible cases and completed cases are different from the numbers presented in Table 2-2. The discrepancy is caused by the discrepancy between the status on the school
control data file and the ISR on the final school DOC file.

School Questionnaire
Response Experience in NTPS 2015-16

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 2-3.

School Questionnaire
Response Experience in NTPS 2015-16

2

Our approach was to use as our base analysis a fit of a weighted linear regression model on the
completion status variable (1 if complete, 0 if nonrespondent or ineligible) as a dependent variable,
with class predictor variables by urbanicity, school size, school span, and poverty status. We chose a
linear regression model rather than a logistic regression model as we wanted the predicted
propensity estimates values to have the same mean within each domain as the completion status
variable. This only occurs with linear regression; logistic regression has a nonlinear ‘link’ function,
and the completion propensities derived from the nonlinear model will not be fully consistent with
the actual completion rates.3 Table 2-4 presents the results from this weighted linear regression
model on the SASS 2011-12 school sample. The weights are the school base weights, and the
standard errors are based on the replicate base weights from SASS 2011-12 (PROC SURVEYREG
on SAS was utilized to do this fit). For all domain sets, the parameter estimates are in terms of the
“last” level (rural for urbanicity, high poverty for poverty, greater than 1,000 enrollment for school
size). The F-statistics for testing the null hypothesis of no effect for the domain set as a predictor of
completion rates are highly significant for all domain sets. Table 2-5 presents the same model fitted
to the NTPS 2015-16 school sample (with its weights and replicate weights).
The NTPS 2015-16 differences in Table 2-5 are less than the SASS 2011-12 differences in Table 2-4,
indicating a success in the adaptive design for NTPS 2015-16, which had as its objective the
reduction of differences in response rates across these important school domains. The response
gaps between city and suburban schools on one side, and town and rural schools on the other side,
were smaller in NTPS 2015-16 compared to SASS 2011-12. The response gap between high poverty
(percent students eligible for free or reduced-price lunch greater than 75%) and other schools was
smaller in NPTS 2015-16 compared to SASS 2011-12. The gap between combined schools and
other schools was smaller as well. The current paradigm for reducing response bias stresses the
reduction of differences between response propensities as much as increasing the overall response
rate4, and on this score NTPS 2015-16 was a great success.

3

One drawback to a linear regression model is that the predicted propensity values can be less than 0 or greater than 1.
A logistic regression model avoids this. But this did not occur in this model fit.

4

See for example Schouten, B., Cobben, F., and Bethlehem, J. (2009), Indicators for the Representativeness of Survey
Response,” Survey Methodology 35, 101-113.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

7

School Questionnaire
Response Experience in NTPS 2015-16

Table 2-4.

2

SASS 2011-12 Linear regression parameter estimates, standard errors, t-statistics,
p-values for school completion rates5
Domain

Primary
Middle
High
Combined
City
Suburban
Town
Rural
Percent FRPL* < 35%
35% <= Percent FRPL < 50%
50% <= Percent FRPL < 75%
75% <= Percent FRPL
Enrollment < 100
100 <= Enrollment < 200
200 <= Enrollment < 500
500 <= Enrollment < 750
750 <= Enrollment < 1000
1000 <= Enrollment

Parameter estimate
(percentage completion)
11.40%
12.64%
7.13%
0.00%
-24.58%
-15.05%
-2.32%
0.00%
5.63%
5.08%
5.77%
0.00%
-11.18%
-5.51%
2.23%
0.26%
-1.00%
0.00%

Standard error
of estimate
3.13%
3.49%
3.13%
0.00%
1.84%
1.67%
2.49%
0.00%
2.13%
2.06%
1.62%
0.00%
2.94%
3.41%
1.85%
2.79%
2.59%
0.00%

T-statistic
3.64
3.63
2.27
.
-13.34
-8.99
-0.93
.
2.64
2.46
3.56
.
-3.80
-1.61
1.21
0.09
-0.39
.

P-value
0.0005
0.0005
0.0254
.
<.0001
<.0001
0.3551
.
0.0098
0.0158
0.0006
.
0.0003
0.1101
0.2313
0.9254
0.6991
.

* Percent of students eligible for free or reduced-price lunch.

5

Note that this is identical to Table E-2A in our “NTPS 2017-18 Public School Sample Design Including State-Level
Estimates” report dated March 3, 2017.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

8

School Questionnaire
Response Experience in NTPS 2015-16

Table 2-5.

NTPS 2015-16 Linear regression parameter estimates, standard errors, t-statistics,
p-values for school completion rates6
Domain

Primary
Middle
High
Combined
City
Suburban
Town
Rural
Percent FRPL* < 35%
35% <= Percent FRPL < 50%
50% <= Percent FRPL < 75%
75% <= Percent FRPL
Enrollment < 100
100 <= Enrollment < 200
200 <= Enrollment < 500
500 <= Enrollment < 750
750 <= Enrollment < 1000
1000 <= Enrollment
*

2

Parameter estimate
(percentage completion)
3.63%
3.59%
-0.25%
0.00%
-16.09%
-10.83%
-2.13%
0.00%
-2.40%
1.95%
2.03%
0.00%
-9.06%
8.80%
3.08%
2.97%
0.66%
0.00%

Standard error
of estimate
1.86%
2.13%
2.21%
0.00%
1.70%
1.65%
1.88%
0.00%
1.84%
1.90%
1.71%
0.00%
3.71%
2.47%
1.96%
1.90%
2.24%
0.00%

T-statistic
1.95
1.68
-0.11
.
-9.47
-6.57
-1.13
.
-1.3
1.03
1.19
.
-2.44
3.56
1.57
1.56
0.3
.

P-value
0.0528
0.0936
0.9112
.
<.0001
<.0001
0.258
.
0.1936
0.3063
0.2373
.
0.0154
0.0005
0.1175
0.1196
0.7667
.

Percent of students eligible for free or reduced-price lunch.

2.2

Follow Up

Among the schools that responded to the school questionnaire, the degree of follow-up necessary to
achieve a positive response outcome (i.e., a completed interview) is studied. In NTPS 2015-16,
schools received different follow-up depending on whether the school had a survey coordinator or
not, and whether the school is a priority school.
Table 2-6 presents the number of sampled schools by priority status and by survey coordinator
status. The priority status was determined based on the response rates for SASS 2011-12. School
domains with lower response rates got priority in terms of follow-up effort in order to boost the
response rates of the low-responding domains for this cycle. Most of the schools in special districts
are priority schools while almost all non-special district schools are non-priority ones. City schools,
high poverty schools, large schools, and alternative schools have higher percentages of priority
schools than other types of schools. The survey coordinator status also varies by school domain.
The domains with higher percentages of priority schools coincide with the domains with lower
6

Note that this is identical to Table E-2Bd in our “NTPS 2017-18 Public School Sample Design Including State-Level
Estimates” report dated March 3, 2017.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

9

School Questionnaire
Response Experience in NTPS 2015-16

2

percentages of schools with a survey coordinator (a survey coordinator was recruited at each school:
those schools without a coordinator were self-selected as such). This can be evidence that these
domains are still less cooperative than others. These domains are shaded in the table.
Among the final school-interview respondents that are non-priority schools with a survey
coordinator, we break out weighted percentages to the following four categories (every final school
interview falls into one of these follow-up-level categories):


School interview completed before third school mailout (no or limited follow-up);



School interview completed before telephone follow-up (survey coordinator telephone
reminder and third or fourth mailout);



School interview completed before Phase 2 field follow-up (follow-up including all
telephone and mailout phases, no field); or



School interview completed after Phase 2 field follow-up (follow-up including field
follow-up).

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

10

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 2-6.

Number of schools by priority status and by survey coordinator status, NTPS 2015-16

Domain
Special district
Not special district

Sampled schools
1,449
6,851

Non-priority
schools
267
6,809

Priority
schools
1,182
42

Percent
priority
schools
81.6%
0.6%

Schools without a
survey
coordinator
927
3,398

Schools with a
survey
coordinator
522
3,453

Percent
schools
with a survey
coordinator
36.0%
50.4%

11

1,010
6,066

163
1,061

13.9%
14.9%

641
3,684

532
3,443

45.4%
48.3%

Primary
Middle
High
Combined

3,708
1,441
2,054
1,097

3,112
1,230
1,753
981

596
211
301
116

16.1%
14.6%
14.7%
10.6%

1,862
754
1,143
566

1,846
687
911
531

49.8%
47.7%
44.4%
48.4%

City
Suburban
Town
Rural

2,507
2,585
1,201
2,007

1,725
2,196
1,174
1,981

782
389
27
26

31.2%
15.0%
2.2%
1.3%

1,526
1,358
560
881

981
1,227
641
1,126

39.1%
47.5%
53.4%
56.1%

Northeast
Midwest
South
West

1,352
1,924
3,200
1,824

1,131
1,803
2,642
1,500

221
121
558
324

16.3%
6.3%
17.4%
17.8%

733
880
1,729
983

619
1,044
1,471
841

45.8%
54.3%
46.0%
46.1%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

2,416
1,395
2,331
2,158

2,188
1,251
2,058
1,579

228
144
273
579

9.4%
10.3%
11.7%
26.8%

1,186
664
1,164
1,311

1,230
731
1,167
847

50.9%
52.4%
50.1%
39.2%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

404
551
2,820
2,151
1,035
1,339

339
502
2,463
1,830
851
1,091

65
49
357
321
184
248

16.1%
8.9%
12.7%
14.9%
17.8%
18.5%

217
268
1,418
1,092
573
757

187
283
1,402
1,059
462
582

46.3%
51.4%
49.7%
49.2%
44.6%
43.5%

Regular
Special education
Vocational
Alternative

7,641
158
111
390

6,543
133
97
303

1,098
25
14
87

14.4%
15.8%
12.6%
22.3%

3,914
91
57
263

3,727
67
54
127

48.8%
42.4%
48.6%
32.6%

2

1,173
7,127

School Questionnaire
Response Experience in NTPS 2015-16

Charter
Non-charter

School Questionnaire
Response Experience in NTPS 2015-16

2

These categories are ordered by the level of follow-up that was required for responding schoolinterviews. Appendix A provides details regarding the assumption and analyses based on the
available paradata that led to defining each responding school questionnaire to one of these four
categories. Table 2-7 presents the distribution of four response follow-up categories among the nonpriority schools with a survey coordinator that completed the school-interview. Table 2-7 shows that
almost three fourths of the responding schools participated in the survey with no or limited followup and almost 90% of the responding schools responded before any telephone or field follow-up
phases. The table also suggests that up to about 7% of the school respondents would have been lost
without field follow-up. It should be noted that since some of them might have eventually
responded without field follow-up, the potential lost could have been smaller than 7%.
Table 2-7.

Response follow-up experience with school interview final respondents, among nonpriority schools with a survey coordinator, NTPS 2015-16

Response follow-up category
No or limited follow-up
Survey coordinator telephone reminder & mailout
Telephone and mail follow-up, no field
Field follow-up
Total respondents

Unweighted
respondent
count
2,202
552
164
209
3,127

Unweighted
respondent
percent
70.42%
17.65%
5.24%
6.68%
100.00%

Weighted
respondent
count
34,421
8,424
2,328
3,041
48,213

Weighted
respondent
percent
71.39%
17.47%
4.83%
6.31%
100.00%

Standard
error
0.88%
0.75%
0.42%
0.47%

For the final school-interview respondents that are priority schools or non-priority schools without a
survey coordinator, we break out weighted percentages to the following three categories.


School interview completed before Phase 1 field follow-up (no or limited follow-up);



School interview completed after Phase 1 field follow-up (follow-up with Phase 1 field
follow-up); and



School interview completed after further follow-up after Phase 1 field follow-up
(follow-up including all phases);

Tables 2-8 and 2-9 show the response follow-up experience for non-priority schools without a
survey coordinator and for priority schools, respectively. Due to low response rate from these
schools in the past cycle, these schools received field follow-up more promptly than the non-priority
school with a survey coordinator.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

12

School Questionnaire
Response Experience in NTPS 2015-16

Table 2-8.

Response follow-up experience with school interview final respondents, among nonpriority schools without a survey coordinator

Response follow-up category
No or limited follow-up
Phase 1 field follow-up
Further follow-up after phase 1
Total respondents

Table 2-9.

2

Unweighted
respondent
count
561
1,139
248
1,948

Unweighted
respondent
percent
28.80%
58.47%
12.73%
100.00%

Weighted
respondent
count
9,067
17,180
3,750
29,997

Weighted
respondent
percent
30.23%
57.27%
12.50%
100.00%

Standard
error
1.18%
1.23%
0.89%

Response follow-up experience with school interview final respondents, among
priority schools

Response follow-up category
No or limited follow-up
Phase 1 field follow-up
Further follow-up after phase 1
Total respondents

Unweighted
respondent
count
315
299
85
699

Unweighted
respondent
percent
45.06%
42.78%
12.16%
100.00%

Weighted
respondent
count
5,315
5,437
1,442
12,195

Weighted
respondent
percent
43.59%
44.59%
11.83%
100.00%

Standard
error
1.89%
1.86%
1.32%

Almost 90% of non-priority schools without a survey coordinator completed the survey without
further follow-up after phase 1 field follow-up, and only about 30% of non-priority schools without
a survey coordinator completed the survey before any field follow-up.
Similar to non-priority schools without a survey coordinator almost 90% of priority schools
completed the survey without further follow-up after phase 1 field follow-up. However, about 45%
of responding priority schools completed the survey before any filed follow-up.
As mentioned earlier, some priority schools had a survey coordinator and the others did not.
Although the data collection procedure was the same for priority schools in NTPS 2015-16
regardless of whether a survey coordinator is recruited or not, we looked at the follow-up experience
of priority schools by survey coordinator status in Tables 2-10 and 2-11. This was done in an effort
not only to see the relationship between having a survey coordinator and degree of follow-up
necessary to achieve a positive response outcome among priority schools, but also to control a
possible confounding effect of having a survey coordinator when priority schools are compared to
non-priority schools.
Table 2-10.

Response follow-up experience with school interview final respondents, among
priority schools with a survey coordinator

Response follow-up category

Unweighted

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Unweighted

13

Weighted

Weighted

Standard

2

School Questionnaire
Response Experience in NTPS 2015-16

No or limited follow-up
Phase 1 field follow-up
Further follow-up after phase 1
Total respondents

Table 2-11.

respondent
count
238
97
28
363

respondent
percent
65.56%
26.72%
7.71%
100.00%

respondent
count
3,985
1,663
475
6,123

respondent
percent
65.08%
27.16%
7.76%
100.00%

error
2.54%
2.40%
1.32%

Response follow-up experience with school interview final respondents, among
priority schools without a survey coordinator

Response follow-up category
No or limited follow-up
Phase 1 field follow-up
Further follow-up after phase 1
Total respondents

Unweighted
respondent
count
77
202
57
336

Unweighted
respondent
percent
22.92%
60.12%
16.96%
100.00%

Weighted
respondent
count
153
240
142
535

Weighted
respondent
percent
21.91%
62.16%
15.93%
100.00%

Standard
error
2.31%
2.79%
2.15%

Figure 2-1 presents the unweighted number of school respondents and weighted percentage for
follow-up experience summarizing Tables 2-7, 2-8, 2-10, and 2-11.
Figure 2-1.

Follow-up required prior to response with school-interview final respondents by
survey coordinator status and by priority status, NTPS 2015-16

Among responding priority schools, schools with a survey coordinator are more likely to respond
with lower degree follow-up effort than schools without one, which is consistent with our findings
from SASS 2011-2012. About 66% of responding priority schools with a survey coordinator sent
their completed questionnaire before Phase 1 field follow-up (consistent timing with the third

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

14

School Questionnaire
Response Experience in NTPS 2015-16

2

mailout) and this is not too far from the figure for non-priority schools with a survey coordinator
(71%, see Table 2-7). Even for schools that have very low response propensities historically, schools
with a survey coordinator require a lot less follow-up effort before they respond. Although it is not
possible to find whether this is because survey coordinators play a role in schools responding more
promptly or schools’ willingness and ability to find a survey coordinator shows their willingness to
participate, the survey coordinator status is a good indicator for the school’s cooperation again in
NTPS 2015-16.
As expected, non-priority schools with a survey coordinator were more likely to respond without
much follow-up effort than were priority schools or non-priority schools without a survey
coordinator. Regardless of priority status, schools had different follow-up experience depending on
whether the school had a survey coordinator.
Appendix C provides further detailed tables.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

15

Principal Questionnaire
Response Experience in NTPS 2015-16

3

Table 3-1 presents eligibility and response rates for NTPS 2015-16 principal questionnaires. The
statistics are the same as those in Table 2-3. The principal base full sample and replicate weights are
used for any statistics for weighted rates. The response rates for domains with a statistically
significant difference in response rate are in bold and italic. Rao-Scott Chi-Square tests are
conducted to detect any differences. Similar to the results for school questionnaires, response rates
for principal questionnaires differ by special district flag, school span, urbanicity, Census region,
poverty level, and school size. Response rates are lower for principals in special districts, city and
suburban schools, Northeastern schools, low poverty schools (percentage of students eligible for
free or reduced-price lunch less than 34%), and high poverty schools (percentage of students eligible
for free or reduced-price lunch equal to or greater than 75%). Response rates are higher for
combined schools, Midwestern schools, and those with an enrollment of 100-199 students in NTPS
2015-16.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

16

1,449
6,851

1,420
6,605

814
4,943

98.0%
96.4%

57.3%
74.8%

97.0%
95.8%

57.8%
75.3%

1.2%
0.6%

55.5%
74.2%

60.0%
76.4%

Charter
Non-charter

1,173
7,127

1,093
6,932

775
4,982

93.2%
97.3%

70.9%
71.9%

92.5%
96.3%

72.6%
72.3%

1.4%
0.5%

69.8%
71.2%

75.3%
73.4%

Primary
Middle
High
Combined

3,708
1,441
2,054
1,097

3,622
1,420
1,981
1,002

2,575
1,004
1,396
782

97.7%
98.5%
96.4%
91.3%

71.1%
70.7%
70.5%
78.0%

97.7%
98.3%
93.8%
87.5%

71.9%
72.2%
71.3%
78.2%

0.7%
1.2%
1.3%
1.4%

70.4%
69.8%
68.8%
75.5%

73.4%
74.6%
73.9%
81.0%

City
Suburban
Town
Rural

2,507
2,585
1,201
2,007

2,394
2,519
1,159
1,953

1,544
1,707
940
1,566

95.5%
97.4%
96.5%
97.3%

64.5%
67.8%
81.1%
80.2%

94.8%
96.6%
95.5%
96.9%

64.5%
68.6%
81.2%
80.4%

1.1%
0.9%
1.3%
0.9%

62.4%
66.7%
78.7%
78.6%

66.6%
70.4%
83.7%
82.2%

Northeast
Midwest
South
West

1,352
1,924
3,200
1,824

1,331
1,849
3,087
1,758

878
1,435
2,185
1,259

98.4%
96.1%
96.5%
96.4%

66.0%
77.6%
70.8%
71.6%

98.0%
94.8%
96.0%
96.0%

66.4%
77.7%
71.5%
72.0%

1.3%
1.0%
0.9%
1.2%

63.9%
75.7%
69.8%
69.7%

68.9%
79.6%
73.2%
74.2%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

2,416
1,395
2,331
2,158

2,321
1,357
2,270
2,077

1,624
1,024
1,699
1,410

96.1%
97.3%
97.4%
96.2%

70.0%
75.5%
74.8%
67.9%

95.8%
96.7%
95.8%
96.1%

69.8%
75.7%
75.6%
69.3%

1.1%
1.2%
1.0%
1.2%

67.8%
73.3%
73.7%
67.0%

71.9%
78.1%
77.6%
71.5%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

404
551
2,820
2,151
1,035
1,339

343
512
2,731
2,096
1,022
1,321

258
427
2,013
1,501
695
863

84.9%
92.9%
96.8%
97.4%
98.7%
98.7%

75.2%
83.4%
73.7%
71.6%
68.0%
65.3%

80.6%
94.7%
97.4%
97.8%
98.9%
99.0%

73.1%
83.8%
73.6%
71.1%
68.0%
65.5%

2.8%
1.8%
1.0%
1.0%
1.5%
1.4%

67.6%
80.3%
71.7%
69.1%
65.0%
62.8%

78.6%
87.4%
75.5%
73.2%
70.9%
68.2%

Regular
Special education
Vocational
Alternative

7,641
158
111
390

7,480
125
99
321

5,372
89
76
220

97.9%
79.1%
89.2%
82.3%

71.8%
71.2%
76.8%
68.5%

97.8%
74.8%
92.1%
78.8%

72.5%
71.0%
76.2%
68.2%

0.5%
5.6%
5.9%
3.3%

71.5%
60.0%
64.5%
61.7%

73.5%
82.0%
87.9%
74.7%

Domain
All

Sample
size
8,300

Number of
Eligible
schools
Completes
8,025
5,757

Unweighted
Eligibility
Response
rate
rate
96.7%
71.7%

Eligibility
rate
96.0%

Response
rate
72.3%

Weighted
Std.
error
0.5%

Lower
bound CI
71.3%

Upper
bound CI
73.3%

3

Special district
Not special district

17

NTPS 2015-16 principal eligibility and response rates, major domains

Principal Questionnaire
Response Experience in NTPS 2015-16

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 3-1.

Teacher Listing Form
Response Experience in NTPS 2015-16

4

Table 4-1 presents eligibility and response rates for NTPS 2015-16 teacher listing forms (TLFs). The
statistics are the same as those in Tables 2-3 and 3-1. Any school with a completed TLF was
considered a respondent regardless of the data collection method: completed by the school, through
a vendor list, or through clerical research. The school base full sample and replicate weights are used
for any statistics for weighted rates. The response rates for domains with a statistically significant
difference in response rate are in bold and italic. Rao-Scott Chi-Square tests are conducted to detect
any differences. Similar to the results for school and principal questionnaires, response rates for
TLFs differ by special district flag, urbanicity, Census region, poverty level, and school size. Unlike
those questionnaires, TLF response rates also differ by charter and type of school, and do not differ
by school span. Response rates are lower for TLFs in special districts, charter schools, city and
suburban schools, and low poverty schools (percentage of students eligible for free or reduced-price
lunch less than 34%). Response rates for TLFs from alternative schools were lower than for regular
schools. Response rates are higher for Midwestern schools and those with an enrollment of 100-199
students in NTPS 2015-16.
Note that the overall TLF response rate of 83.6% is statistically significantly higher than the
corresponding SASS 2011-12 response rate of 82.2%. This reflects the benefits of allowing the
alternative methods of collecting TLFs used in NTPS 2015-16, as mentioned above.
Among the schools that completed a TLF, we break out weighted percentages to the following four
categories (every completed TLF falls into one of these follow-up-level categories):


TLF sent in by school before phase 1 field follow-up or third school mailout (no or
limited follow-up);



TLF sent in by school before vendor file matching or clerical research (during phase 1
field or third school mailout);



TLF obtained through vendor file matching or clerical research (vendor/clerical followup) and no TLF from school; or



TLF sent in by school after vendor/clerical processes failed (follow-up after
vendor/clerical failed).

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

18

Teacher Listing Form
Response Experience in NTPS 2015-16

Table 4-1.

NTPS 2015-16 TLF eligibility and response rates, major domains
Number of

Domain

4

Sample

Eligible

size

schools

Completes

Unweighted

Weighted

Eligibility

Response

Eligibility

Response

Std.

Lower

Upper

bound

bound

rate

rate

rate

rate

error

CI

CI

All

8,300

8,025

6,659

96.7%

83.0%

96.0%

83.6%

0.4%

82.7%

84.4%

Special district
Not special district

1,449
6,851

1,421
6,604

998
5,661

98.1%
96.4%

70.2%
85.7%

97.2%
95.8%

69.9%
86.4%

1.1%
0.5%

67.7%
85.5%

72.1%
87.3%

Charter
Non-charter

1,173
7,127

1,093
6,932

832
5,827

93.2%
97.3%

76.1%
84.1%

92.4%
96.3%

77.3%
84.0%

1.4%
0.5%

74.6%
83.1%

80.0%
84.9%

Primary
Middle
High
Combined

3,708
1,441
2,054
1,097

3,624
1,420
1,979
1,002

3,001
1,179
1,644
835

97.7%
98.5%
96.3%
91.3%

82.8%
83.0%
83.1%
83.3%

97.7%
98.3%
93.6%
87.8%

83.7%
84.0%
82.9%
83.4%

0.6%
1.1%
1.2%
1.4%

82.5%
81.9%
80.6%
80.6%

85.0%
86.1%
85.2%
86.1%

City
Suburban
Town
Rural

2,507
2,585
1,201
2,007

2,393
2,517
1,161
1,954

1,895
2,003
1,038
1,723

95.5%
97.4%
96.7%
97.4%

79.2%
79.6%
89.4%
88.2%

94.9%
96.5%
95.7%
97.0%

79.4%
80.4%
88.9%
88.9%

0.9%
0.7%
1.0%
0.8%

77.7%
79.0%
86.9%
87.4%

81.1%
81.9%
90.8%
90.4%

Northeast
Midwest
South
West

1,352
1,924
3,200
1,824

1,332
1,850
3,085
1,758

1,064
1,601
2,555
1,439

98.5%
96.2%
96.4%
96.4%

79.9%
86.5%
82.8%
81.9%

98.1%
94.9%
95.9%
96.1%

80.0%
86.9%
83.3%
82.9%

1.1%
0.8%
0.7%
0.9%

77.8%
85.3%
82.0%
81.2%

82.2%
88.6%
84.7%
84.6%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

2,416
1,395
2,331
2,158

2,318
1,358
2,270
2,079

1,851
1,159
1,956
1,693

95.9%
97.3%
97.4%
96.3%

79.9%
85.3%
86.2%
81.4%

95.6%
96.8%
95.9%
96.3%

79.5%
85.8%
86.8%
83.1%

0.9%
1.1%
0.8%
0.9%

77.7%
83.6%
85.2%
81.4%

81.3%
87.9%
88.3%
84.9%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

404
551
2,820
2,151
1,035
1,339

342
512
2,731
2,097
1,022
1,321

278
446
2,305
1,769
813
1,048

84.7%
92.9%
96.8%
97.5%
98.7%
98.7%

81.3%
87.1%
84.4%
84.4%
79.5%
79.3%

80.5%
94.7%
97.4%
97.9%
98.9%
98.9%

79.0%
88.3%
85.0%
84.2%
79.9%
80.2%

2.8%
1.5%
0.8%
0.8%
1.3%
1.2%

73.6%
85.4%
83.5%
82.6%
77.3%
77.9%

84.5%
91.3%
86.5%
85.7%
82.5%
82.5%

Regular
Special education
Vocational
Alternative

7,641
158
111
390

7,480
125
98
322

6,246
98
79
236

97.9%
79.1%
88.3%
82.6%

83.5%
78.4%
80.6%
73.3%

97.8%
74.8%
91.3%
79.5%

84.3%
82.1%
78.2%
72.6%

0.4%
3.7%
5.9%
3.4%

83.5%
74.9%
66.6%
65.8%

85.1%
89.4%
89.7%
79.3%

While all the schools received the same sequence of follow-up, we present the weighted percentages
to the four categories for three separate groups of schools, non-priority schools with a survey
coordinator, non-priority schools without a survey coordinator, and priority schools, in Tables 4-2
through 4-4. The priority schools are then broken out by survey coordinator status in Tables 4-5 and
4-6. They show differences in follow-up effort required, comparable to the findings of Tables 2-7
through 2-9.
Among non-priority schools with a survey coordinator that completed a TLF (Table 4-2), more than
70% of schools completed a TLF before vendor or clerical research, almost all of them didn’t need
much follow-up effort, and only about 17% of TLFs were obtained through vendor or clerical
NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

19

Teacher Listing Form
Response Experience in NTPS 2015-16

4

research. Considerably less intensive follow-up effort was required for TLF completion for nonpriority schools with a survey coordinator, which is consistent with the results for other
questionnaires.
Table 4-2.

Response follow-up experience for final TLF respondents among non-priority schools
with a survey coordinator, NTPS 2015-16

Response follow-up category
No or limited follow-up
Phase 1 field or mailout
Through vendor or clerical research
After vendor/clerical process
Total respondents

Unweighted
respondent
count
2,260
170
572
318
3,320

Unweighted
respondent
percent
68.07%
5.12%
17.23%
9.58%
100.00%

Weighted
respondent
count
30,919
2,118
7,473
4,329
44,841

Weighted
respondent
percent
68.95%
4.72%
16.67%
9.66%
100.00%

Standard
error
0.89%
0.39%
0.72%
0.58%

For non-priority schools without a survey coordinator that completed a TLF (Table 4-3), two thirds
of schools completed before vendor or clerical research but more than half of those schools
required phase 1 or mailout follow-up effort. About one third of the TLFs were found through
vendor or clerical research. There were only a few TLFs completed after the vendor or clerical
research procedure.
Table 4-3.

Response follow-up experience for final TLF respondents among non-priority schools
without a survey coordinator, NTPS 2015-16

Response follow-up category
No or limited follow-up
Phase 1 field or mailout
Through vendor or clerical research
After vendor/clerical process
Total respondents

Unweighted
respondent
count
542
1106
811
16
2,475

Unweighted
respondent
percent
21.90%
44.69%
32.77%
0.65%
100.00%

Weighted
respondent
count
7,583
14,703
10,677
138
33,101

Weighted
respondent
percent
22.91%
44.42%
32.26%
0.42%
100.00%

Standard
error
0.94%
1.10%
1.03%
0.10%

Table 4-4 shows for priority schools that one half of schools that completed a TLF submitted it
before the vendor or clerical research procedure started and about 40% of TLFs were obtained
through vendor or clerical research.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

20

Teacher Listing Form
Response Experience in NTPS 2015-16

Table 4-4.

4

Response follow-up experience for final TLF respondents among priority schools,
NTPS 2015-16

Response follow-up category
No or limited follow-up
Phase 1 field or mailout
Through vendor or clerical research
After vendor/clerical process
Total respondents

Unweighted
respondent
count
297
146
347
74
864

Unweighted
respondent
percent
34.38%
16.90%
40.16%
8.56%
100.00%

Weighted
respondent
count
4,066
2,124
4,914
1,041
12,145

Weighted
respondent
percent
33.48%
17.49%
40.46%
8.57%
100.00%

Standard
error
1.58%
1.40%
1.65%
1.01%

Similar to Tables 2-10 and 2-11 for the school analyses, Tables 4-5 and 4-6 show the TLF follow-up
effort for priority schools by survey coordinator status. While responding priority schools with a
survey coordinator require more follow-up effort than non-priority schools with a survey
coordinator (40% versus 31% in terms of percent respondents that responded after phase 1 field or
mailout started), the differences were much smaller than the differences between priority schools
with and without a survey coordinator (40% verses 87%).
Table 4-5.

Response follow-up experience for final TLF respondents among priority schools
with a survey coordinator, NTPS 2015-16

Response follow-up category
No or limited follow-up
Phase 1 field or mailout
Through vendor or clerical research
After vendor/clerical process
Total respondents

Table 4-6.

Unweighted
respondent
count
229
32
84
30
375

Unweighted
respondent
percent
61.07%
8.53%
22.40%
8.00%
100.00%

Weighted
respondent
count
3,171
447
1,236
395
5,249

Weighted
respondent
percent
60.42%
8.52%
23.54%
7.52%
100.00%

Standard
error
2.66%
1.58%
2.44%
1.51%

Response follow-up experience for final TLF respondents among priority schools
without a survey coordinator, NTPS 2015-16

Response follow-up category
No or limited follow-up
Phase 1 field or mailout
Through vendor or clerical research
After vendor/clerical process
Total respondents

Unweighted
respondent
count
68
114
263
44
489

Unweighted
respondent
percent
13.91%
23.31%
53.78%
9.00%
100.00%

Weighted
respondent
count
895
1,677
3,678
647
6,897

Weighted
respondent
percent
12.98%
24.32%
53.33%
9.38%
100.00%

Standard
error
1.44%
2.12%
2.27%
1.42%

Figure 4-1 presents the number of TLF unweighted respondents and weighted percentage for
follow-up experience summarizing Tables 4-2, 4-3, 4-5, and 4-6.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

21

Teacher Listing Form
Response Experience in NTPS 2015-16

Figure 4-1.

4

Follow-up required prior to response with TLF final respondents by survey
coordinator status and by priority status, NTPS 2015-16

Another aspect of the NTPS 2015-16 design beyond TLF response rates was the desire to accelerate
the process of getting the teacher samples out to the field. This allows more time for collecting
teacher questionnaires and should improve teacher questionnaire response rates. Table 4-7 presents
a distribution of the number of days to field teacher questionnaire samples, using the initial teacher
mailing date7 as the date for the fielded teacher questionnaires. The weighted means and percentiles
are of the number of days from the initial school mail-out (October 11, 2011 for SASS 2011-12 and
September 4, 2015 for NTPS 2015-16), weighted by the number of teacher questionnaires that went
out that date (regardless of their ultimate status: completed questionnaire, nonrespondent, ineligible).
The initial school mail-out did not occur until October 11, 2011 for SASS 2011-12 due to delays in
receiving OMB clearance to conduct the data collection. The SASS initial school mail-out was 5
weeks after it was for NTPS 2015-16, so NTPS 2015-2016 would have more time for follow up with
late wave teachers even without any change. Consequently, the first initial teacher packages were also
sent out later for SASS 2011-12 than for NTPS 2015-16 (November 17, 2011 versus October 1,
2015). For an analysis independent of the delay in receiving OMB clearance, we used the initial
school mail-dates as starting points.

7

In some cases, this initial teacher mailing date was missing for some ineligible cases and nonresponding teachers in
SASS 2011-12 and for some LEA refusal or hard refusal cases in NTPS 2015-16. In these cases, if there was only one
initial teacher mailing date for the teacher wave, then that mailing date is used. If there are multiple teacher mailing
dates for the teacher wave (only happened in SASS 2011-12), then a mean value is computed (weighted by the number
of teacher questionnaires associated with the particular initial teacher mailing date).

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

22

Teacher Listing Form
Response Experience in NTPS 2015-16

Table 4-7.

4

Weighted distribution of number of days after initial school mail-out for initially
fielding teacher questionnaires for SASS 2011-12 and NTPS 2015-16.

Distribution of first mail day
Minimum
5th percentile
10th percentile
25th percentile
50th percentile
75th percentile
90th percentile
95th percentile
Maximum
Mean

SASS
days after Oct 11, 2011
37
51
66
66
102
120
185
198
218
108.2

NTPS
days after Sep 4, 2015
27
39
54
102
143
158
192
192
209
130.3

The mean value for SASS 2011-12 is about 20 days earlier: a mean value of 130.3 days (January 13,
2015 with an initial school mail-out date of September 4, 2015) vs. a mean value of 108.2 days
(January 28, 2012 with an initial school mail-out date October 11, 2011). While the teacher mail-out
process went faster in SASS 2011-12 for most of the data teacher mailout period up through the 90th
percentile of the teacher mailouts, the new procedures implemented for the NTPS allowed the last
five percent to get out about a week earlier. The overall process in SASS 2011-12 might have been
expedited without a month delay in data collection, which reduced the difference between fixed
dates across SASS 2011-12 and NTPS 2015-16 for the earlier period. The new TLF procedures have
increased the TLF response rate and reduced the number of days necessary to get the teacher
questionnaire samples into the field for teachers that were sampled very late in the data collection
period. However, this is not a clean test though because of the OMB problems in SASS 2011-12.
The last issue is the quality of the TLFs. In order to evaluate the quality of the teacher listing by the
source of the list, we conducted weighted regression analysis. The dependent variables are the
estimated number of eligible teachers on the teacher list and the estimated eligibility rate; the
explanatory variables are the school domain variables, the full-time equivalent teacher (FTE) count,
and TLF source variables; the full-sample and replicate weights adjusted for the TLF nonresponse
are used. Since the teacher eligibility status is only known for sampled teachers, the eligibility rate for
each school was estimated from the teacher file assuming that the eligibility rate for the teachers that
were not sampled is similar to the eligibility rate for the sampled teachers. The eligibility rate was
unweighted because the teacher sample is a simple random sample of the teachers from the listing
form within a school. The number of eligible teachers on the teacher listing form is estimated by the
product of the number of teachers on the form and the estimated eligibility rate from the teacher
file. For the explanatory variables, a set of school domain variables were selected through the

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

23

Teacher Listing Form
Response Experience in NTPS 2015-16

4

stepwise method (provided by SAS PROC GLMSELECT) for each regression model, and the
selected variables and the TLF source variable were included in the final model.
There are two sets of analyses presented in Tables 4-8 and 4-9. The analyses on the left-hand side
combine clerical search and vendor canvassing into one indicator. The analyses on the right-hand
side separate clerical search and vendor canvassing as two separate indicators.
In terms of the number of eligible teachers, the quality of the respondent-filled TLFs was not better
than the TLFs listed clerically or through vendors. The respondent-filled TLFs had about 0.7 more
eligible teachers than the TLFs listed through vendors on average. Although the difference is
statistically significant in the regression model, the difference does not appear large enough to make
a practical difference. Comparing the TLFs listed clerically to the TLFs listed through vendors, the
difference itself is about 9 teachers, which is considerably large, but the difference is not statistically
significant given that there were only 83 TLFs listed clerically. It is possible that further use of
clerical methods would increase the number of eligible teachers, but a larger number of TLFs would
have to be developed clerically to test this. Thus, it is difficult to conclude that any of these data
collection method was superior to the others with respect to the number of eligible teachers on the
list.
The regression analysis for eligibility rates shows that the quality of the TLFs listed through vendors
is poorer than the TLFs from other sources. Vendor-supplied TLFs had 6% lower eligibility rates
than those filled out by the school. Clerically-supplied TLFs were half-way in between vendor and
school-supplied.
To summarize, the new TLF data collection methodology in NTPS 2015-16 increased marginally the
TLF response rate (82.2% to 83.6%), reduced the number of days necessary to get the teacher
questionnaire samples into the field by a week for teachers that were sampled very late in the data
collection period with a slight drop in list quality (slightly higher ineligibility rates) for the 26% (1,730
out of 6,659 completed TLFs) not supplied by the schools.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

24

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 4-8.

Regression coefficients for number of eligible teachers

Parameter
Intercept
FTE
Respondent-filled
Clerical or Vendor

Coefficient
8.35
0.8
0.34
0

Std
error
3.76
0.06
0.45
0

25

2.43
2.24
0.41
0

0.81
0.35
0.32
0

Northeast
Midwest
South
West

1.51
-0.1
-0.66
0

0.86
0.66
0.8
0

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

2.64
1.23
0.93
0

0.51
0.47
0.6
0

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

-7.24
-8.73
-8.65
-7.1
-5.74
0

4.2
3.76
3.27
2.58
2.26
0

Regular
Special education
Vocational
Alternative

3.61
-5.54
1.43
0

1.27
3.1
3.99
0

.
3.02
6.45
1.28
.
1.75
-0.15
-0.82
.
5.2
2.64
1.55
.
-1.72
-2.32
-2.64
-2.76
-2.54
.
2.84
-1.78
0.36
.

Coefficient
7.86
0.8
0.69
9.01
0

Std
error
3.72
0.06
0.3
7.39
0

City
Suburb
Town
Rural

2.32
2.22
0.44
0

0.69
0.34
0.32
0

Northeast
Midwest
South
West

1.43
-0.14
-0.68
0

0.89
0.66
0.79
0

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

2.64
1.23
0.92
0

0.5
0.45
0.55
0

0.086
0.021
0.009
0.006
0.012
.

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

-7.49
-8.84
-8.63
-7.07
-5.73
0

4.09
3.77
3.28
2.59
2.27
0

0.005
0.076
0.721
.

Regular
Special education
Vocational
Alternative

3.79
-5.23
1.34
0

1.14
2.86
4.16
0

Pr > |t|
0.028
<.0001
0.451
.
0.003
<.0001
0.203
.
0.081
0.877
0.411
.
<.0001
0.009
0.123
.

Parameter
Intercept
FTE
Respondent-filled
Clerical
Vendor

t Value
2.11
14.14
2.31
1.22

Pr > |t|
0.036
<.0001
0.022
0.224
.

3.36
6.59
1.37

0.001
<.0001
0.172
.

1.62
-0.21
-0.86

0.107
0.836
0.391
.

5.27
2.74
1.68

<.0001
0.007
0.095
.

-1.83
-2.34
-2.63
-2.73
-2.53

0.069
0.02
0.009
0.007
0.012
.

3.32
-1.83
0.32

0.001
0.068
0.747
.

.

.

.

.

.

.

Teacher Listing Form
Response Experience in NTPS 2015-16

City
Suburb
Town
Rural

t Value
2.22
14.14
0.75

4

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 4-9.

Regression coefficients for teacher eligibility rate

Parameter
Intercept
Respondent-filled
Clerical or Vendor
FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

Coefficient
0.9
0.05
0
0.02
0.01
0.01
0

Std
error
0.01
0.01
0
0.01
0
0
0

t Value
156.32
9.53
.
2.95
3.3
2.03
.

Pr > |t|
<.0001
<.0001
.
0.004
0.001
0.043
.

Parameter
Intercept
Respondent-filled
Clerical
Vendor
FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

Coefficient
0.9
0.06
0.03
0

Std
error
0.01
0.01
0.02
0

0.02
0.01
0.01
0

0.01
0
0
0

t Value
152.38
9.49
1.64

Pr > |t|
<.0001
<.0001
0.103
.

2.97
3.32
2.06

0.003
0.001
0.041
.

.

.

26
Teacher Listing Form
Response Experience in NTPS 2015-16

4

Teacher Questionnaire
Response Rate in NTPS 2015-16

5

Tables 5-1 and 5-2 below present unweighted and weighted teacher counts from the SASS 2011-12
and the NTPS 2015-16 teacher samples, respectively. The NTPS teacher counts by the source of
TLFs are also shown in Table 5-2. The weighted counts utilize the teacher base weight (the sum of
the weighted counts are an estimate of the total number of teachers). The unweighted and weighted
percentages are quite similar for both SASS and NTPS.
Table 5-1.

SASS 2011-12 public school teacher response and eligibility rates8

2011-12
SASS Teacher Outcomes
Completes
Nonrespondents
Ineligible
Total sample

Table 5-2.

Teacher
Sample Size
37,497
11,332
2,233
51,062

Unweighted
Percent
73.43%
22.19%
4.37%
100.00%

Weighted Teacher
Count
1,837,847
528,644
104,312
2,470,803

Weighted Percent
74.38%
21.40%
4.22%
100.00%

NTPS 2015-16 public school teacher response and eligibility rates9

2015-16
Teacher sample
NTPS Teacher outcomes
size
All teachers
Completes
31,945
Nonrespondents
14,784
Ineligible
2,258
Total sample
48,987
From respondent-filled TLFs
Completes
26,859
Nonrespondents
7,718
Ineligible
1,194
Total sample
35,771
TLFs from vendors or through clerical research
Completes
5,086
Nonrespondents
7,066
Ineligible
1,064
Total sample
13,216

Unweighted
percent

Weighted
teacher count

Weighted
percent

65.21%
30.18%
4.61%
100.00%

1,795,587
840,840
127,322
2,763,749

64.97%
30.42%
4.61%
100.00%

75.09%
21.58%
3.34%
100.00%

1,493,366
421,106
65,828
1,980,300

75.41%
21.26%
3.32%
100.00%

38.48%
53.47%
8.05%
100.00%

302,222
419,733
61,495
783,449

38.58%
53.58%
7.85%
100.00%

8

This table is the same as Table F-1A in “NTPS 2017-2018 Public School Sample Design Including State-Level
Estimates,” March 3, 2017.

9

This table is the same as Table F-1B in “NTPS 2017-2018 Public School Sample Design Including State-Level
Estimates,” March 3, 2017.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

27

Teacher Questionnaire
Response Experience in NTPS 2015-16

5

As can be seen, the conditional response rate for teachers is considerably lower for NTPS 2015-16
than for SASS 2011-12 (68.1% versus 77.7%). However, the response rates by the source of TLFs
are very different. The conditional response rate for teachers sampled from TLFs that were filled out
by their schools in NTPS 2015-16 (78.0%) is comparable to the response rate for SASS 2011-12
(77.7%). On the other hand, the conditional response rate for teachers sampled from TLFs through
clerical research or vendor file matching is significantly lower (41.9%). We suspect that this is an
artifact of these schools being much less cooperative, leading to lower response rates among their
teachers. There is no sensible direct causal link between the TLF being collected in a certain way and
the teacher’s response propensity for the questionnaire. It is also worth noting that teachers listed on
TLFs obtained through alternative sources had more than double the ineligibility rate compared to
teachers listed on TLFs that were filled out by their schools. There are certainly good reasons for
these alternative TLFs having a higher prevalence of ineligible teachers.
Table 5-3 shows the impact of including the alternative TLF collection methods on the teacher
response rates. These alternative collection methods increased the sample yield, the TLF response
rate, and the overall teacher response rate (despite teachers from the alternative source schools being
less cooperative), provided information on teachers from schools that did not send TLFs, and gave a
chance to participate in the survey to some teachers who would have not been in the teacher frame
through the traditional listing method.
Table 5-3.

NTPS 2015-16 teacher weighted response rates, with and without alternative TLFs
included

Including TLFs from vendors or through clerical research
Excluding TLFs from vendors or through clerical research

TLF
response
rate
83.56%
62.25%

Teacher
response
rate
conditional
on TLF
completion
68.11%
78.00%

Overall
teacher
response
rate
56.91%
48.56%

Table 5-4 presents eligibility and response rates for NTPS 2015-16 teacher questionnaires by major
school domains. The rows and columns are the same as those in Table 2-3. The teacher base full
sample and replicate weights are used for any statistics for weighted rates. The response rates for
teacher domains with a statistically significant difference in response rate are in bold and italic. RaoScott Chi-Square tests are conducted to detect any differences. The response rates for teacher
questionnaires vary by special district flag, charter school flag, urbanicity, Census region, poverty
level, and school size. Response rates in NTPS 2015-16 are lower for teachers from schools in

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

28

Teacher Questionnaire
Response Experience in NTPS 2015-16

5

special districts, charter schools, city and suburban schools, and high poverty schools (percentage of
students eligible for free or reduced-price lunch 75% or more). Teacher response rates are higher for
Midwestern schools and schools with 100-199 students.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

29

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 5-4.

NTPS 2015-16 teacher eligibility and response rates, major school domains

Domain
All

Sample
size
48,987

Number of
Eligible
teachers
Completes
46,458
32,753

Unweighted
Eligibility
Response
rate
rate
94.8%
70.5%

Eligibility
rate
94.8%

Response
rate
70.3%

Weighted
Std.
error
0.4%

Lower
bound CI
69.5%

Upper
bound CI
71.1%

30

7,849
38,609

4,863
27,890

95.1%
94.8%

62.0%
72.2%

95.1%
94.7%

62.3%
72.0%

1.1%
0.4%

60.2%
71.1%

64.4%
72.8%

Charter
Non-charter

5,313
43,674

5,034
41,424

3,438
29,315

94.7%
94.8%

68.3%
70.8%

94.7%
94.8%

68.1%
70.4%

1.2%
0.4%

65.8%
69.6%

70.4%
71.3%

Primary
Middle
High
Combined

19,149
9,480
15,121
5,237

18,060
9,035
14,395
4,968

13,041
6,332
9,686
3,694

94.3%
95.3%
95.2%
94.9%

72.2%
70.1%
67.3%
74.4%

94.4%
95.3%
95.1%
94.8%

72.3%
69.9%
66.4%
74.5%

0.6%
0.9%
0.9%
1.0%

71.2%
68.2%
64.6%
72.4%

73.5%
71.7%
68.1%
76.6%

City
Suburban
Town
Rural

14,753
16,520
6,886
10,828

13,930
15,725
6,501
10,302

8,964
10,804
5,071
7,914

94.4%
95.2%
94.4%
95.1%

64.4%
68.7%
78.0%
76.8%

94.2%
95.2%
94.2%
95.1%

64.0%
68.7%
77.8%
77.0%

0.8%
0.7%
0.9%
0.7%

62.5%
67.2%
76.0%
75.6%

65.6%
70.2%
79.6%
78.4%

Northeast
Midwest
South
West

8,762
11,021
19,251
9,953

8,366
10,444
18,130
9,518

5,522
8,146
12,702
6,383

95.5%
94.8%
94.2%
95.6%

66.0%
78.0%
70.1%
67.1%

95.3%
94.7%
94.0%
95.8%

65.6%
77.6%
70.3%
66.6%

0.9%
0.7%
0.7%
0.9%

63.8%
76.1%
69.0%
64.8%

67.5%
79.0%
71.6%
68.3%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

14,735
8,536
13,628
12,088

14,131
8,097
12,875
11,355

10,071
5,852
9,368
7,462

95.9%
94.9%
94.5%
93.9%

71.3%
72.3%
72.8%
65.7%

95.9%
94.7%
94.5%
93.7%

70.9%
71.8%
72.2%
66.1%

0.8%
1.0%
0.9%
0.8%

69.3%
69.9%
70.5%
64.5%

72.5%
73.7%
74.0%
67.7%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

1,089
2,229
13,927
12,850
6,990
11,902

1,035
2,109
13,174
12,164
6,638
11,338

750
1,580
9,604
8,786
4,609
7,424

95.0%
94.6%
94.6%
94.7%
95.0%
95.3%

72.5%
74.9%
72.9%
72.2%
69.4%
65.5%

95.7%
93.9%
94.5%
94.7%
94.9%
95.1%

69.3%
77.7%
73.1%
72.0%
68.9%
64.8%

3.3%
1.5%
0.7%
0.7%
1.0%
1.0%

62.9%
74.7%
71.8%
70.5%
66.9%
62.9%

75.8%
80.7%
74.4%
73.5%
70.9%
66.8%

Regular
Special education
Vocational
Alternative

46,803
539
496
1,149

44,405
498
465
1,090

31,331
374
341
707

94.9%
92.4%
93.8%
94.9%

70.6%
75.1%
73.3%
64.9%

94.8%
92.5%
93.6%
95.2%

70.4%
73.3%
71.8%
64.2%

0.4%
4.7%
2.7%
3.4%

69.6%
64.0%
66.4%
57.5%

71.2%
82.6%
77.2%
70.9%

5

8,254
40,733

Teacher Questionnaire
Response Experience in NTPS 2015-16

Special district
Not special district

Teacher Questionnaire
Response Rate in NTPS 2015-16

5

Teachers had a different amount of time to respond depending on when their schools sent their
TLFs because a teacher sample of a school can be drawn only after a TLF from the school was
received. In SASS 2011-12, the data collection was not very successful for teachers that belong to
schools that sent the TLF later in the field period. In NTPS 2015-16, there were 24 teacher waves.
The waves were based on the time that the TLF was received and processed and a teacher sample
was drawn. The initial mail packages were sent out on a set of days for teachers in one to three
teacher waves. The teacher waves were grouped into three teacher wave groups (early/middle/late).
Table 5-5 shows how the 24 teacher waves were grouped, the initial mailout date of each teacher
wave, and the number of sampled teachers in each teacher wave.
Table 5-5.

NTPS 2015-16 public school teacher wave and initial mailout date

Teacher wave group
Early

Middle

Late

Teacher wave
1
3
4
5
6
7
8
9
10
Total
11
12
13
14
15
Total
16
17
18
19
20
21
22
23
24
Total

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Initial mailout date
10/01/2015
10/13/2015
10/19/2015
10/28/2015
11/02/2015
11/09/2015
11/16/2015
11/23/2015
11/23/2015
12/15/2015
12/15/2015
1/04/2016
1/25/2016
1/25/2016
2/09/2016
2/09/2016
2/09/2016
2/22/2016
3/14/2016
3/14/2016
3/31/2016
3/31/2016
3/31/2016

31

Number of
sampled teachers
1,296
811
1,046
1,850
1,766
1,578
834
1,233
739
11,153
4,141
2,781
1,669
3,302
2,129
14,022
1,249
1,275
10,432
3,755
15
6,887
55
91
53
23,812

Teacher Questionnaire
Response Rate in NTPS 2015-16

5

Figure 5-1 presents the weighted response rates for the three teacher wave groups (TWGs). The
later the TLF was received, the lower the response rate was. The response rate drops dramatically
from the middle teacher wave group to the late teacher wave group, which is consistent with the
results from SASS 2011-12.
Figure 5-1.

NTPS 2015-16 public school teacher response rate by teacher wave group

The eligibility and response rates by teacher wave group are presented in Tables 5-5 through 5-7.
The teachers in the early teacher wave group had sufficient time to respond since their schools sent
their teacher listing forms early, which resulted in higher response rates than for the later teacher
wave groups. The response rate for this group is over 90%. Within the early wave group, teachers
from schools that are not in special districts, teachers from town or rural schools, or schools in the
Midwest region have higher response rates (see Table 5-6).
The response rate for the middle teacher wave group is around 80%. While teachers in the middle
wave group had less time to respond, they had a decent response rate. Within this wave group,
teachers from schools that are not in special districts, non-charter schools, or are in town or rural
schools have higher response rates (see Table 5-7). Regular schools have higher response rates than
alternative or vocational schools. Western schools and those with less than 100 students or more
than 1,000 students are less likely to respond (see Table 5-8).

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

32

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 5-6.

NTPS 2015-16 teacher eligibility and response rates for early teacher wave group, major school domains

Domain
All

Sample
size
11,153

Number of
Eligible
Completes
10,823
9,790
teachers

Unweighted
Eligibility
Response
97.0%
90.5%
rate
rate

Eligibility
97.0%
rate

Response
90.6%
rate

Weighted
Std.
0.5%
error

Lower
89.5%CI
bound

Upper
91.6%CI
bound

33

1,604
9,549

1,574
9,249

1,270
8,520

98.1%
96.9%

80.7%
92.1%

98.0%
96.9%

80.9%
92.3%

2.0%
0.5%

76.9%
91.3%

84.8%
93.2%

Charter
Non-charter

1,164
9,989

1,124
9,699

1,006
8,784

96.6%
97.1%

89.5%
90.6%

96.6%
97.0%

89.8%
90.6%

1.7%
0.5%

86.6%
89.5%

93.1%
91.7%

Primary
Middle
High
Combined

4,332
2,261
3,291
1,269

4,177
2,212
3,219
1,215

3,821
2,012
2,854
1,103

96.4%
97.8%
97.8%
95.7%

91.5%
91.0%
88.7%
90.8%

96.5%
97.9%
97.8%
95.5%

91.5%
91.2%
88.5%
90.7%

0.8%
1.2%
1.2%
1.5%

90.0%
88.8%
86.1%
87.6%

93.0%
93.6%
90.8%
93.7%

City
Suburban
Town
Rural

2,403
3,833
1,910
3,007

2,325
3,722
1,861
2,915

2,027
3,306
1,758
2,699

96.8%
97.1%
97.4%
96.9%

87.2%
88.8%
94.5%
92.6%

96.8%
97.0%
97.4%
97.0%

86.9%
89.2%
94.4%
92.9%

1.5%
1.1%
0.8%
0.8%

84.0%
87.0%
92.9%
91.4%

89.9%
91.4%
95.9%
94.4%

Northeast
Midwest
South
West

1,746
3,068
4,298
2,041

1,700
2,978
4,147
1,998

1,499
2,794
3,770
1,727

97.4%
97.1%
96.5%
97.9%

88.2%
93.8%
90.9%
86.4%

97.3%
97.0%
96.4%
98.2%

88.3%
94.3%
90.9%
85.8%

1.5%
0.7%
0.8%
1.5%

85.3%
92.8%
89.3%
82.8%

91.4%
95.7%
92.5%
88.8%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

3,895
2,118
3,235
1,905

3,783
2,069
3,132
1,839

3,431
1,854
2,886
1,619

97.1%
97.7%
96.8%
96.5%

90.7%
89.6%
92.1%
88.0%

97.0%
97.7%
96.8%
96.8%

91.0%
90.0%
92.3%
87.4%

0.9%
1.3%
1.0%
1.5%

89.2%
87.4%
90.4%
84.4%

92.8%
92.6%
94.2%
90.4%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

227
539
3,337
2,928
1,444
2,678

221
523
3,214
2,841
1,407
2,617

204
477
2,965
2,594
1,264
2,286

97.4%
97.0%
96.3%
97.0%
97.4%
97.7%

92.3%
91.2%
92.3%
91.3%
89.8%
87.4%

96.9%
97.4%
96.4%
97.0%
97.4%
97.7%

89.7%
91.8%
92.7%
91.5%
89.2%
87.3%

3.9%
2.1%
0.7%
1.0%
1.7%
1.4%

82.1%
87.6%
91.3%
89.5%
85.9%
84.6%

97.3%
96.1%
94.1%
93.6%
92.5%
90.0%

10,730
121
107
195

10,421
116
101
185

9,428
108
92
162

97.1%
95.9%
94.4%
94.9%

90.5%
93.1%
91.1%
87.6%

97.1%
94.9%
93.8%
95.2%

90.6%
90.1%
92.2%
88.6%

0.5%
4.7%
6.6%
2.9%

89.5%
80.9%
79.2%
83.0%

91.6%
99.3%
100.0%
94.3%

Regular
Special education
Vocational
Alternative

Teacher Questionnaire
Response Experience in NTPS 2015-16

Special district
Not special district

5

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 5-7.

NTPS 2015-16 teacher eligibility and response rates for middle teacher wave group, major school domains

Domain
All

Sample
size
14,022

Number of
Eligible
teachers
Completes
13,505
11,011

Unweighted
Eligibility
Response
rate
rate
96.3%
81.5%

Eligibility
rate
96.3%

Response
rate
82.0%

Weighted
Std.
error
0.6%

Lower
bound CI
80.9%

Upper
bound CI
83.2%

34

1,820
11,685

1,304
9,707

96.4%
96.3%

71.6%
83.1%

96.6%
96.3%

73.5%
83.3%

2.1%
0.6%

69.4%
82.1%

77.7%
84.6%

Charter
Non-charter

1,547
12,475

1,478
12,027

1,128
9,883

95.5%
96.4%

76.3%
82.2%

95.6%
96.3%

76.9%
82.3%

2.0%
0.6%

73.1%
81.1%

80.8%
83.5%

Primary
Middle
High
Combined

5,779
2,756
3,884
1,603

5,533
2,670
3,759
1,543

4,618
2,132
2,973
1,288

95.7%
96.9%
96.8%
96.3%

83.5%
79.9%
79.1%
83.5%

95.8%
97.0%
96.9%
96.0%

84.0%
79.9%
79.2%
83.9%

0.8%
1.5%
1.2%
1.4%

82.5%
77.1%
76.9%
81.2%

85.6%
82.8%
81.5%
86.6%

City
Suburban
Town
Rural

3,750
4,801
2,117
3,354

3,591
4,627
2,040
3,247

2,772
3,690
1,777
2,772

95.8%
96.4%
96.4%
96.8%

77.2%
79.7%
87.1%
85.4%

95.9%
96.5%
96.2%
96.6%

77.7%
80.5%
87.5%
86.1%

1.1%
1.1%
1.1%
1.0%

75.5%
78.4%
85.4%
84.1%

80.0%
82.6%
89.6%
88.1%

Northeast
Midwest
South
West

2,478
3,386
5,202
2,956

2,376
3,262
5,008
2,859

1,867
2,782
4,185
2,177

95.9%
96.3%
96.3%
96.7%

78.6%
85.3%
83.6%
76.1%

96.1%
96.4%
96.2%
96.6%

79.2%
85.3%
84.6%
76.3%

1.5%
1.0%
0.9%
1.3%

76.2%
83.4%
82.9%
73.8%

82.3%
87.2%
86.4%
78.9%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

4,284
2,465
4,016
3,257

4,152
2,384
3,855
3,114

3,343
1,971
3,214
2,483

96.9%
96.7%
96.0%
95.6%

80.5%
82.7%
83.4%
79.7%

97.0%
96.6%
95.9%
95.7%

80.9%
83.2%
83.8%
80.6%

1.2%
1.3%
1.0%
1.2%

78.6%
80.7%
81.9%
78.2%

83.2%
85.6%
85.7%
83.0%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

364
726
4,202
3,893
2,062
2,775

335
695
4,028
3,755
1,999
2,693

290
568
3,302
3,152
1,621
2,078

92.0%
95.7%
95.9%
96.5%
96.9%
97.0%

86.6%
81.7%
82.0%
83.9%
81.1%
77.2%

93.0%
96.0%
95.7%
96.5%
97.0%
96.9%

82.6%
85.3%
82.8%
84.1%
81.0%
77.6%

3.7%
2.0%
0.9%
1.0%
1.7%
1.6%

75.2%
81.3%
81.0%
82.1%
77.7%
74.5%

89.9%
89.2%
84.6%
86.1%
84.2%
80.7%

13,358
182
178
304

12,883
166
170
286

10,514
149
130
218

96.4%
91.2%
95.5%
94.1%

81.6%
89.8%
76.5%
76.2%

96.4%
90.0%
95.3%
93.8%

82.2%
91.1%
72.7%
78.6%

0.6%
2.8%
2.8%
3.7%

81.0%
85.6%
67.2%
71.2%

83.4%
96.5%
78.2%
85.9%

Regular
Special education
Vocational
Alternative

5

1,887
12,135

Teacher Questionnaire
Response Experience in NTPS 2015-16

Special district
Not special district

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 5-8.

NTPS 2015-16 teacher eligibility and response rates for late teacher wave group, major school domains

Domain
All

Sample
size
23,812

Number of
Eligible
teachers
Completes
22,130
11,952

Unweighted
Eligibility
Response
rate
rate
92.9%
54.0%

Eligibility
rate
92.9%

Response
rate
53.8%

Weighted
Std.
error
0.6%

Lower
bound CI
52.7%

Upper
bound CI
55.0%

35

4,455
17,675

2,289
9,663

93.5%
92.8%

51.4%
54.7%

93.6%
92.7%

51.8%
54.4%

1.3%
0.6%

49.3%
53.1%

54.4%
55.6%

Charter
Non-charter

2,602
21,210

2,432
19,698

1,304
10,648

93.5%
92.9%

53.6%
54.1%

93.4%
92.8%

53.0%
53.9%

1.7%
0.6%

49.8%
52.7%

56.3%
55.1%

Primary
Middle
High
Combined

9,038
4,463
7,946
2,365

8,350
4,153
7,417
2,210

4,602
2,188
3,859
1,303

92.4%
93.1%
93.3%
93.4%

55.1%
52.7%
52.0%
59.0%

92.5%
93.0%
93.3%
93.6%

55.3%
52.8%
51.2%
59.6%

0.9%
1.3%
1.0%
1.7%

53.6%
50.2%
49.2%
56.3%

57.1%
55.5%
53.3%
62.8%

City
Suburban
Town
Rural

8,600
7,886
2,859
4,467

8,014
7,376
2,600
4,140

4,165
3,808
1,536
2,443

93.2%
93.5%
90.9%
92.7%

52.0%
51.6%
59.1%
59.0%

92.9%
93.6%
90.7%
92.6%

51.9%
51.7%
59.3%
58.7%

0.9%
1.0%
1.5%
1.3%

50.1%
49.8%
56.4%
56.2%

53.8%
53.6%
62.2%
61.3%

Northeast
Midwest
South
West

4,538
4,567
9,751
4,956

4,290
4,204
8,975
4,661

2,156
2,570
4,747
2,479

94.5%
92.1%
92.0%
94.0%

50.3%
61.1%
52.9%
53.2%

94.2%
92.0%
91.8%
94.5%

49.9%
60.2%
53.2%
53.1%

1.4%
1.3%
0.9%
1.1%

47.2%
57.5%
51.5%
50.9%

52.6%
62.8%
54.9%
55.3%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

6,556
3,953
6,377
6,926

6,196
3,644
5,888
6,402

3,297
2,027
3,268
3,360

94.5%
92.2%
92.3%
92.4%

53.2%
55.6%
55.5%
52.5%

94.5%
92.1%
92.6%
92.0%

52.6%
55.2%
54.9%
53.3%

1.1%
1.3%
1.1%
1.1%

50.4%
52.7%
52.7%
51.1%

54.8%
57.8%
57.1%
55.4%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

498
964
6,388
6,029
3,484
6,449

479
891
5,932
5,568
3,232
6,028

256
535
3,337
3,040
1,724
3,060

96.2%
92.4%
92.9%
92.4%
92.8%
93.5%

53.4%
60.0%
56.3%
54.6%
53.3%
50.8%

97.3%
90.3%
92.8%
92.6%
92.6%
93.4%

52.9%
62.7%
56.0%
54.7%
53.1%
50.3%

3.5%
2.4%
1.0%
1.1%
1.4%
1.2%

46.1%
58.1%
54.0%
52.5%
50.3%
48.0%

59.7%
67.3%
58.0%
56.9%
56.0%
52.6%

22,715
236
211
650

21,101
216
194
619

11,389
117
119
327

92.9%
91.5%
91.9%
95.2%

54.0%
54.2%
61.3%
52.8%

92.8%
93.0%
91.4%
95.7%

53.8%
57.2%
60.6%
53.5%

0.6%
8.1%
5.6%
3.0%

52.6%
41.2%
49.5%
47.7%

54.9%
73.1%
71.6%
59.4%

Regular
Special education
Vocational
Alternative

5

4,763
19,049

Teacher Questionnaire
Response Experience in NTPS 2015-16

Special district
Not special district

Teacher Questionnaire
Response Rate in NTPS 2015-16

5

Teachers in the late wave group have a response rate that is significantly lower than the earlier wave
groups, only 53.4%. It might be because these teachers did have less time to respond than the earlier
groups, but it also might be that teachers from schools that are slower to respond are less likely to
respond. City and suburban school teachers or teachers from larger schools (enrollment 200+) are
less likely to respond. Teachers in combined schools and in the Midwest are more likely to respond.
In contrast to the school domains used in these tables, Tables D-1 through D-4 in Appendix D
provide similar tables for teacher domains. While there are no differences between response rates
for full-time and part-time teachers, those whose teaching status is missing (mostly coming from the
alternative TLF sources) had much lower response rates in the middle and late waves.
Among teachers who were assigned to telephone nonresponse follow-up, an experiment was
conducted to test whether special afternoon calls would help to boost the response rate. About half
of the teachers who were assigned to telephone nonresponse follow-up received special afternoon
calls (experimental group 1) and the rest of them received nonresponse follow-up calls according to
the regular call schedule (experimental group 2). Experimental group 1 received calls between 2:00
and 5:00 pm in the afternoon, and experimental group 2 throughout the school day. Table 5-9
compares the weighted conditional teacher response rates of the two experimental groups. Overall,
the teacher response rates of the two groups are not different, which means that special afternoon
calls were not particularly helpful in terms of boosting the response rate. Tables D-5 and D-6 in the
appendix provide a detailed breakdown by school domain and teacher domain, respectively. The
experiment was most effective with teachers from schools in towns, primary school teachers whose
major subject is other, and high school teachers whose main subject is vocational/technical.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

36

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 5-9.

Domain
All

NTPS 2015-16 teacher eligibility and response rates by nonresponse follow-up experimental group, major school domains
Teacher experimental group 1 special afternoon calls
Number of
Weighted
Sample
Eligible
Response
Std
size
teachers
Completes
rate
error
9,560
9,215
5,124
55.5%
0.7%

Teacher experimental group 2 regular call schedule
Number of
Weighted
Sample
Eligible
Response
Std
size
teachers
Completes
rate
error
9,439
9,104
5,004
54.7%
0.7%

37

Teacher Questionnaire
Response Rate in NTPS 2015-16

5

Teacher Questionnaire
Response Experience in NTPS 2015-16

6

In this section, we study the response follow-up experience of teachers. Among the teacher
respondents, Tables 6-1 through 6-3 present the distribution of the following response experience
categories:


Teacher interview completed before telephone follow-up (no or limited follow-up);



Teacher interview completed before phase 2 field follow-up (telephone and mailout
follow-up only);



Teacher interview completed after phase 2 field follow-up (follow-up including field
follow-up).

Appendix B provides the definition of these three categories and details regarding the assumptions
that led to the definition.
Table 6-1.

Response follow-up experience with teacher interview final respondents

Response follow-up category
No or limited follow-up
Telephone/mail follow-up, no field
Field follow-up
Total respondents

Unweighted
respondent
count
19,412
6,728
6,613
32,753

Unweighted
respondent
percent
59.27%
20.54%
20.19%
100.00%

Weighted
respondent
count
2,285,853
755,874
784,709
3,826,436

Weighted
respondent
percent
59.74%
19.75%
20.51%
100.00%

Standard
error
0.54%
0.35%
0.49%

Tables 6-2 through 6-4 present the distribution of three response follow-up categories by teacher
wave group.
Table 6-2.

Response follow-up experience with teacher interview final respondents in early
teacher wave group

Response follow-up category
No or limited follow-up
Telephone/mail follow-up, no field
Field follow-up
Total respondents

Unweighted
respondent
count
6,282
3,073
435
9,790

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Unweighted
respondent
percent
64.17%
31.39%
4.44%
100.00%

38

Weighted
respondent
count
707,328
334,260
45,244
1,086,832

Weighted
respondent
percent
65.08%
30.76%
4.16%
100.00%

Standard
error
0.80%
0.71%
0.35%

Teacher Questionnaire
Response Experience in NTPS 2015-16

Table 6-3.

Response follow-up experience with teacher interview final respondents in middle
teacher wave group

Response follow-up category
No or limited follow-up
Telephone/mail follow-up, no field
Field follow-up
Total respondents

Table 6-4.

6

Unweighted
respondent
count
7,141
1,869
2,001
11,011

Unweighted
respondent
percent
64.85%
16.97%
18.17%
100.00%

Weighted
respondent
count
839,089
217,819
227,233
1,284,141

Weighted
respondent
percent
65.34%
16.96%
17.70%
100.00%

Standard
error
0.89%
0.63%
0.67%

Response follow-up experience with teacher interview final respondents in late
teacher wave group

Response follow-up category
No or limited follow-up
Telephone/mail follow-up, no field
Field follow-up
Total respondents

Unweighted
respondent
count
5,989
1,786
4,177
11,952

Unweighted
respondent
percent
50.11%
14.94%
34.95%
100.00%

Weighted
respondent
count
739,436
203,795
512,232
1,455,463

Weighted
respondent
percent
50.80%
14.00%
35.19%
100.00%

Standard
error
0.84%
0.42%
0.85%

Figure 6-1 presents the unweighted number of teachers and the weighted percentages for follow-up
experience categories by teacher wave group summarizing Tables 6-1 through 6-4.
Figure 6-1.

Follow-up required prior to response with teacher-interview final respondents by
teacher wave group, NTPS 2015-16

Not surprisingly, the early teacher wave group achieved a high response rate (>90%) without much
extensive follow-up effort. Only about 4% of the teacher respondents responded after receiving

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

39

Teacher Questionnaire
Response Experience in NTPS 2015-16

6

field follow-up. Teachers in later teacher wave group not only had lower response rates but also
received more extensive follow-up. For example, a considerable portion (34%) of respondents in the
late teacher wave group responded after receiving field follow-up. There could be a number of
factors such as


Teachers in the later wave groups didn’t have sufficient time to respond;



Teachers from schools that are slow to send the teacher listing form are less likely to
respond; and



More extensive follow-up procedures started early on for the later wave groups.

Tables 6-5 through 6-8 present the distribution of three response follow-up categories by survey
coordinator status and priority status of teacher respondents’ school.
Table 6-5.

Response follow-up experience with teacher interview final respondents from nonpriority schools without a survey coordinator

Response follow-up category
No or limited follow-up
Telephone/mail follow-up, no field
Field follow-up
Total respondents

Table 6-6.

Unweighted
respondent
percent
65.21%
20.69%
14.10%
100.00%

Weighted
respondent
count
1,389,794
420,694
299,339
1,086,832

Weighted
respondent
percent
65.87%
19.94%
14.19%
100.00%

Standard
error
0.65%
0.45%
0.54%

Response follow-up experience with teacher interview final respondents from nonpriority schools with a survey coordinator

Response follow-up category
No or limited follow-up
Telephone/mail follow-up, no field
Field follow-up
Total respondents

Table 6-7.

Unweighted
respondent
count
12,350
3,919
2,671
9,790

Unweighted
respondent
count
4,989
2,050
2,729
9,768

Unweighted
respondent
percent
51.07%
20.99%
27.94%
100.00%

Weighted
respondent
count
592,710
234,048
313,748
1,140,505

Weighted
respondent
percent
51.97%
20.52%
27.51%
100.00%

Standard
error
0.77%
0.67%
0.78%

Response follow-up experience with teacher interview final respondents from
priority schools without a survey coordinator

Response follow-up category
No or limited follow-up
Telephone/mail follow-up, no field
Field follow-up
Total respondents

Unweighted
respondent
count
1,358
436
424
2,218

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Unweighted
respondent
percent
61.23%
19.66%
19.12%
100.00%

40

Weighted
respondent
count
196,906
58,697
56,353
311,956

Weighted
respondent
percent
63.12%
18.82%
18.06%
100.00%

Standard
error
2.01%
1.45%
1.66%

Teacher Questionnaire
Response Experience in NTPS 2015-16

Table 6-8.

6

Response follow-up experience with teacher interview final respondents from
priority schools with a survey coordinator

Response follow-up category
No or limited follow-up
Telephone/mail follow-up, no field
Field follow-up
Total respondents

Unweighted
respondent
count
715
323
789
1,827

Unweighted
respondent
percent
39.14%
17.68%
43.19%
100.00%

Weighted
respondent
count
106,443
42,435
115,269
264,147

Weighted
respondent
percent
40.30%
16.06%
43.64%
100.00%

Standard
error
1.84%
1.38%
2.13%

Figure 6-2 summarizes Tables 6-5 through 6-8 (the unweighted counts and weighted percentages are
shown). Teacher respondents from schools with a survey coordinator required considerably less
follow-up effort than teacher respondents from schools without a survey coordinator, regardless of
their priority status, which is consistent with findings from SASS 2011-12 and findings from the
school and TLF analysis of this cycle. The teacher follow-up effort prior to response doesn’t appear
different by priority status for teachers from schools with a survey coordinator. On the other hand,
teachers from priority schools without a survey coordinator were slower to respond than the ones
from non-priority schools without a survey coordinator.
Figure 6-2.

Follow-up required prior to response with teacher-interview final respondents by
survey coordinator status and by priority status, NTPS 2015-16

We also looked at the response follow-up experience by experimental group in Table 6-9. Overall,
the special afternoon reminder call did not help in terms of reducing the follow-up effort.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

41

Teacher Questionnaire
Response Experience in NTPS 2015-16

Table 6-9.

6

Response follow-up experience with teacher-interview final respondents by
nonresponse experimental group, NTPS 2015-16

Unweighted
respondent
Response follow-up category
count
Teacher experimental group 1 special afternoon calls
Telephone and mail follow-up, no field
1,787
Field follow-up
3,337
Total respondents
5,124
Teacher experimental group 2 regular call schedule
Telephone and mail follow-up, no field
1,728
Field follow-up
3,276
Total respondents
5,004

Unweighted
respondent
percent

Weighted
respondent
count

Weighted
respondent
percent

34.88%
65.12%
100.00%

201,692
392,807
594,499

33.93%
66.07%
100.00%

0.93%
0.93%

34.53%
65.47%
100.00%

198,112
391,902
590,014

33.58%
66.42%
100.00%

0.91%
0.91%

Appendix E provides further details by school and teacher domains.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

42

Standard
error

Experimental Study in NTPS 2015-16

7

In the 2014-15 NTPS pilot study, an experiment was conducted comparing the paper questionnaire
mode and the internet questionnaire mode for all school-level survey components (school
questionnaire; principal questionnaire; teacher listing form) and it was found that the response rate
of the internet mode group was considerably lower than that of the paper mode group. NPTS 201516 had a similar experimental study testing whether offering an internet questionnaire mode for the
school questionnaire, principal questionnaire, and teacher listing form with improved contact
materials at the onset of data collection altered the response rates at the school level. One thousand
schools were assigned to an experimental group receiving an internet questionnaire mode. During
the data collection period, the follow-up effort for the experimental group ended after the third
school mailing. For that reason, the response rate comparison is based on schools that responded
before the third mailout. Completed cases with no or limited follow-up effort in Section 2 are
considered completed for this comparison. Only response rate comparisons at the school level are
available since teachers were not sampled from the teacher listing forms of the experimental group
schools. Since there were no school weights created for the experimental group, we carry out
unweighted analyses for the experiment.
For the school questionnaire and teacher listing form, the overall response rate is significantly lower
for the internet mode experimental group than the main study group. However, the difference in
principal response rate between the two groups is minimal (<1%) and not statistically significant.
Table 7-1 shows the response rate for the school questionnaire, principal questionnaire, and teacher
listing form.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

43

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table 7-1.

Response rate comparison by experimental group and by questionnaire, NTPS 2015-16
Main Study

Questionnaire
School
Principal
TLF

Sample
Size
8,300
8,300
8,300

Eligible
schools
8,029
8,025
8,025

Completes
3,078
3,169
3,099

Response
rate
38.3%
39.5%
38.6%

Standard
error
0.5%
0.5%
0.5%

Sample
Size
1,000
1,000
1,000

Experimental Group
Eligible
Response
schools
Completes
rate
980
288
29.4%
978
380
38.9%
980
291
29.7%

Standard
error
1.4%
1.5%
1.4%

44
Experimental Study in NTPS 2015-16

7

Experimental Study in NTPS 2015-16

7

We analyzed for significant differences between the main study group and the experimental group in
terms of response rates using a logistic regression model with one of the response status variables as
a dependent variable and the experimental group variable (main study vs. experimental group) and
other school domain variables (up to two-way interaction terms). The logistic regression analysis was
conducted unweighted assuming an independent, identically distributed with replacement
distribution. Firstly, we selected a set of explanatory variables for the response propensity model for
each questionnaire by including all of the school domain variables and their two-way interaction
terms and finding variables that are selected by three model selection methods (forward, backward,
and stepwise). Then, we included the experimental status in addition to the selected variables in the
logistic regression in order to find out if initial internet contacts affected the response rate for any
questionnaire conditional on school domains. Detailed tables are included in Appendix F.
Conditional on school domains, the school response rate and the teacher listing form response rate
are lower for the experimental group than the for main study group, which is consistent with the
findings from the pilot study. However, the experimental group variable is not significant in the
logistic regression model for the principal questionnaire.
We also conducted chi-square tests to test independence of the school characteristics and the data
collection modes among school-interview respondents, and found that only the distribution by
special district status differed by mode. In particular, the percentage of special district schools is
lower with the internet mode than with the paper mode among responding schools.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

45

Summary and Conclusions

8

Much has been learned in this NTPS 2015-16 cycle as to methodology for data collection. The
accelerated field collection efforts for priority schools and for non-priority schools which do not
designate a school coordinator was very successful in reducing the gap in response rates for many
of the school domains which have had traditionally lower response rates (city and suburban schools,
high-poverty schools, high and combined schools). The overall response rates were not improved,
but on the other hand they did not decrease from the previous cycle, which had happened quite
often in SASS. This improvement is certainly in the right direction, and these policies should be
continued and further developed in future cycles.
It is still the case that schools with a survey coordinator are likely to have higher response rates, and
lower levels of needed follow-up operations. We do not know how much of is because of the survey
coordinator’s efforts, and how much is because schools that designate survey coordinators are
generally more cooperative in all phases of data collection. Probably both factors contribute to the
difference, but it is unknown how much can be attributed quantitatively to each without some type
of experiment being carried out explicitly.
The new methods to collect teacher listing forms from alternative sources (from a commercial
vendor; through clerical searches of the internet) were generally successful. The teacher listing form
response rate was marginally increased (SASS 2011-12 was 82.2%; NTPS 2015-16 was 83.6%). The
TLFs from the vendor and/or clerical search have a somewhat higher ineligibility rate. On the
negative side, the teacher questionnaire response rate was much lower in NTPS 2015-16 than in
SASS 2011-12 (for unit response rate, SASS 2011-12 was 77.7% vs. NTPS 2015-16 was 68.1%; for
overall response rate, SASS 2011-12 was 63.9% vs. NTPS 2015-16 was 56.9%). However, the
alternative sources allowed to collect more TLFs and provided information on teachers that would
not have been available through the traditional method. In addition, the overall teacher response rate
is higher than what it would have been without the alternative sources. We believe these methods
should continue to be explored in future cycles of NTPS.
A small experimental study was done for teachers who required a telephone nonresponse follow-up
(so that teachers who responded before this scheduled telephone follow-up would not be part of
this experiment). The experimental treatment was receiving the telephone calls in the afternoon

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

46

Summary and Conclusions

8

rather than the regular call schedule. This experiment showed a very small (but not statistically
significant) increase in response rate among those who received the special afternoon calls.
A larger experimental study was done of 1,000 schools that were asked to respond by internet. This
is less expensive in terms of data collection costs if it is even neutral in terms of cooperation levels.
In NTPS 2015-16 as in NTPS 2014-15 pilot study, the school questionnaire response rates and the
teacher listing form response rates were all significantly lower in the experimental group: about 10
percentage points lower. Intriguingly, the same was not true of the principal questionnaire response
rates. We have not found any significant difference in distribution of school-interview respondents
between the main study schools and the experimental group schools except for the distribution by
special district status. The principal questionnaire response rates in the experimental group were
about the same as for the main study.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

47

Appendix A
School Control Data File Analysis:
Details and Assumptions Made

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

A-1

School Control Data File Analysis:
Details and Assumptions Made

A

This appendix documents the procedures used and assumptions made for the creation of the followup experience flags used in the paradata analyses. Each completed school questionnaire (SQ),
principal questionnaire (PQ), and teacher listing form (TLF) was assigned a flag indicating the extent
of follow-up that was needed before the case was coded as complete. The School Control Database
(SCD) file was processed for this work.
For the SQ, PQ, and TLF, the school domains of interest are:


Special District Flag;



Charter Status;



School Span (primary, middle, high, combined);



Urbanicity (city, suburban, town, rural);



Census Region (Northeast, Central, South, West);



Percentage students eligible for free or reduced price lunch (FRPL: 0 to 35%, 35% to
50%, 50% to 75%, 75% to 100%, not participating);



School Size (enrollment 0 to 99, 100 to 199, 200 to 499, 500 to 749, 750 to 999, 1000 or
more); and



School Type (regular, special education, vocational, alternative).

The first two school domains listed above were already on the SCD file, but the rest were not. They
were recreated using the same logic that was used during frame development. During this process,
the following issues and decisions arose:


School Span: Low grade and high grade variables had been revised at some point
during the data collection, and both the original and revised version of the variables
were on the SCD file. A few revised low grade values were “DC” or “K,” which are
invalid values. We used the original low grade value (instead of “DC”) and “KG”
(instead of “K”) in these cases;



FRPL: We could not assign any schools to the “not participating” category because
missing data (both truly missing and inapplicable) had been fully imputed on the SCD
file. Also, during the process of computing FRPL, the ratio of number of free or
reduced price lunch students to total enrollment is computed. For three schools, this
ratio was equal to or greater than 1. For two of these schools, the ratio was 1 and so not
a concern, but for one school, the ratio was 40.31. A ratio of 1 was used for all three
schools, i.e., assuming that all students were eligible for free or reduced price lunch;

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

A-2

School Control Data File Analysis:
Details and Assumptions Made


A

School Type: 15 schools on the SCD had missing school type, and were assumed to be
regular schools;

Follow-up experience flags were defined as follows. For the operational study SQ and PQ, nonpriority schools with a survey coordinator were assigned as follows:


NPSC1: Interview completed before third school mailout (Limited or No Follow-up);



NPSC2: Interview completed before telephone follow-up (Follow-up with Survey
Coordinator and Mailouts);



NPSC3: Interview completed before Phase 2 field follow-up (Follow-up Including All
Telephone and Mailout Phases);



NPSC4: Interview completed after Phase 2 field follow-up (Follow-up including Field
Follow-up).

For the operational study SQ and PQ, priority schools and non-priority schools without survey
coordinators were assigned as follows:


OTH1: Interview completed before Phase 1 Field Follow-up (Limited or No Followup);



OTH2: Interview completed after Phase 1 Field Follow-up (Follow-up with Phase 1
Field Follow-up);



OTH3: Interview completed after further follow-up after Phase 1 Field Follow-up
(Follow-up Including All Phases);

For the experimental study SQ and PQ, all schools were assigned as follows:


NPSC1: Interview completed before third school mailout (Limited or No Follow-up);



NPSC2: Interview completed after third school mailout (Follow-up with Survey
Coordinator and Mailouts)

For the operational and experimental studies TLF, all schools were assigned as follows:


TLF1: TLF sent in by school before Phase 1 Field Follow-up or Third School Mailout
(Limited or No Follow-up);



TLF2: TLF sent in by school before Vendor File Matching (Phase 1 Field or Third
School Mailout);

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

A-3

School Control Data File Analysis:
Details and Assumptions Made

A



TLF3: TLF obtained through Vendor File Matching or Clerical Research
(Vendor/Clerical Follow-up) and no TLF from school;



TLF4: TLF sent in by school after Vendor/Clerical processes failed (Follow-up after
Vendor/Clerical failed).

Interviews were considered complete if STATUS_SQ was “01” or “02,” STATUS_PQ was “01” or
“02,” or STATUS_TLF was “01,” for the SQ, PQ, and TLF respectively.
In order to assign the follow-up experience flags, “in” flags for each stage of data collection were
assigned first. The “in” flags identified completed cases that were considered to have gone through
that stage of data collection. All of these flags were set for completed cases only; this analysis is only
for completed cases. Different stages of data collection were relevant to different types of cases as
shown in Table A-1.
Table A-1.

Relevant stages of data collection by type of case

Type of case
Operational study SQ and PQ, non-priority schools
with a survey coordinator
Operational study SQ and PQ, priority schools and
non-priority schools without survey coordinators
Experimental study SQ and PQ, all schools
Operational study TLF, all schools
Experimental study TLF, all schools

Relevant stages of data collection
MAIL3, FFU1, REMIND, MAIL4, NRFU, FFU2
FFU1, REMIND, MAIL4, NRFU, FFU2
MAIL3, REMIND
MAIL3, FFU1, VENDOR, RSRCH, REMIND, MAIL4
MAIL3, VENDOR, RSRCH, REMIND

Notes: MAIL3 is Third School Mailout, FFU1 is Phase 1 Field Follow-up, REMIND is Survey Coordinator Reminder Operation, MAIL4 is
Fourth School Mailout including FedEx, NRFU is Telephone Follow-up, FFU2 is Phase 2 Field Follow-up, VENDOR is vendor file matching,
and RSRCH is clerical research operation. We did not expect non-priority schools with survey coordinators to go through FFU1, but
some did. We did not expect priority schools and non-priority schools without survey coordinators to go through REMIND because most
of these don’t even have a survey coordinator, but some did. For the TLF, all operational schools, regardless of survey coordinator
status or priority status, went through the same processes. Follow-up for the experimental schools was limited.

The “in” flags for each stage of data collection were set as shown in Table A-2.
Table A-2.

Setting the “in” flags for each process

Stage of data collection
MAIL3
FFU1
REMIND
MAIL4, PQ or SQ
MAIL4, TLF
NRFU
FFU2
VENDOR
RSRCH

Completed cases counted as “in” if:
Flagged for MAIL3 and no LMR date
Flagged for FFU1 and (no LMR date or LMR date is after 11/9/2015)
Flagged for REMIND
Flagged for MAIL4 and no LMR date, or flagged for FedEx
Flagged for FedEx
Flagged for NRFU and no LMR date or LMR date is after 2/16/2016
Flagged for FFU2 and no LMR date or LMR date is after 3/21/2016
Flagged for VENDOR
Flagged for RSRCH and not pulled due to receipt of paper questionnaire
[i.e., not (LMR date of 11/19/2015 and TLF source is paper)]

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

A-4

School Control Data File Analysis:
Details and Assumptions Made

A

Notice that cases with LMR dates were handled differently (i.e., counted as “in” or not) depending
on the stage of data collection. These decisions had to do with the actual LMR dates as they related
to the process start dates, and how far apart the processes were from each other. For example, only
cases with no LMR date were considered “in” MAIL3 because the LMR date was 11/9/15 and the
process started on 11/15/15. Similarly, only cases with no LMR date were considered “in” MAIL4
because the LMR date was 1/20/16, nearly a month before that process started. On the other hand,
cases were considered “in” REMIND regardless of the LMR date. This was because the LMR date
was 1/4/16, which was in the middle of the REMIND process that ran from 12/1/15 – 12/18/15,
took a break and then resumed from 1/4/16 – 1/22/16. The FFU1, NRFU, and FFU2 processes
had LMR cutoff dates. Only cases with no LMR date or LMR dates after the cutoff were considered
“in” the process.
There were 49 SQ records and 48 PQ records with the FFU2 flag missing and an LMR date filled in.
These records were treated as not in FFU2.
The information in Tables A-1 and A-2 was based on patterns and details observed in the actual
data. These may be different than the data collection plans outlined in the Operations Overview
dated 5/19/2015.
The follow-up experience flags were assigned as shown in Table A-3.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

A-5

School Control Data File Analysis:
Details and Assumptions Made

Table A-3.

A

Rules for assigning follow-up experience flags by type of case

Type of case
Operational study SQ
and PQ, non-priority
schools with a survey
coordinator

Operational study SQ
and PQ, priority
schools and nonpriority schools
without survey
coordinators

Experimental study
SQ and PQ, all
schools

Operational study
TLF, all schools

Experimental study
TLF, all schools

Follow-up experience
flag value
NPSC1
(Limited or no follow-up)
NPSC2
(Follow-up with survey
coordinator and mailouts)
NPSC3
(Follow-up including all
telephone and mailout
phases)
NPSC4
(Follow-up including field
follow-up)
OTH1
(Limited or no follow-up)
OTH2
(Follow-up with Field Followup Only or Phone/Mail
Follow up Only)
OTH3
(Follow-up Including All
Phases)
NPSC1
(Limited or no follow-up)
NPSC2
(Follow-up with Survey
Coordinator and Mailouts)
TLF1
(Limited or no follow-up)
TLF2
(Phase 1 Field or Third
School Mailout)
TLF3
(Vendor/Clerical Follow-up)
TLF4
(Follow-up After
Vendor/Clerical Failed)
TLF1
(Limited or no follow-up)
TLF2
(Phase 1 Field or Third
School Mailout)
TLF3
(Vendor/Clerical Follow-up)
TLF4
(Follow-up After
Vendor/Clerical Failed)

Rules
Not in MAIL3, FFU1, REMIND, MAIL4, NRFU, or FFU2
In MAIL3 or REMIND and not in NRFU or FFU1 or FFU2

In NRFU and not in FFU1 or FFU2

In FFU1 or FFU2

Not in FFU1, REMIND, MAIL4, NRFU, or FFU2
In FFU1 or FFU2 and not in REMIND, MAIL4, or NRFU,
OR
Not in FFU1 or FFU2 and in REMIND, MAIL4, or NRFU
In FFU1 or FFU2 and in REMIND, MAIL4, or NRFU

Not in MAIL3 or REMIND
In MAIL3 or REMIND

Not in MAIL3 or FFU1
Not in VENDOR, OR
In VENDOR and source of TLF is not vendor or clerical
and not in RSRCH or REMIND or MAIL4
Source of TLF is vendor or clerical
In VENDOR and source of TLF is not vendor or clerical
and in REMIND or MAIL4 OR
In RSRCH and source of TLF is not vendor or clerical
Not in MAIL3
Not in VENDOR, OR
In VENDOR and source of TLF is not vendor or clerical
and not in RSRCH or REMIND
Source of TLF is vendor or clerical
In VENDOR and REMIND and source of TLF is not
vendor or clerical OR
In RSRCH and source of TLF is not vendor or clerical

Note: The terms “not in” or “in” above reference use of the “in” flags.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

A-6

Appendix B
Teacher Control Data File Analysis:
Details and Assumptions Made

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

B-1

Teacher Control Data File Analysis:
Details and Assumptions Made

B

This appendix documents the procedures used and assumptions made for the creation of the followup experience flags used in the paradata analyses. Each completed teacher questionnaire (TQ) was
assigned a flag indicating the extent of follow-up that was needed before the case was coded as
complete. The Teacher Control Database (TCD) file was processed for this work.
For the TQ, the school domains described in Appendix A were brought over to the teacher file. In
addition to these, the teacher domain Teacher Subject within School Span was created by crossing
School Span (primary, middle, high, combined) with Teacher Subject (special ed, general elementary,
math, science, English/language arts, social studies, vocational/technical, other). The “high schoolgeneral elementary” category was combined with “high school-other,” and the “primaryvocational/technical” category was combined with “primary-other,” yielding 30 distinct categories.
The follow-up experience flag was defined as follows:


TQ1: Interview completed before telephone follow-up (Limited or No Follow-up);



TQ2: Interview completed before Phase 2 field follow-up (Telephone and Mailout
Follow-up Only);



TQ3: Interview completed after Phase 2 field follow-up (Follow-up including Field
Follow-up).

Interviews were considered complete if STATUS_TCH was “01” or “02”.
In order to assign the follow-up experience flags, “in” flags for each stage of data collection were
assigned first. The “in” flags identified completed cases that were considered to have gone through
that stage of data collection. All of these flags were set for completed cases only; this analysis is only
for completed cases. Teachers went through the same stages of data collection regardless of school
survey coordinator status, teacher treatment (whether the initial contact was by email, mail, or
paper), or other factors. The stages of data collection relevant to teachers are shown in Table B-1.
Table B-1.

Relevant stages of data collection for teachers

Type of case
All teachers

Relevant stages of data collection
MAIL, EMAIL, REMIND, NRFU, FFU2, RMDEXT

Notes: MAIL is the three Teacher Follow-up Mailouts, EMAIL is the three Teacher Follow-up Emails, REMIND is the Survey Coordinator
Reminder Operation, NRFU is Telephone Follow-up, FFU2 is Phase 2 Field Follow-up, RMDEXT is the Phase 2 Reminder Operation.
Cases that went through REMIND or FFU2 did not also go through RMDEXT, and vice versa. The RMDEXT process was for teacher waves
19-24 only.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

B-2

Teacher Control Data File Analysis:
Details and Assumptions Made

B

The “in” flags for each stage of data collection were set as shown in Table B-2.
Table B-2.

Setting the “in” flags for each process

Stage of data collection
MAIL
EMAIL
REMIND
NRFU
FFU2
RMDEXT

Completed cases counted as “in” if:
Flagged for any of the three follow-up mailouts
Flagged for any of the three follow-up emails
Flagged for REMIND
Flagged for NRFU and no LMR date or LMR date is after 2/16/2016
Flagged for FFU2 and no LMR date or LMR date is after 3/21/2016
Flagged for RMDEXT and no LMR date or LMR date is after 6/5/2016

Notice that cases with LMR dates were handled differently (i.e., counted as “in” or not) depending
on the stage of data collection. These decisions had to do with the actual LMR dates as they related
to the process start dates, and how far apart the processes were from each other. One complication
regarding LMR dates that arose with teachers that didn’t happen with the schools, principals, or
TLFs is that the teachers were sampled and their data were collected on a flow basis. This meant
that while the start dates for each process were appropriately staggered, the end date for one process
often was well after the start date for the next process. This made identifying cases that were
completed before the next process began difficult. Cases were considered “in” MAIL regardless of
the LMR date. Cases were also considered “in” REMIND regardless of the LMR date. This was
because the earliest LMR date was 1/4/16, which was in the middle of the REMIND process that
ran from 12/1/15 – 12/18/15, took a break and then resumed from 1/4/16 – 1/22/16. The
NRFU, FFU2, and MDEXT processes had LMR cutoff dates. Only cases with no LMR date or
LMR dates after the cutoff were considered “in” the process.
The information in Tables B-1 and B-2 was based on patterns and details observed in the actual
data. These may be different than the data collection plans outlined in the Operations Overview
dated 5/19/2015.
The follow-up experience flag was assigned as shown in Table B-3.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

B-3

Teacher Control Data File Analysis:
Details and Assumptions Made

Table B-3.

Rules for assigning follow-up experience flag by flag value, all teachers

Follow-up experience flag value
TQ1
(Limited or no follow-up)
TQ2
(Telephone and Mailout Follow-up Only)
TQ3
(Follow-up including Field Follow-up)

Rules
Not in REMIND, NRFU, FFU2, or RMDEXT
In REMIND, NRFU, or RMDEXT and not in FFU2
In FFU2

Note: The terms “not in” or “in” above reference use of the “in” flags.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

B-4

B

Appendix C
School Questionnaire Experience:
Detailed Tables

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-1

School Questionnaire Experience:
Detailed Tables

C

Tables C-1, C-2, C-3, and C-4 show the rates of responding non-priority schools with a survey
coordinator by response follow-up category (no/limited follow-up; survey coordinator telephone
reminder and mailout; telephone and mail follow-up, no field; field follow-up) as a percentage of all
responding non-priority schools with a survey coordinator across the school domains (special
district flag, charter school status, school span, urbanicity, Census region, poverty level, school size,
and school type). The tables presents the number of completed school interviews; the number,
unweighted percentage and weighted percentage of completed school interviews with one of levels
of follow-up; and the standard error of the weighted percentage. We used the Rao-Scott Chi-Square
statistic for testing independence between response follow-up experience and school domain. The
statistic takes into consideration design effects from the complex survey design. Its p-value is also
presented in the tables. Any significant difference in response follow-up experience by domain is
shaded in the tables in Appendix C.
At a significance level of 0.05, the four tables show that among responding non-priority schools with
a survey coordinator,


The percentage of schools that received no to limited follow-up varies by charter school
status, urbanicity, Census region, and poverty status;



The percentage of schools that received survey coordinator follow-up or mailout varies
by poverty status;



The percentage of schools that received telephone and mail follow-up varies by charter
school status and Census region; and



The percentage of schools that received field follow-up varies by charter school status,
urbanicity, and poverty status.

The following schools tend to require more follow-up to achieve final response: charter schools, city
schools, schools in the Northeast region, and schools where 75% of more of the students are eligible
for free or reduced price lunch.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-2

C

School Questionnaire Experience:
Detailed Tables

Table C-1.

Percentage of school respondents that received no/limited follow-up by school
domain, among non-priority schools with a survey coordinator
Weighted percent
Weighted
percent
no or limited
Standard
follow-up
error
Pvalue
79.3%
4.6% 0.1214
71.2%
0.9%

Number of
completes
95
3,032

Number of
completes with
no or limited
follow-up
74
2,128

Unweighted
percent
no or
limited
follow-up
77.9%
70.2%

Charter
Non-charter

408
2,719

246
1,956

60.3%
71.9%

59.8%
72.2%

2.8%
0.9%

0.0000

Primary
Middle
High
Combined

1,435
551
708
433

997
404
500
301

69.5%
73.3%
70.6%
69.5%

71.3%
73.7%
70.1%
70.7%

1.3%
1.9%
2.0%
2.3%

0.6089

City
Suburban
Town
Rural

617
921
579
1,010

391
660
419
732

63.4%
71.7%
72.4%
72.5%

64.6%
71.5%
73.2%
74.4%

1.9%
1.6%
1.9%
1.6%

0.0005

Northeast
Midwest
South
West

480
910
1,142
595

329
686
768
419

68.5%
75.4%
67.3%
70.4%

70.3%
76.5%
68.4%
70.3%

2.2%
1.6%
1.5%
2.1%

0.0032

975
606
963
583

729
432
677
364

74.8%
71.3%
70.3%
62.4%

76.4%
72.1%
71.2%
63.0%

1.5%
2.1%
1.8%
2.3%

0.0000

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

141
244
1,160
825
349
408

98
171
821
572
255
285

69.5%
70.1%
70.8%
69.3%
73.1%
69.9%

73.4%
72.7%
71.8%
69.8%
72.9%
69.8%

4.1%
3.0%
1.5%
1.6%
2.6%
2.4%

0.8477

Regular
Special education
Vocational
Alternative

2,961
45
40
81

2,091
31
29
51

70.6%
68.9%
72.5%
63.0%

71.7%
72.3%
69.3%
65.1%

0.9%
7.4%
8.4%
5.6%

0.6066

Domain
Special district
Not special district

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-3

C

School Questionnaire Experience:
Detailed Tables

Table C-2.

Percentage of school respondents that received survey coordinator follow-up
(telephone reminder) and mailout by school domain, among non-priority schools
with a survey coordinator
Weighted percent
Weighted
percent
coordinator
follow-up
Standard
mailout
error
13.9%
3.9%
17.6%
0.8%

Number of
completes
95
3,032

Number of
completes
with
coordinator
follow-up
/mailout
12
540

Unweighted
percent
coordinator
follow-up
mailout
12.6%
17.8%

Charter
Non-charter

408
2,719

82
470

20.1%
17.3%

21.3%
17.2%

2.4%
0.8%

0.0883

Primary
Middle
High
Combined

1,435
551
708
433

268
87
119
78

18.7%
15.8%
16.8%
18.0%

17.9%
15.8%
17.5%
17.7%

1.1%
1.5%
1.6%
2.0%

0.7663

City
Suburban
Town
Rural

617
921
579
1,010

128
152
104
168

20.7%
16.5%
18.0%
16.6%

20.9%
17.3%
17.9%
15.4%

1.7%
1.2%
1.7%
1.2%

0.0532

Northeast
Midwest
South
West

480
910
1,142
595

81
148
221
102

16.9%
16.3%
19.4%
17.1%

17.4%
15.8%
18.9%
17.4%

1.9%
1.2%
1.3%
1.7%

0.4147

975
606
963
583

150
100
172
130

15.4%
16.5%
17.9%
22.3%

14.7%
16.8%
17.3%
22.9%

1.2%
1.8%
1.5%
1.9%

0.0038

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

141
244
1,160
825
349
408

27
46
199
157
52
71

19.1%
18.9%
17.2%
19.0%
14.9%
17.4%

15.2%
17.2%
17.2%
19.1%
16.1%
17.5%

2.7%
2.6%
1.1%
1.4%
2.4%
2.0%

0.8040

Regular
Special education
Vocational
Alternative

2,961
45
40
81

517
7
7
21

17.5%
15.6%
17.5%
25.9%

17.3%
13.9%
19.3%
23.8%

0.8%
6.0%
7.0%
4.8%

0.4107

Domain
Special district
Not special district

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-4

Pvalue
0.4077

School Questionnaire Experience:
Detailed Tables

Table C-3.

C

Percentage of school respondents that received telephone and mail follow-up by
school domain, among non-priority schools with a survey coordinator
Weighted percent
Weighted
percent
telephone
and phone
Standard
follow-up
error
Pvalue
4.2%
2.0% 0.7819
4.8%
0.4%

Number of
completes
95
3,032

Number of
completes
with
telephone and
phone followup
5
159

Unweighted
percent
telephone
and phone
follow-up
5.3%
5.2%

Charter
Non-charter

408
2,719

30
134

7.4%
4.9%

7.5%
4.6%

1.4%
0.4%

0.0201

Primary
Middle
High
Combined

1,435
551
708
433

71
25
46
22

4.9%
4.5%
6.5%
5.1%

4.6%
4.2%
6.3%
4.1%

0.6%
0.9%
1.0%
0.9%

0.2903

City
Suburban
Town
Rural

617
921
579
1,010

40
42
31
51

6.5%
4.6%
5.4%
5.0%

5.5%
4.6%
5.2%
4.5%

0.9%
0.8%
0.9%
0.8%

0.8075

Northeast
Midwest
South
West

480
910
1,142
595

28
32
70
34

5.8%
3.5%
6.1%
5.7%

5.1%
2.7%
5.9%
5.7%

1.0%
0.5%
0.7%
1.1%

0.0079

975
606
963
583

41
33
57
33

4.2%
5.4%
5.9%
5.7%

3.7%
4.9%
6.1%
4.5%

0.7%
0.9%
0.9%
0.9%

0.1568

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

141
244
1,160
825
349
408

9
11
65
42
14
23

6.4%
4.5%
5.6%
5.1%
4.0%
5.6%

6.4%
3.5%
4.7%
5.0%
4.1%
5.9%

2.6%
1.0%
0.6%
0.8%
1.2%
1.4%

0.7695

Regular
Special education
Vocational
Alternative

2,961
45
40
81

154
4
2
4

5.2%
8.9%
5.0%
4.9%

4.8%
6.7%
4.8%
4.9%

0.4%
3.4%
3.5%
3.0%

0.9764

Domain
Special district
Not special district

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-5

School Questionnaire Experience:
Detailed Tables

Table C-4.

C

Percentage of school respondents that received field follow-up by school domain,
among non-priority schools with a survey coordinator
Weighted percent
Weighted
percent
field followStandard
up
error
Pvalue
2.6%
1.6% 0.1119
6.4%
0.5%

Number of
completes
95
3,032

Number of
completes
with
field follow-up
4
205

Unweighted
percent
field followup
4.2%
6.8%

Charter
Non-charter

408
2,719

50
159

12.3%
5.8%

11.4%
6.0%

1.7%
0.5%

0.0003

Primary
Middle
High
Combined

1,435
551
708
433

99
35
43
32

6.9%
6.4%
6.1%
7.4%

6.2%
6.3%
6.1%
7.5%

0.7%
1.2%
1.0%
1.2%

0.8807

City
Suburban
Town
Rural

617
921
579
1,010

58
67
25
59

9.4%
7.3%
4.3%
5.8%

9.0%
6.6%
3.7%
5.7%

1.2%
0.8%
0.8%
0.9%

0.0064

Northeast
Midwest
South
West

480
910
1,142
595

42
44
83
40

8.8%
4.8%
7.3%
6.7%

7.3%
5.0%
6.8%
6.5%

1.2%
0.8%
0.8%
1.3%

0.4182

975
606
963
583

55
41
57
56

5.6%
6.8%
5.9%
9.6%

5.2%
6.3%
5.4%
9.6%

0.8%
1.0%
0.8%
1.4%

0.0082

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

141
244
1,160
825
349
408

7
16
75
54
28
29

5.0%
6.6%
6.5%
6.5%
8.0%
7.1%

5.0%
6.6%
6.3%
6.2%
6.9%
6.8%

1.8%
1.8%
0.8%
0.9%
1.4%
1.3%

0.9749

Regular
Special education
Vocational
Alternative

2,961
45
40
81

199
3
2
5

6.7%
6.7%
5.0%
6.2%

6.3%
7.1%
6.6%
6.3%

0.5%
4.3%
4.7%
2.6%

0.9978

Domain
Special district
Not special district

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

Tables C-5, C-6, and C-7 show the rates of responding priority schools by response follow-up
category (no/limited follow-up, phase 1 field follow-up, further follow-up after phase 1) as a
percentage of all responding priority schools across the school domains (special district flag, charter
school status, school span, urbanicity, Census region, poverty level, school size, and school type).
The statistics of the tables are identical to the ones in Tables C-1 through C-4.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-6

School Questionnaire Experience:
Detailed Tables

C

At a significant level of 0.05, the three tables show that among priority schools,


The percentage of schools that received no to limited follow-up varies by urbanicity,
Census region, and poverty status;



The percentage of schools that received phase 1 field follow-up varies by urbanicity and
Census region;



The percentage of schools that received further follow-up after phase 1 varies by
urbanicity, Census region, poverty status, and school type.

The following schools tend to require more follow-up to achieve final response: city schools, town
schools, schools in the Northeast, Midwest, and South regions (i.e., only the West stands out with
less followup needed), and schools where 50%-75% are eligible for free or reduced price lunch and
more strongly schools were 75% of more of the students are eligible for free or reduced price lunch.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-7

C

School Questionnaire Experience:
Detailed Tables

Table C-5.

Percentage of school respondents that received no/limited follow-up by school
domain, among priority schools
Weighted percent
Weighted
percent
no or limited
Standard
follow-up
error
Pvalue
44.2%
1.9%
0.1356
27.2%
10.7%

Number of
completes
675
24

Number of
completes
with
no or limited
follow-up
309
6

Unweighted
percent
no or limited
follow-up
45.8%
25.0%

Charter
Non-charter

101
598

48
267

47.5%
44.6%

48.6%
43.2%

5.1%
2.0%

0.3348

Primary
Middle
High
Combined

352
126
152
69

165
55
63
32

46.9%
43.7%
41.4%
46.4%

46.2%
41.9%
36.8%
44.7%

2.6%
4.9%
4.7%
6.4%

0.2774

City
Suburban
Town
Rural

433
223
24
19

171
122
9
13

39.5%
54.7%
37.5%
68.4%

39.0%
53.3%
30.9%
62.4%

2.4%
3.5%
10.1%
13.6%

0.0039

Northeast
Midwest
South
West

121
65
295
218

36
23
138
118

29.8%
35.4%
46.8%
54.1%

28.2%
34.9%
46.0%
52.9%

4.7%
6.6%
3.2%
3.5%

0.0007

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

128
76
167
328

76
41
71
127

59.4%
53.9%
42.5%
38.7%

59.8%
54.5%
39.9%
37.8%

4.9%
6.8%
4.3%
2.8%

0.0007

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

35
27
213
193
96
135

15
11
88
92
48
61

42.9%
40.7%
41.3%
47.7%
50.0%
45.2%

34.4%
33.8%
41.6%
47.4%
50.2%
43.4%

10.1%
10.7%
3.4%
3.5%
5.3%
4.7%

0.4837

Regular
Special education
Vocational
Alternative

627
16
7
49

286
8
4
17

45.6%
50.0%
57.1%
34.7%

44.8%
41.8%
48.1%
32.4%

2.0%
13.7%
27.4%
7.9%

0.4967

Domain
Special district
Not special district

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-8

School Questionnaire Experience:
Detailed Tables

Table C-6.

C

Percentage of school respondents that received phase 1 follow-up by school
domain, among priority schools
Weighted percent
Weighted
percent
coordinator
follow-up
Standard
mailout
error
Pvalue
43.9%
1.8%
0.0851
63.3%
11.4%

Number of
completes
675
24

Number of
completes
with
coordinator
follow-up
mailout
284
15

Unweighted
percent
Phase 1 field
follow-up
42.1%
62.5%

Charter
Non-charter

101
598

36
263

35.6%
44.0%

35.9%
45.2%

4.9%
2.0%

0.1046

Primary
Middle
High
Combined

352
126
152
69

153
53
70
23

43.5%
42.1%
46.1%
33.3%

43.8%
43.0%
51.1%
34.3%

2.7%
4.8%
5.4%
6.3%

0.3019

City
Suburban
Town
Rural

433
223
24
19

208
75
10
6

48.0%
33.6%
41.7%
31.6%

49.5%
34.0%
46.8%
37.6%

2.4%
3.5%
11.7%
13.6%

0.0099

Northeast
Midwest
South
West

121
65
295
218

79
29
110
81

65.3%
44.6%
37.3%
37.2%

64.9%
47.7%
37.8%
39.5%

5.3%
6.3%
2.7%
3.6%

0.0001

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

128
76
167
328

47
28
70
154

36.7%
36.8%
41.9%
47.0%

37.6%
37.0%
45.7%
47.8%

4.9%
6.7%
4.5%
2.9%

0.2512

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

35
27
213
193
96
135

12
11
105
76
37
58

34.3%
40.7%
49.3%
39.4%
38.5%
43.0%

42.9%
46.0%
49.7%
40.0%
41.0%
44.3%

10.0%
11.0%
3.5%
3.2%
5.3%
4.4%

0.6172

Regular
Special education
Vocational
Alternative

627
16
7
49

273
6
3
17

43.5%
37.5%
42.9%
34.7%

45.0%
50.7%
51.9%
37.9%

2.0%
14.2%
27.4%
8.5%

0.8018

Domain
Special district
Not special district

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-9

C

School Questionnaire Experience:
Detailed Tables

Table C-7.

Percentage of school respondents that received further follow-up after phase 1 by
school domain, among priority schools
Weighted percent

Number of
completes
675
24

Number of
completes
with
telephone
and phone
follow-up
82
3

Unweighted
percent
further
follow-up
after
phase 1
12.1%
12.5%

Weighted
percent
telephone
and phone
follow-up
11.9%
9.4%

Standard
error
1.4%
5.4%

Charter
Non-charter

101
598

17
68

16.8%
11.4%

15.6%
11.5%

3.9%
1.4%

0.2911

Primary
Middle
High
Combined

352
126
152
69

34
18
19
14

9.7%
14.3%
12.5%
20.3%

10.0%
15.1%
12.2%
21.0%

1.7%
3.3%
3.2%
5.4%

0.1500

City
Suburban
Town
Rural

433
223
24
19

54
26
5
-

12.5%
11.7%
20.8%
0.0%

11.5%
12.7%
22.3%
0.0%

1.7%
2.6%
9.8%
0.0%

0.0000

Northeast
Midwest
South
West

121
65
295
218

6
13
47
19

5.0%
20.0%
15.9%
8.7%

6.9%
17.3%
16.2%
7.6%

3.0%
4.6%
2.3%
2.0%

0.0206

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

128
76
167
328

5
7
26
47

3.9%
9.2%
15.6%
14.3%

2.6%
8.5%
14.4%
14.4%

1.4%
3.7%
3.0%
2.1%

0.0072

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

35
27
213
193
96
135

8
5
20
25
11
16

22.9%
18.5%
9.4%
13.0%
11.5%
11.9%

22.7%
20.1%
8.7%
12.6%
8.8%
12.3%

8.2%
8.8%
1.9%
2.4%
2.8%
3.0%

0.1154

Regular
Special education
Vocational
Alternative

627
16
7
49

68
2
15

10.8%
12.5%
0.0%
30.6%

10.2%
7.5%
0.0%
29.6%

1.3%
5.7%
0.0%
7.1%

0.0000

Domain
Special district
Not special district

Pvalue
0.6883

Tables C-8, C-9, and C-10 show the rates of responding non-priority schools without a survey
coordinator by response follow-up category (no/limited follow-up, phase 1 field follow-up, further
follow-up after phase 1) as a percentage of all responding non-priority schools without a survey
coordinator across the school domains (special district flag, charter school status, school span,
NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-10

School Questionnaire Experience:
Detailed Tables

C

urbanicity, Census region, poverty level, school size, and school type). The statistics of the tables are
identical to the ones in Tables C-1 through C-4.
At a significance level of 0.05, the three tables show that among non-priority schools without a
survey coordinator,


The percentage of schools that received no to limited follow-up varies by charter status;



The percentage of schools that received phase 1 field follow-up varies by Census region;
and



The percentage of schools that received further follow-up after phase 1 varies by
Census region.

The following schools tend to require more follow-up to achieve final response: charter schools and
schools in the Southern region (more phase 1 field follow-up).

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-11

C

School Questionnaire Experience:
Detailed Tables

Table C-8.

Percentage of school respondents that received no/limited follow-up by school
domain, among non-priority schools without a survey coordinator
Weighted percent
Weighted
percent
no or limited
Standard
follow-up
error
Pvalue
45.4%
8.5% 0.0553
29.8%
1.2%

Number of
completes
52
1,896

Number of
completes
with
no or limited
follow-up
17
544

Unweighted
percent
no or limited
follow-up
32.7%
28.7%

274
1,674

61
500

22.3%
29.9%

22.1%
30.8%

2.8%
1.2%

0.0070

Primary
Middle
High
Combined

820
326
517
285

248
85
142
86

30.2%
26.1%
27.5%
30.2%

31.4%
25.6%
30.7%
29.9%

1.8%
2.4%
2.7%
3.0%

0.3444

City
Suburban
Town
Rural

514
571
327
536

138
150
106
167

26.8%
26.3%
32.4%
31.2%

30.3%
28.6%
32.0%
31.2%

2.6%
2.1%
2.7%
2.4%

0.7822

Northeast
Midwest
South
West

289
459
744
456

70
159
205
127

24.2%
34.6%
27.6%
27.9%

27.2%
36.0%
28.4%
29.6%

3.0%
2.5%
1.9%
2.8%

0.0963

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

518
341
573
516

136
103
188
134

26.3%
30.2%
32.8%
26.0%

29.6%
28.8%
33.7%
28.0%

2.6%
2.8%
2.2%
2.3%

0.3275

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

86
157
654
501
247
303

32
46
192
155
57
79

37.2%
29.3%
29.4%
30.9%
23.1%
26.1%

40.3%
30.4%
29.7%
32.0%
23.6%
25.8%

6.4%
4.1%
2.0%
2.1%
2.9%
2.7%

0.0656

1,804
28
27
89

522
10
6
23

28.9%
35.7%
22.2%
25.8%

29.9%
36.1%
33.8%
33.0%

1.2%
13.0%
12.1%
6.8%

0.9161

Domain
Special district
Not special district
Charter
Non-charter

Regular
Special education
Vocational
Alternative

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-12

School Questionnaire Experience:
Detailed Tables

Table C-9.

C

Percentage of school respondents that received phase 1 follow-up by school
domain, among non-priority schools without a survey coordinator
Weighted percent
Weighted
percent
coordinator
follow-up
Standard
mailout
error
Pvalue
43.6%
8.6%
0.1176
57.6%
1.2%

Number of
completes
52
1,896

Number of
completes
with
coordinator
follow-up
mailout
28
1,111

Unweighted
percent
Phase 1 field
follow-up
53.8%
58.6%

274
1,674

174
965

63.5%
57.6%

63.3%
56.8%

3.2%
1.3%

0.0701

Primary
Middle
High
Combined

820
326
517
285

460
205
312
162

56.1%
62.9%
60.3%
56.8%

55.6%
63.1%
57.4%
56.6%

1.8%
2.9%
2.6%
3.5%

0.1735

City
Suburban
Town
Rural

514
571
327
536

317
352
178
292

61.7%
61.6%
54.4%
54.5%

58.2%
60.7%
52.8%
54.6%

2.6%
2.3%
2.9%
2.7%

0.1484

Northeast
Midwest
South
West

289
459
744
456

197
251
416
275

68.2%
54.7%
55.9%
60.3%

65.2%
52.3%
55.4%
59.6%

3.1%
2.6%
2.0%
2.9%

0.0179

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

518
341
573
516

325
188
320
306

62.7%
55.1%
55.8%
59.3%

60.5%
56.9%
54.0%
57.9%

2.5%
3.0%
2.6%
2.4%

0.3315

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

86
157
654
501
247
303

44
84
382
284
164
181

51.2%
53.5%
58.4%
56.7%
66.4%
59.7%

48.7%
52.9%
58.5%
55.9%
65.4%
59.5%

6.5%
4.6%
2.1%
2.3%
3.5%
3.1%

0.1098

1,804
28
27
89

1,048
17
18
56

58.1%
60.7%
66.7%
62.9%

57.1%
61.3%
57.5%
58.7%

1.3%
13.4%
12.0%
6.9%

0.9866

Domain
Special district
Not special district
Charter
Non-charter

Regular
Special education
Vocational
Alternative

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-13

School Questionnaire Experience:
Detailed Tables

Table C-10.

C

Percentage of school respondents that received further follow-up after phase 1 by
school domain, among non-priority schools without a survey coordinator
Weighted percent
Weighted
percent
further
follow-up
after
Standard
phase 1
error
11.0%
4.6%
12.5%
0.9%

Number of
completes
52
1,896

Number of
completes
with further
follow-up
after
phase 1
7
241

Unweighted
percent
further
follow-up
after
phase 1
13.5%
12.7%

274
1,674

39
209

14.2%
12.5%

14.6%
12.3%

2.3%
1.0%

0.3446

Primary
Middle
High
Combined

820
326
517
285

112
36
63
37

13.7%
11.0%
12.2%
13.0%

13.0%
11.3%
11.9%
13.5%

1.3%
1.9%
1.7%
2.4%

0.8319

City
Suburban
Town
Rural

514
571
327
536

59
69
43
77

11.5%
12.1%
13.1%
14.4%

11.5%
10.7%
15.3%
14.2%

1.5%
1.4%
2.3%
2.1%

0.2369

Northeast
Midwest
South
West

289
459
744
456

22
49
123
54

7.6%
10.7%
16.5%
11.8%

7.6%
11.7%
16.3%
10.8%

1.9%
1.7%
1.5%
1.9%

0.0073

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

518
341
573
516

57
50
65
76

11.0%
14.7%
11.3%
14.7%

9.9%
14.3%
12.3%
14.1%

1.3%
2.2%
1.9%
1.7%

0.2319

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

86
157
654
501
247
303

10
27
80
62
26
43

11.6%
17.2%
12.2%
12.4%
10.5%
14.2%

11.1%
16.7%
11.8%
12.1%
11.0%
14.7%

5.2%
3.4%
1.3%
1.5%
2.2%
2.2%

0.7090

1,804
28
27
89

234
1
3
10

13.0%
3.6%
11.1%
11.2%

13.1%
2.6%
8.7%
8.3%

1.0%
2.7%
6.4%
3.3%

0.2758

Domain
Special district
Not special district
Charter
Non-charter

Regular
Special education
Vocational
Alternative

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-14

Pvalue
0.7566

School Questionnaire Experience:
Detailed Tables

C

In summary, school response rates are significantly lower for schools in special districts, high
schools, city schools, suburban schools, schools in the Northeast region, low or high poverty
schools, and larger schools. School response rates are significantly higher for combined schools,
town and rural schools, Midwest schools, and schools with enrollment 100-199.
For city schools, suburban schools, Northeastern schools, low poverty schools, and high poverty
schools, this lower cooperativeness also translated into a need for more field activity to capture the
respondents that were gained. For charter schools, city schools, and high poverty schools, this
reduced cooperativeness showed up in lower rates for no or limited follow-up and high rates for
extensive field follow-up.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

C-15

Appendix D
Teacher Questionnaire Response Rates:
Detailed Tables

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

D-1

Teacher Questionnaire Response Rates:
Detailed Tables

D

Table D-1 presents eligibility and response rates for NTPS 2015-16 teacher questionnaires by
teacher domain. Rao-Scott Chi-Square tests are conducted to detect any differences. While there is
no significant difference in response rate for full-time teachers and for part-time teachers, teachers
with a missing status on the teacher listing form (mainly from the alternative TLF sources) have a
lower response rate. The response rates are lower for English teachers in primary schools, general
teachers in middle schools, and teachers with a missing subject or other subject in high or combined
schools.
The eligibility and response rates by teacher wave group are presented for teacher domains in Tables
D-2 through D-4. The response rates for teacher domains with a statistically significant difference in
response rate are in bold and italic in Tables D-1 through D-4.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

D-2

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-1.

NTPS 2015-16 teacher eligibility and response rates, teacher domains
Number of

Weighted

D-3

Response
rate
69.5%
76.8%

Eligibility
rate
94.9%
92.4%

Response
rate
69.1%
76.9%

Std.
error
4.4%
1.2%

Lower
bound
CI
60.3%
74.6%

Upper
bound
CI
77.9%
79.2%

10,836
567
245
933
117
3,325

7,922
441
184
599
85
2,258

96.0%
96.8%
97.2%
93.2%
95.9%
89.8%

73.1%
77.8%
75.1%
64.2%
72.6%
67.9%

96.2%
96.4%
97.1%
93.0%
97.2%
89.7%

73.1%
80.4%
77.0%
64.3%
71.6%
68.2%

0.7%
1.9%
3.1%
1.8%
4.9%
0.9%

71.8%
76.6%
70.9%
60.8%
62.0%
66.4%

74.3%
84.1%
83.1%
67.8%
81.3%
70.0%

157
1,152

152
1,104

89
848

96.8%
95.8%

58.6%
76.8%

96.5%
95.8%

62.5%
76.7%

8.1%
1.4%

46.6%
73.8%

78.4%
79.5%

426
1,191
917
1,812
778
208
2,839

403
1,166
895
1,727
765
204
2,619

224
906
674
1,186
592
165
1,648

94.6%
97.9%
97.6%
95.3%
98.3%
98.1%
92.3%

55.6%
77.7%
75.3%
68.7%
77.4%
80.9%
62.9%

94.7%
98.0%
97.5%
95.3%
98.4%
97.9%
92.3%

55.4%
77.6%
75.4%
68.7%
77.4%
80.5%
62.6%

3.0%
1.2%
1.5%
1.2%
1.5%
2.8%
1.3%

49.5%
75.3%
72.5%
66.3%
74.5%
75.0%
60.1%

61.4%
80.0%
78.4%
71.2%
80.4%
86.0%
65.1%

273
1,663
1,933
1,772
2,425
1,602
1,092
4,361

250
1,572
1,870
1,717
2,334
1,561
1,057
4,034

152
1,169
1,303
1,171
1,510
1,066
817
2,498

91.6%
94.5%
96.7%
96.9%
96.2%
97.4%
96.8%
92.5%

60.8%
74.4%
69.7%
68.2%
64.7%
68.3%
77.3%
61.9%

92.0%
94.6%
96.7%
96.6%
96.1%
97.4%
96.7%
92.5%

60.7%
73.8%
69.0%
67.3%
63.4%
68.4%
76.5%
60.5%

4.8%
1.4%
1.3%
1.3%
1.3%
1.5%
1.5%
1.2%

51.3%
71.0%
66.4%
64.8%
60.8%
65.4%
73.6%
58.1%

70.1%
76.5%
71.6%
69.9%
66.0%
71.4%
79.5%
62.9%

Eligible
teachers
177
1,860

11,285
586
252
1,001
122
3,703

Middle - Missing
Middle - Special ed
Middle - General
elementary
Middle - Math
Middle - Science
Middle - English
Middle - Social
Middle - Vo/Tech
Middle - Other
High - Missing
High - Special ed
High - Math
High - Science
High - English
High - Social
High - Vo/Tech
High - Other

Teacher Questionnaire Response Rates:
Detailed Tables

Completes
123
1,429

Eligibility
rate
95.2%
92.4%

Sample
size
186
2,014

Domain
Primary - Missing
Primary - Special ed
Primary - General
elementary
Primary - Math
Primary - Science
Primary - English
Primary - Social
Primary - Other

Unweighted

D

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-1.

NTPS 2015-16 teacher eligibility and response rates, teacher domains (continued)
Number of

Domain
Combined - Missing
Combined - Special ed
Combined - General
elementary
Combined - Math
Combined - Science
Combined - English
Combined - Social
Combined - Vo/Tech
Combined - Other

D-4

Full-time teachers
Part-time teachers
Status missing

Unweighted

Weighted

Completes
80
615

Eligibility
rate
96.5%
94.0%

Response
rate
57.6%
80.1%

Eligibility
rate
96.8%
93.4%

Response
rate
55.8%
79.7%

Std.
error
5.9%
2.2%

Lower
bound
CI
44.1%
75.3%

Upper
bound
CI
67.5%
84.1%

940
503
399
579
349
214
1,077

739
387
303
422
264
174
710

95.3%
96.5%
96.6%
95.9%
98.6%
98.6%
91.2%

78.6%
76.9%
75.9%
72.9%
75.6%
81.3%
65.9%

95.2%
96.7%
96.8%
95.8%
98.7%
98.9%
91.1%

77.8%
77.5%
76.9%
73.7%
76.0%
83.8%
65.6%

1.6%
1.8%
2.4%
2.2%
2.6%
2.9%
2.0%

74.6%
73.9%
72.3%
69.5%
70.8%
78.1%
61.7%

81.1%
81.2%
81.5%
78.0%
81.1%
89.6%
69.6%

30,601
2,182
13,675

24,568
1,732
6,453

97.0%
92.0%
90.7%

80.3%
79.4%
47.2%

97.0%
91.8%
90.8%

80.6%
80.2%
47.0%

0.4%
1.1%
0.7%

79.8%
78.1%
45.6%

81.4%
82.3%
48.5%

Sample
size
144
817

Eligible
teachers
139
768

986
521
413
604
354
217
1,181
31,544
2,373
15,070

Teacher Questionnaire Response Rates:
Detailed Tables

D

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-2.

NTPS 2015-16 teacher eligibility and response rates for teacher early wave group, teacher domains

Domain
Primary - Missing
Primary - Special ed
Primary - General
elementary
Primary - Math
Primary - Science
Primary - English
Primary - Social
Primary - Other

Sample
size
43
472
2,690

Number of
Eligible
teachers
Completes
41
33
435
394
2,630
2,388

Unweighted
Eligibility
Response
rate
rate
95.3%
80.5%
92.2%
90.6%
97.8%
90.8%

Eligibility
rate
95.1%
92.2%
98.0%

Response
rate
81.7%
90.7%
90.9%

Weighted
Std.
error
5.8%
1.6%
0.9%

Lower
bound CI
70.2%
87.5%
89.1%

Upper
bound CI
93.2%
93.8%
92.6%

D-5

141
55
168
23
684

137
54
153
23
639

97.2%
100.0%
96.6%
95.8%
93.8%

97.2%
98.2%
91.1%
100.0%
93.4%

97.0%
100.0%
96.5%
98.2%
93.5%

97.4%
97.4%
91.7%
100.0%
93.3%

1.4%
2.6%
2.2%
0.0%
1.0%

94.7%
92.2%
87.3%
100.0%
91.3%

100.0%
100.0%
96.1%
100.0%
95.3%

Middle - Missing
Middle - Special ed
Middle - General
elementary
Middle - Math
Middle - Science
Middle - English
Middle - Social
Middle - Vo/Tech
Middle - Other

23
309
44

23
300
44

22
276
41

100.0%
97.1%
100.0%

95.7%
92.0%
93.2%

100.0%
97.2%
100.0%

96.1%
91.9%
92.8%

4.4%
1.8%
4.3%

87.5%
88.5%
84.4%

100.0%
95.4%
100.0%

348
271
434
221
85
526

346
268
427
217
82
505

313
243
379
205
78
455

99.4%
98.9%
98.4%
98.2%
96.5%
96.0%

90.5%
90.7%
88.8%
94.5%
95.1%
90.1%

99.4%
99.0%
98.4%
98.5%
96.1%
96.0%

90.3%
90.8%
89.7%
94.4%
94.8%
90.4%

1.8%
1.8%
2.0%
1.6%
2.6%
1.6%

86.8%
87.2%
85.9%
91.2%
89.7%
87.2%

93.8%
94.4%
93.6%
97.6%
99.8%
93.6%

High - Missing
High - Special ed
High - Math
High - Science
High - English
High - Social
High - Vo/Tech
High - Other

17
391
435
410
491
381
330
836

15
383
432
405
489
379
322
794

15
353
379
374
433
333
287
680

88.2%
98.0%
99.3%
98.8%
99.6%
99.5%
97.6%
95.0%

100.0%
92.2%
87.7%
92.3%
88.5%
87.9%
89.1%
85.6%

88.7%
97.9%
99.6%
98.9%
99.5%
99.4%
97.5%
94.8%

100.0%
91.9%
87.1%
92.2%
88.0%
87.7%
89.5%
85.5%

0.0%
1.6%
2.0%
1.6%
1.6%
1.8%
2.0%
1.5%

100.0%
88.8%
83.2%
89.2%
85.0%
84.3%
85.6%
82.5%

100.0%
95.1%
91.0%
95.3%
91.1%
91.2%
93.4%
88.5%

Teacher Questionnaire Response Rates:
Detailed Tables

145
55
174
24
729

D

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-2.

NTPS 2015-16 teacher eligibility and response rates for teacher early wave group, teacher domains (continued)

Domain
Combined - Missing
Combined - Special ed
Combined - General
elementary
Combined - Math
Combined - Science
Combined - English
Combined - Social
Combined - Vo/Tech
Combined - Other

D-6

Full-time teachers
Part-time teachers
Status missing

Sample
size
15
225
243

Number of
Eligible
teachers
Completes
14
13
213
201
231
219

Unweighted
Eligibility
Response
rate
rate
93.3%
92.9%
94.7%
94.4%
95.1%
94.8%

Eligibility
rate
92.8%
93.8%
95.1%

Response
rate
92.3%
93.3%
94.8%

Weighted
Std.
error
8.0%
3.0%
1.9%

Lower
bound CI
76.5%
87.3%
91.1%

Upper
bound CI
100.0%
99.2%
98.5%

136
113
127
96
63
251

134
108
126
95
61
233

124
98
118
81
56
193

98.5%
95.6%
99.2%
99.0%
96.8%
92.8%

92.5%
90.7%
93.7%
85.3%
91.8%
82.8%

98.7%
95.6%
99.2%
99.1%
97.4%
92.0%

93.3%
91.9%
94.3%
85.9%
92.2%
81.8%

2.1%
2.5%
2.4%
3.5%
3.6%
4.0%

89.1%
87.0%
89.6%
78.9%
85.0%
74.0%

97.5%
96.9%
98.9%
92.9%
99.4%
89.7%

10,123
743
287

9,854
691
278

8,928
616
246

97.3%
93.0%
96.9%

90.6%
89.1%
88.5%

97.4%
92.4%
97.0%

90.7%
90.0%
88.3%

0.5%
1.5%
4.0%

89.6%
87.1%
80.4%

91.7%
92.8%
96.1%

Teacher Questionnaire Response Rates:
Detailed Tables

D

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-3.

NTPS 2015-16 teacher eligibility and response rates for middle teacher wave group, teacher domains

Domain
Primary - Missing
Primary - Special ed
Primary - General
elementary
Primary - Math
Primary - Science
Primary - English
Primary - Social
Primary - Other

D-7

High - Missing
High - Special ed
High - Math
High - Science
High - English
High - Social
High - Vo/Tech
High - Other

Number of
Eligible
teachers
Completes
47
40
622
534
3,419
2,879

Unweighted
Eligibility
Response
rate
rate
90.4%
85.1%
92.7%
85.9%
97.2%
84.2%

Eligibility
rate
90.1%
92.6%
97.4%

Response
rate
84.0%
86.4%
84.6%

Weighted
Std.
error
6.3%
1.5%
0.8%

Lower
bound CI
71.5%
83.5%
82.9%

Upper
bound CI
96.5%
89.4%
86.2%

201
86
219
47
987

197
84
213
47
904

157
68
169
35
736

98.0%
97.7%
97.3%
100.0%
91.6%

79.7%
81.0%
79.3%
74.5%
81.4%

98.5%
98.0%
97.5%
100.0%
91.7%

82.6%
82.2%
81.2%
70.3%
81.9%

2.9%
4.7%
2.9%
8.0%
1.5%

76.8%
73.0%
75.5%
54.5%
79.0%

88.3%
91.4%
86.9%
86.1%
84.7%

51
392
56

48
378
53

36
304
43

94.1%
96.4%
94.6%

75.0%
80.4%
81.1%

93.5%
96.4%
95.6%

74.7%
80.4%
82.7%

9.0%
2.7%
5.7%

56.9%
75.2%
71.4%

92.5%
85.7%
94.0%

430
329
503
282
81
632

419
325
488
279
81
599

348
254
392
222
60
473

97.4%
98.8%
97.0%
98.9%
100.0%
94.8%

83.1%
78.2%
80.3%
79.6%
74.1%
79.0%

97.7%
98.7%
97.1%
99.0%
100.0%
94.9%

83.6%
78.1%
80.7%
79.6%
73.3%
78.6%

1.7%
2.5%
1.9%
2.5%
5.2%
2.2%

80.2%
73.3%
77.0%
74.6%
63.1%
74.3%

87.0%
83.0%
84.4%
84.5%
83.5%
83.0%

63
443
527
447
551
451
397
1,005

51
423
521
442
541
444
388
949

39
356
415
349
429
343
308
734

81.0%
95.5%
98.9%
98.9%
98.2%
98.4%
97.7%
94.4%

76.5%
84.2%
79.7%
79.0%
79.3%
77.3%
79.4%
77.3%

82.3%
95.7%
98.8%
98.1%
98.4%
98.5%
97.8%
94.8%

77.2%
83.7%
79.9%
79.4%
79.5%
78.5%
79.0%
77.0%

10.1%
1.9%
1.9%
2.1%
1.9%
2.4%
2.6%
1.9%

57.3%
80.0%
76.1%
75.2%
75.7%
73.7%
73.8%
73.3%

97.1%
87.4%
83.6%
83.5%
83.3%
83.3%
84.3%
80.8%

Teacher Questionnaire Response Rates:
Detailed Tables

Middle - Missing
Middle - Special ed
Middle - General
elementary
Middle - Math
Middle - Science
Middle - English
Middle - Social
Middle - Vo/Tech
Middle - Other

Sample
size
52
671
3,516

D

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-3.

NTPS 2015-16 teacher eligibility and response rates for middle teacher wave group, teacher domains (continued)

Domain
Combined - Missing
Combined - Special ed
Combined - General
elementary
Combined - Math
Combined - Science
Combined - English
Combined - Social
Combined - Vo/Tech
Combined - Other

D-8

Full-time teachers
Part-time teachers
Status missing

Sample
size
32
257
299

Number of
Eligible
teachers
Completes
31
20
244
214
287
253

Unweighted
Eligibility
Response
rate
rate
96.9%
64.5%
94.9%
87.7%
96.0%
88.2%

Eligibility
rate
97.7%
93.8%
95.5%

Response
rate
55.7%
87.3%
88.6%

Weighted
Std.
error
11.1%
2.3%
2.0%

Lower
bound CI
33.7%
82.6%
84.5%

Upper
bound CI
77.6%
91.9%
92.6%

172
133
177
118
97
318

164
132
175
116
97
297

133
114
139
95
82
238

95.3%
99.2%
98.9%
98.3%
100.0%
93.4%

81.1%
86.4%
79.4%
81.9%
84.5%
80.1%

95.6%
99.2%
98.9%
98.4%
100.0%
93.1%

82.0%
88.3%
80.9%
82.7%
87.1%
79.7%

3.0%
2.7%
3.2%
3.8%
4.2%
2.9%

76.0%
83.0%
74.6%
75.2%
78.8%
74.0%

88.0%
93.6%
87.1%
90.2%
95.4%
85.3%

12,657
1,004
361

12,254
913
338

10,016
723
272

96.8%
90.9%
93.6%

81.7%
79.2%
80.5%

96.8%
90.9%
93.5%

82.2%
80.3%
81.1%

0.6%
1.7%
3.2%

81.0%
77.0%
74.7%

83.4%
83.6%
87.5%

Teacher Questionnaire Response Rates:
Detailed Tables

D

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-4.

NTPS 2015-16 teacher eligibility and response rates for late teacher wave group, teacher domains

Domain
Primary - Missing
Primary - Special ed
Primary - General
elementary
Primary - Math
Primary - Science
Primary - English
Primary - Social
Primary - Other

Sample
size
91
871
5,079

Number of
Eligible
teachers
Completes
89
50
803
501
4,787
2,655

Unweighted
Eligibility
Response
rate
rate
97.8%
56.2%
92.2%
62.4%
94.3%
55.5%

Eligibility
rate
97.7%
92.3%
94.5%

Response
rate
54.9%
62.4%
55.5%

Weighted
Std.
error
5.9%
2.0%
1.0%

Lower
bound CI
43.3%
58.4%
53.4%

Upper
bound CI
66.6%
66.4%
57.6%

240
111
608
51
1,987

229
106
552
47
1,737

147
62
277
27
883

95.4%
95.5%
90.8%
92.2%
87.4%

64.2%
58.5%
50.2%
57.4%
50.8%

94.1%
95.0%
90.5%
93.9%
87.3%

66.6%
62.4%
49.9%
60.3%
51.6%

3.6%
5.5%
2.5%
8.1%
1.4%

59.5%
51.6%
45.0%
44.3%
48.9%

73.7%
73.3%
54.9%
76.4%
54.3%

D-9

83
451
326

81
426
306

31
268
140

97.6%
94.5%
93.9%

38.3%
62.9%
45.8%

97.6%
94.2%
93.8%

44.0%
62.9%
45.7%

10.0%
2.5%
3.1%

24.4%
58.0%
39.6%

63.7%
67.8%
51.8%

413
317
875
275
42
1,681

401
302
812
269
41
1,515

245
177
415
165
27
720

97.1%
95.3%
92.8%
97.8%
97.6%
90.1%

61.1%
58.6%
51.1%
61.3%
65.9%
47.5%

97.1%
95.1%
92.8%
97.6%
97.5%
90.3%

60.9%
59.0%
51.4%
61.5%
66.7%
47.6%

2.6%
3.0%
1.8%
3.1%
7.3%
1.7%

55.8%
53.1%
47.8%
55.3%
52.3%
44.3%

66.0%
64.9%
55.0%
67.7%
81.1%
50.9%

High - Missing
High - Special ed
High - Math
High - Science
High - English
High - Social
High - Vo/Tech
High - Other

193
829
971
915
1,383
770
365
2,520

184
766
917
870
1,304
738
347
2,291

98
460
509
448
648
390
222
1,084

95.3%
92.4%
94.4%
95.1%
94.3%
95.8%
95.1%
90.9%

53.3%
60.1%
55.5%
51.5%
49.7%
52.8%
64.0%
47.3%

95.1%
92.5%
94.5%
95.0%
94.0%
95.8%
94.9%
90.9%

53.1%
59.8%
55.3%
51.0%
48.5%
53.3%
62.3%
46.0%

5.3%
2.2%
2.0%
1.8%
1.8%
2.1%
2.7%
1.5%

42.6%
55.5%
51.4%
47.5%
45.0%
49.2%
56.9%
43.0%

63.6%
64.2%
59.2%
54.6%
52.1%
57.3%
67.7%
48.9%

Teacher Questionnaire Response Rates:
Detailed Tables

Middle - Missing
Middle - Special ed
Middle - General
elementary
Middle - Math
Middle - Science
Middle - English
Middle - Social
Middle - Vo/Tech
Middle - Other

D

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-4.

NTPS 2015-16 teacher eligibility and response rates for late teacher wave group, teacher domains (continued)

Domain
Combined - Missing
Combined - Special ed
Combined - General
elementary
Combined - Math
Combined - Science
Combined - English
Combined - Social
Combined - Vo/Tech
Combined - Other

D-10

Full-time teachers
Part-time teachers
Status missing

Sample
size
97
335
444

Number of
Eligible
teachers
Completes
94
47
311
200
422
267

Unweighted
Eligibility
Response
rate
rate
96.9%
50.0%
92.8%
64.3%
95.0%
63.3%

Eligibility
rate
97.1%
92.8%
95.0%

Response
rate
50.7%
65.7%
62.8%

Weighted
Std.
error
6.3%
3.8%
2.6%

Lower
bound CI
38.2%
58.3%
57.7%

Upper
bound CI
63.2%
73.1%
68.0%

213
167
300
140
57
612

205
159
278
138
56
547

130
91
165
88
36
279

96.2%
95.2%
92.7%
98.6%
98.2%
89.4%

63.4%
57.2%
59.4%
63.8%
64.3%
51.0%

96.3%
95.7%
92.5%
98.7%
98.4%
89.7%

64.3%
57.5%
60.1%
63.6%
68.0%
51.2%

3.5%
4.1%
3.7%
4.6%
6.5%
2.9%

57.4%
49.3%
52.8%
54.5%
55.3%
45.4%

71.1%
65.7%
67.5%
72.8%
80.7%
56.9%

8,764
626
14,422

8,493
578
13,059

5,624
393
5,935

96.9%
92.3%
90.5%

66.2%
68.0%
45.4%

96.9%
92.4%
90.6%

66.7%
68.1%
45.5%

0.9%
2.4%
0.7%

65.0%
63.4%
44.0%

68.4%
72.8%
46.9%

Teacher Questionnaire Response Rates:
Detailed Tables

D

Teacher Questionnaire Response Rates:
Detailed Tables

D

Tables D-5A and D-5B compare the weighted conditional teacher response rates of two
experimental groups for the major school domains and teacher domains respectively. There are
three domains with a significant difference in response rate between two experimental groups, with
higher response rates for teachers in the experimental groups: teachers from schools in towns,
primary school teachers whose major subject is other and high school teachers whose main subject
is vocational/technical (bold in Tables D-5A and D-5B).

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

D-11

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-5A.

NTPS 2015-16 teacher eligibility and response rates by nonresponse follow-up experimental group, major school domains

Domain
All

Teacher experimental group 1 special afternoon call
Number of
Weighted
Sample
Eligible
Response
Std
size
teachers
Completes
rate
error
9,560
9,215
5,124
55.5%
0.7%

Teacher experimental group 2 regular call schedule
Number of
Weighted
Sample
Eligible
Response
Std
size
teachers
Completes
rate
error
9,439
9,104
5,004
54.7%
0.7%

D-12

2,051
7,509

1,978
7,237

952
4,172

47.9%
57.6%

1.5%
0.7%

2,038
7,401

1,959
7,145

959
4,045

48.9%
56.3%

1.5%
0.7%

Charter
Non-charter

1,217
8,343

1,162
8,053

662
4,462

56.7%
55.4%

1.6%
0.7%

1,134
8,305

1,074
8,030

594
4,410

55.8%
54.6%

1.8%
0.7%

Primary
Middle
High
Combined

3,721
1,876
2,988
975

3,553
1,799
2,920
943

2,074
956
1,524
570

58.6%
53.0%
51.1%
60.6%

1.0%
1.5%
1.3%
2.0%

3,601
1,834
3,038
966

3,461
1,776
2,946
921

1,975
953
1,502
574

57.3%
53.8%
49.9%
62.6%

1.1%
1.5%
1.3%
1.9%

City
Suburban
Town
Rural

3,288
3,258
1,165
1,849

3,163
3,151
1,113
1,788

1,631
1,660
730
1,103

51.1%
52.6%
66.7%
62.3%

1.2%
1.3%
1.7%
1.6%

3,227
3,247
1,120
1,845

3,098
3,141
1,072
1,793

1,638
1,620
670
1,076

52.5%
51.7%
62.7%
60.0%

1.1%
1.2%
2.0%
1.5%

Northeast
Midwest
South
West

1,913
1,901
3,634
2,112

1,853
1,821
3,498
2,043

935
1,129
1,968
1,092

50.2%
61.3%
56.4%
53.6%

1.5%
1.5%
1.0%
1.4%

1,909
1,866
3,605
2,059

1,838
1,786
3,482
1,998

928
1,085
1,922
1,069

50.5%
60.1%
54.8%
53.7%

1.6%
1.5%
1.1%
1.4%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

2,750
1,601
2,551
2,658

2,678
1,548
2,446
2,543

1,430
881
1,410
1,403

53.6%
56.4%
57.1%
55.4%

1.3%
1.6%
1.2%
1.3%

2,720
1,568
2,516
2,635

2,630
1,517
2,421
2,536

1,367
841
1,395
1,401

51.8%
55.2%
56.8%
55.5%

1.3%
1.4%
1.2%
1.3%

Teacher Questionnaire Response Rates:
Detailed Tables

Special district
Not special district

D

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-5A.

NTPS 2015-16 teacher eligibility and response rates by nonresponse follow-up experimental group, major school domains
(continued)

Domain
Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+
Regular
Special education
Vocational
Alternative

Teacher experimental group 1 special afternoon call
Number of
Weighted
Sample
Eligible
Response
Std
size
teachers
Completes
rate
error
259
249
150
57.0%
4.8%
432
419
252
63.2%
3.0%
2,702
2,589
1,518
58.5%
1.1%
2,481
2,383
1,368
57.6%
1.4%
1,364
1,313
718
54.9%
1.8%
2,322
2,262
1,118
48.6%
1.4%
9,073
143
101
243

8,748
135
97
235

4,864
82
53
125

55.6%
59.6%
57.9%
48.1%

0.7%
6.5%
4.8%
4.0%

Teacher experimental group 2 regular call schedule
Number of
Weighted
Sample
Eligible
Response
Std
size
teachers
Completes
rate
error
241
228
131
53.3%
3.4%
421
403
247
66.1%
2.9%
2,665
2,562
1,460
56.5%
1.2%
2,432
2,334
1,323
57.0%
1.3%
1,363
1,313
714
54.4%
1.7%
2,317
2,264
1,129
48.9%
1.4%
8,926
129
112
272

8,622
115
105
262

4,735
72
52
145

54.8%
63.6%
47.1%
52.9%

0.7%
9.8%
4.4%
3.1%

D-13
Teacher Questionnaire Response Rates:
Detailed Tables

D

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-5B.

NTPS 2015-16 teacher eligibility and response rates by nonresponse follow-up experimental group, teacher domains

Domain
Primary - Missing
Primary - Special ed
Primary - General
Primary - Math
Primary - Science
Primary - English
Primary - Social
Primary - Other

Teacher experimental group 1 special afternoon call
Number of
Weighted
Sample
Eligible
Response
Std
size
teachers
Completes
rate
error
40
38
22
56.2%
8.1%
347
320
190
59.6%
3.0%
2,152
2,077
1,222
58.8%
1.3%
120
117
77
70.3%
4.6%
59
58
37
68.4%
6.4%
221
213
108
50.5%
3.3%
30
28
14
47.9%
9.6%
752
702
404
58.1%
1.9%

Teacher experimental group 2 regular call schedule
Number of
Weighted
Sample
Eligible
Response
Std
size
teachers
Completes
rate
error
39
38
20
53.5%
8.8%
376
358
207
58.6%
3.2%
2,121
2,057
1,223
59.8%
1.2%
104
102
61
61.5%
4.8%
48
47
28
62.4%
7.8%
184
175
90
52.8%
3.9%
17
17
11
61.9%
12.5%
712
667
335
49.5%
2.1%

D-14

26
209
85
267
181
349
156
31
572

25
195
84
259
178
336
153
30
539

11
106
39
157
96
169
94
13
271

44.8%
52.6%
47.5%
60.3%
54.2%
51.2%
62.0%
42.9%
49.8%

8.3%
4.0%
6.4%
2.9%
4.1%
2.8%
3.8%
8.8%
2.5%

24
219
81
204
185
354
160
36
571

23
213
76
199
181
337
157
36
554

15
121
35
115
100
177
92
19
279

66.1%
57.7%
46.1%
57.2%
55.5%
52.8%
58.3%
52.5%
50.4%

14.0%
3.6%
5.9%
3.5%
3.6%
2.8%
3.6%
9.0%
2.5%

High - Missing
High - Special ed
High - Math
High - Science
High - English
High - Social
High - Vo/Tech
High - Other

72
275
387
329
493
304
238
890

68
269
380
329
483
301
231
859

38
141
217
164
237
161
144
422

57.9%
52.5%
56.9%
50.6%
47.2%
54.0%
62.3%
46.1%

7.1%
2.9%
2.7%
3.4%
2.6%
3.3%
3.6%
2.2%

64
288
377
369
453
349
216
922

57
276
372
361
440
343
209
888

29
158
193
208
212
169
109
424

50.2%
55.5%
52.4%
55.9%
46.5%
48.7%
50.6%
46.7%

8.4%
2.9%
2.9%
2.6%
2.6%
2.7%
4.2%
2.0%

Teacher Questionnaire Response Rates:
Detailed Tables

Middle - Missing
Middle - Special ed
Middle - General
Middle - Math
Middle - Science
Middle - English
Middle - Social
Middle - Vo/Tech
Middle - Other

D

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table D-5B.

NTPS 2015-16 teacher eligibility and response rates by nonresponse follow-up experimental group, teacher domains
(continued)

Domain
Combined - Missing
Combined - Special ed
Combined - General
elementary
Combined - Math
Combined - Science
Combined - English
Combined - Social
Combined - Vo/Tech
Combined - Other

D-15

Full-time teachers
Part-time teachers
Status missing
Primary - Missing

Teacher experimental group 1 special afternoon call
Number of
Weighted
Sample
Eligible
Response
Std
size
teachers
Completes
rate
error
40
39
19
45.8%
9.4%
141
133
96
73.6%
4.6%
167
163
109
64.3%
4.7%

Teacher experimental group 2 regular call schedule
Number of
Weighted
Sample
Eligible
Response
Std
size
teachers
Completes
rate
error
42
41
19
45.3%
8.5%
154
145
97
65.4%
5.2%
170
162
110
67.6%
4.1%

106
71
103
62
44
241

102
69
102
61
44
230

60
38
58
41
29
120

59.6%
57.2%
60.2%
67.7%
71.1%
49.7%

4.8%
6.2%
5.1%
6.3%
6.4%
3.7%

87
82
105
67
36
223

80
79
102
66
34
212

50
56
65
40
19
118

64.8%
72.5%
63.8%
63.6%
55.1%
56.4%

5.7%
5.1%
4.7%
5.9%
9.1%
3.9%

5,955
457
3,148
40

5,771
420
3,024
38

3,535
253
1,336
22

61.7%
61.4%
43.4%
56.2%

0.8%
2.6%
1.1%
8.1%

5,897
450
3,092
39

5,710
420
2,974
38

3,497
247
1,260
20

61.4%
58.2%
41.9%
53.5%

0.9%
2.5%
1.3%
8.8%
Teacher Questionnaire Response Rates:
Detailed Tables

D

Appendix E
Teacher Questionnaire Response Experience:
Detailed Tables

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

E-1

Teacher Questionnaire Response Experience:
Detailed Tables

E

Tables E-1 through E-3 show the percentages of teacher respondents who received no or limited
follow-up, telephone or mail follow-up, and field follow-up, respectively, before completing the
teacher questionnaire within a teacher wave group. The percentages are based on the teacher final
weights. Part A of each table covers school domains, and Part B covers teacher domains. Given a
follow-up experience group within a teacher wave group, the percentages for teacher domains with a
statistically significant difference are in bold and italic in Tables E-1 through E-3.
Across all teacher wave groups, fewer teachers from schools in the West region responded with no
or limited follow-up (before any of telephone, mail and field follow-up). Other than the West region
domain, the following domains are less likely to respond with no or limited follow-up and required
more extensive follow-up than the other types of schools:
 Early TWG: teachers in special districts and teachers from charter schools;
 Middle TWG: teachers from city schools, teachers from high poverty schools, and teachers
with a missing teaching status;
 Late TWG: teachers in special districts, teachers from charter schools, teachers from high
poverty schools, teachers from small schools, and teachers in special education schools.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

E-2

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-1A.

Percentage of teacher respondents with no/limited follow-up by school domain by teacher wave group, NTPS 2015-16

E-3

Early
Wave
Weighted
percent
58.2%
66.4%

Early
Wave
Standard
error
2.5%
0.9%

Middle Wave
Unweighted
respondent
count
1,304
9,707

Middle
Wave
Weighted
percent
67.0%
65.1%

Middle
Wave
Standard
error
2.4%
0.9%

Late Wave
Unweighted
respondent
count
2,289
9,663

Late
Wave
Weighted
percent
42.2%
53.3%

Late
Wave
Standard
error
1.8%
0.9%

Charter
Non-charter

1,006
8,784

58.5%
65.4%

2.7%
0.9%

1,128
9,883

63.3%
65.4%

2.5%
0.9%

1,304
10,648

43.7%
51.2%

2.1%
0.9%

Primary
Middle
High
Combined

3,821
2,012
2,854
1,103

66.1%
65.2%
63.4%
64.9%

1.2%
1.8%
1.7%
2.5%

4,618
2,132
2,973
1,288

65.4%
66.6%
64.9%
62.9%

1.2%
1.8%
1.9%
2.3%

4,602
2,188
3,859
1,303

50.8%
49.4%
51.5%
51.1%

1.3%
1.8%
1.6%
2.5%

City
Suburban
Town
Rural

2,027
3,306
1,758
2,699

62.2%
65.9%
63.5%
67.4%

1.7%
1.5%
2.1%
1.5%

2,772
3,690
1,777
2,772

61.6%
67.2%
67.7%
65.5%

1.8%
1.4%
1.8%
1.5%

4,165
3,808
1,536
2,443

48.3%
50.8%
55.8%
54.1%

1.3%
1.4%
2.0%
1.8%

Northeast
Midwest
South
West

1,499
2,794
3,770
1,727

67.9%
69.7%
63.9%
58.6%

2.4%
1.6%
1.3%
2.0%

1,867
2,782
4,185
2,177

65.5%
66.7%
66.9%
61.1%

2.1%
1.5%
1.2%
1.8%

2,156
2,570
4,747
2,479

46.8%
54.4%
51.6%
50.1%

1.9%
1.9%
1.2%
1.9%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

3,431
1,854
2,886
1,619

67.3%
67.1%
63.5%
60.8%

1.6%
2.1%
1.6%
2.0%

3,343
1,971
3,214
2,483

67.6%
67.8%
64.7%
61.0%

1.7%
1.8%
1.5%
1.6%

3,297
2,027
3,268
3,360

53.2%
53.4%
51.6%
46.3%

1.6%
2.0%
1.7%
1.4%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

204
477
2,965
2,594
1,264
2,286

63.9%
61.4%
65.7%
65.9%
67.1%
63.0%

5.1%
3.4%
1.3%
1.8%
2.6%
1.9%

290
568
3,302
3,152
1,621
2,078

55.4%
63.2%
64.8%
65.0%
65.8%
67.9%

5.9%
3.1%
1.4%
1.6%
2.0%
2.0%

256
535
3,337
3,040
1,724
3,060

32.6%
52.8%
50.7%
50.2%
51.0%
52.5%

4.3%
3.5%
1.5%
1.5%
2.1%
1.6%

Regular
Special education
Vocational
Alternative

9,428
108
92
162

65.1%
64.6%
77.3%
54.9%

0.8%
5.3%
7.3%
5.2%

10,514
149
130
218

65.7%
56.5%
50.7%
67.1%

0.9%
7.2%
6.0%
5.3%

11,389
117
119
327

51.2%
24.3%
54.5%
42.3%

0.8%
5.8%
6.6%
5.6%

No/Limited Follow-up
Special district
Not special district

Teacher Questionnaire Response Experience:
Detailed Tables

Early Wave
Unweighted
respondent
count
1,270
8,520

E

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-1B.

Percentage of teacher respondents with no/limited follow-up by teacher domain by teacher wave group, NTPS 2015-16

E-4

Early
Wave
Weighted
percent
65.5%
74.3%
65.0%
63.2%
66.2%
61.1%
84.8%
66.6%

Early
Wave
Standard
error
10.4%
2.8%
1.5%
4.8%
6.7%
4.4%
9.4%
2.0%

Middle
Wave
Unweighted
respondent
count
40
534
2,879
157
68
169
35
736

Middle
Wave
Weighted
percent
70.1%
69.7%
64.5%
60.5%
67.8%
65.3%
63.1%
66.4%

Middle
Wave
Standard
error
11.7%
2.3%
1.3%
4.2%
6.9%
4.1%
9.1%
2.0%

Late Wave
Unweighte
d
respondent
count
50
501
2,655
147
62
277
27
883

Late
Wave
Weighted
percent
50.0%
55.3%
49.7%
54.4%
38.7%
46.2%
48.1%
47.7%

Late
Wave
Standard
error
49.5%
57.9%
50.3%
55.4%
36.2%
47.4%
52.3%
49.8%

Middle - Missing
Middle - Special ed
Middle - General elem.
Middle - Math
Middle - Science
Middle - English
Middle - Social studies
Middle - Vo/Tech
Middle - Other

22
276
41
313
243
379
205
78
455

54.0%
68.6%
54.0%
69.6%
63.0%
67.7%
60.8%
75.8%
61.1%

22.4%
2.9%
13.1%
2.7%
2.9%
2.7%
3.8%
5.3%
2.9%

36
304
43
348
254
392
222
60
473

62.6%
66.6%
74.9%
65.7%
67.6%
64.9%
67.2%
76.6%
66.0%

15.2%
2.9%
7.3%
3.2%
3.5%
2.9%
3.2%
6.0%
2.6%

31
268
140
245
177
415
165
27
720

48.4%
53.0%
50.0%
46.1%
49.7%
50.8%
47.3%
44.4%
46.9%

40.4%
52.8%
49.1%
48.1%
50.2%
50.6%
46.4%
46.6%
48.7%

High - Missing
High - Special ed
High - Math
High - Science
High - English
High - Social studies
High - Vo/Tech
High - Other

15
353
379
374
433
333
287
680

85.3%
68.8%
66.9%
60.6%
65.5%
61.1%
58.5%
61.3%

7.3%
2.7%
2.8%
3.1%
2.3%
2.7%
3.0%
2.4%

39
356
415
349
429
343
308
734

52.7%
70.9%
65.5%
64.9%
63.9%
64.1%
65.7%
62.9%

11.0%
3.8%
2.3%
3.1%
2.9%
2.9%
3.2%
2.9%

98
460
509
448
648
390
222
1,084

44.9%
58.5%
47.9%
50.4%
52.3%
51.5%
53.2%
49.3%

41.0%
59.8%
46.9%
49.3%
53.1%
51.9%
52.8%
50.4%

No/ Limited Follow-up
Primary - Missing
Primary - Special ed
Primary - General elem.
Primary - Math
Primary - Science
Primary - English
Primary - Social studies
Primary - Other

Teacher Questionnaire Response Experience:
Detailed Tables

Early Wave
Unweighte
d
respondent
count
33
394
2,388
137
54
153
23
639

E

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-1B.

Percentage of teacher respondents with no/limited follow-up by teacher domain by teacher wave group, NTPS 2015-16
(continued)

No/ Limited Follow-up

Combined - Missing
Combined - Special ed
Combined - General elem.
Combined - Math
Combined - Science
Combined - English
Combined - Social studies
Combined - Vo/Tech
Combined - Other

E-5

Status missing
Full-time teachers
Part-time teachers

Early Wave
Unweighted
respondent
count

Early
Wave
Weighted
percent

Early
Wave
Standard
error

Middle Wave
Unweighted
respondent
count

Middle
Wave
Weighted
percent

Middle
Wave
Standard
error

Late Wave
Unweighted
respondent
count

Late
Wave
Weighted
percent

Late
Wave
Standard
error

13
201
219
124
98
118
81
56
193

64.7%
70.5%
67.3%
62.7%
63.9%
65.9%
62.2%
60.8%
59.8%

15.6%
3.7%
4.4%
5.9%
5.2%
4.7%
6.1%
10.5%
4.4%

20
214
253
133
114
139
95
82
238

54.5%
60.0%
62.4%
66.4%
64.5%
65.4%
66.5%
57.7%
63.8%

19.7%
5.7%
3.8%
4.8%
5.0%
4.1%
5.2%
13.1%
3.9%

47
200
267
130
91
165
88
36
279

36.2%
47.0%
53.2%
51.5%
51.6%
55.2%
50.0%
55.6%
49.1%

35.9%
45.1%
52.6%
53.9%
52.9%
55.9%
50.5%
60.4%
51.5%

246
8,928
616

59.0%
65.4%
62.9%

5.3%
0.8%
2.4%

609
10,419
754

65.9%
65.1%
68.5%

3.9%
0.9%
2.1%

5,935
5,624
393

49.1%
51.4%
46.6%

49.9%
51.9%
48.6%
Teacher Questionnaire Response Experience:
Detailed Tables

E

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-2A.

Percentage of teacher respondents with telephone/mail follow-up by school domain by teacher wave group, NTPS 2015-16

E-6

Early
Wave
Standard
error
2.0%
0.8%

Middle Wave
Unweighted
respondent
count
1,304
9,707

Middle
Wave
Weighted
percent
16.0%
17.1%

Middle
Wave
Standard
error
1.4%
0.7%

Late Wave
Unweighted
respondent
count
2,289
9,663

Late
Wave
Weighted
percent
12.2%
14.5%

Late
Wave
Standard
error
0.9%
0.4%

Charter
Non-charter

1,006
8,784

33.9%
30.6%

2.0%
0.8%

1,128
9,883

14.6%
17.1%

1.4%
0.7%

1,304
10,648

15.7%
13.9%

1.6%
0.4%

Primary
Middle
High
Combined

3,821
2,012
2,854
1,103

30.3%
31.1%
31.2%
31.0%

1.1%
1.6%
1.4%
2.2%

4,618
2,132
2,973
1,288

18.2%
14.8%
16.0%
17.0%

0.8%
1.0%
1.6%
1.8%

4,602
2,188
3,859
1,303

13.3%
15.4%
14.4%
13.2%

0.6%
1.0%
0.7%
1.6%

City
Suburban
Town
Rural

2,027
3,306
1,758
2,699

32.4%
30.5%
30.6%
29.6%

1.6%
1.2%
1.9%
1.4%

2,772
3,690
1,777
2,772

18.5%
16.1%
15.2%
17.6%

1.5%
0.9%
1.2%
1.0%

4,165
3,808
1,536
2,443

13.2%
15.0%
14.2%
13.5%

0.7%
0.7%
1.1%
0.8%

Northeast
Midwest
South
West

1,499
2,794
3,770
1,727

28.3%
26.9%
32.1%
35.5%

2.2%
1.4%
1.2%
1.7%

1,867
2,782
4,185
2,177

17.0%
16.2%
15.6%
20.0%

1.3%
0.9%
0.9%
1.9%

2,156
2,570
4,747
2,479

15.9%
14.8%
13.6%
12.2%

1.1%
0.9%
0.7%
0.9%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

3,431
1,854
2,886
1,619

29.4%
27.4%
32.5%
34.4%

1.5%
1.7%
1.5%
1.8%

3,343
1,971
3,214
2,483

16.7%
15.4%
17.3%
18.1%

1.3%
1.3%
1.0%
1.1%

3,297
2,027
3,268
3,360

15.0%
14.1%
13.0%
13.9%

0.8%
1.1%
0.8%
0.8%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

204
477
2,965
2,594
1,264
2,286

35.5%
35.6%
30.2%
30.3%
28.0%
32.4%

5.0%
3.3%
1.2%
1.6%
2.1%
1.7%

290
568
3,302
3,152
1,621
2,078

30.6%
17.0%
18.6%
16.9%
16.1%
13.8%

7.0%
2.3%
1.0%
1.0%
1.3%
1.1%

256
535
3,337
3,040
1,724
3,060

8.3%
12.9%
12.8%
14.1%
15.6%
14.6%

2.7%
1.7%
0.7%
0.8%
1.2%
0.8%

Regular
Special education
Vocational
Alternative

9,428
108
92
162

30.7%
35.4%
18.6%
38.5%

0.7%
5.3%
5.3%
5.5%

10,514
149
130
218

16.5%
30.2%
34.1%
14.4%

0.5%
6.6%
9.3%
3.3%

11,389
117
119
327

14.2%
12.1%
22.0%
6.5%

0.4%
3.4%
4.4%
2.1%

Telephone/Mail Followup
Special district
Not special district

E

Early
Wave
Weighted
percent
36.8%
29.6%

Teacher Questionnaire Response Experience:
Detailed Tables

Early Wave
Unweighted
respondent
count
1,270
8,520

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-2B.

Percentage of teacher respondents with telephone/mail follow-up by teacher domain by teacher wave group,
NTPS 2015-16

Telephone/Mail
Follow-up
Primary - Missing
Primary - Special ed
Primary - General elem.
Primary - Math
Primary - Science
Primary - English
Primary - Social studies
Primary - Other

Early Wave
Unweighted
respondent
count

Early
Wave
Weighted
percent

Early
Wave
Standard
error

Middle Wave
Unweighted
respondent
count

Middle
Wave
Weighted
percent

Middle
Wave
Standard
error

Late Wave
Unweighted
respondent
count

Late
Wave
Weighted
percent

Late
Wave
Standard
error

E-7

26.2%
24.2%
31.1%
32.7%
31.4%
33.7%
15.2%
30.3%

9.3%
2.6%
1.4%
4.6%
6.6%
4.1%
9.4%
2.0%

40
534
2,879
157
68
169
35
736

15.5%
17.1%
18.4%
16.4%
16.7%
17.3%
29.7%
18.2%

9.4%
1.8%
0.9%
2.8%
5.7%
3.2%
9.2%
1.6%

50
501
2,655
147
62
277
27
883

16.5%
8.9%
13.5%
14.3%
23.4%
17.6%
8.2%
13.1%

7.5%
1.4%
0.8%
3.3%
6.4%
2.6%
4.9%
1.4%

Middle - Missing
Middle - Special ed
Middle - General elem.
Middle - Math
Middle - Science
Middle - English
Middle - Social studies
Middle - Vo/Tech
Middle - Other

22
276
41
313
243
379
205
78
455

37.9%
31.1%
42.4%
26.7%
33.4%
28.2%
35.1%
17.7%
34.3%

27.5%
2.9%
13.5%
2.6%
2.9%
2.5%
3.7%
4.4%
2.5%

36
304
43
348
254
392
222
60
473

28.1%
16.0%
13.3%
12.1%
16.3%
16.0%
12.7%
9.8%
15.0%

9.3%
2.4%
4.8%
1.9%
2.7%
1.9%
2.4%
3.9%
1.8%

31
268
140
245
177
415
165
27
720

18.9%
15.0%
15.8%
14.1%
14.4%
14.5%
11.6%
24.4%
16.7%

8.4%
2.4%
3.3%
2.1%
2.7%
2.1%
2.5%
9.2%
1.6%

High - Missing
High - Special ed
High - Math
High - Science
High - English
High - Social studies
High - Vo/Tech
High - Other

15
353
379
374
433
333
287
680

14.7%
27.3%
29.0%
34.4%
29.7%
31.6%
34.8%
32.6%

7.3%
2.5%
2.7%
2.9%
2.3%
2.5%
2.9%
2.1%

39
356
415
349
429
343
308
734

9.8%
15.8%
14.8%
15.5%
16.8%
16.9%
13.6%
17.4%

4.9%
4.0%
1.7%
2.1%
2.3%
2.2%
2.0%
3.0%

98
460
509
448
648
390
222
1,084

20.1%
14.3%
12.7%
15.1%
16.1%
15.7%
12.4%
13.3%

6.6%
1.8%
1.6%
1.8%
1.7%
2.1%
2.5%
1.1%

Teacher Questionnaire Response Experience:
Detailed Tables

33
394
2,388
137
54
153
23
639

E

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-2B.

Percentage of teacher respondents with telephone/mail follow-up by teacher domain by teacher wave group,
NTPS 2015-16 (continued)

Telephone/Mail
Follow-up
Combined - Missing
Combined - Special ed
Combined - General elem.
Combined - Math
Combined - Science
Combined - English
Combined - Social studies
Combined - Vo/Tech
Combined - Other
Status missing
Full-time teachers
Part-time teachers

Early Wave
Unweighted
respondent
count

Early
Wave
Weighted
percent

Early
Wave
Standard
error

Middle Wave
Unweighted
respondent
count

Middle
Wave
Weighted
percent

Middle
Wave
Standard
error

Late Wave
Unweighted
respondent
count

Late
Wave
Weighted
percent

Late
Wave
Standard
error

13
201
219
124
98
118
81
56
193

35.3%
27.3%
27.3%
34.1%
31.1%
27.2%
34.2%
39.2%
34.9%

15.6%
3.4%
3.6%
5.7%
5.2%
4.4%
6.0%
10.5%
3.9%

20
214
253
133
114
139
95
82
238

4.9%
24.9%
15.4%
17.1%
10.7%
18.0%
18.3%
21.5%
12.0%

2.6%
5.8%
2.4%
4.1%
3.1%
3.6%
3.9%
11.6%
2.3%

47
200
267
130
91
165
88
36
279

15.2%
12.9%
16.9%
14.5%
9.2%
12.8%
8.7%
11.3%
12.3%

7.2%
2.8%
4.9%
3.6%
4.1%
3.5%
3.3%
5.0%
2.1%

246
8,928
616

33.9%
30.5%
32.7%

4.6%
0.7%
2.2%

609
10,419
754

17.8%
16.9%
17.0%

3.3%
0.7%
1.7%

5,935
5,624
393

15.9%
11.8%
14.9%

0.7%
0.5%
1.9%

E-8
Teacher Questionnaire Response Experience:
Detailed Tables

E

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-3A.

Percentage of teacher respondents with field follow-up by school domain by teacher wave group, NTPS 2015-16

E-9

Early
Wave
Standard
error
1.0%
0.4%

Middle Wave
Unweighted
respondent
count
1,304
9,707

Middle
Wave
Weighted
percent
17.1%
17.8%

Middle
Wave
Standard
error
1.7%
0.7%

Late Wave
Unweighted
respondent
count
2,289
9,663

Late
Wave
Weighted
percent
45.6%
32.1%

Late
Wave
Standard
error
1.9%
0.9%

Charter
Non-charter

1,006
8,784

7.7%
4.0%

1.5%
0.4%

1,128
9,883

22.1%
17.5%

2.1%
0.7%

1,304
10,648

40.6%
34.9%

2.3%
0.9%

Primary
Middle
High
Combined

3,821
2,012
2,854
1,103

3.6%
3.7%
5.4%
4.1%

0.5%
0.8%
0.8%
0.9%

4,618
2,132
2,973
1,288

16.4%
18.6%
19.1%
20.0%

0.9%
1.3%
1.4%
1.7%

4,602
2,188
3,859
1,303

35.8%
35.3%
34.1%
35.7%

1.2%
1.7%
1.8%
2.6%

City
Suburban
Town
Rural

2,027
3,306
1,758
2,699

5.4%
3.5%
5.9%
3.0%

0.9%
0.6%
1.0%
0.5%

2,772
3,690
1,777
2,772

19.9%
16.7%
17.1%
16.9%

1.2%
1.1%
1.5%
1.1%

4,165
3,808
1,536
2,443

38.5%
34.2%
30.0%
32.3%

1.4%
1.4%
1.9%
1.7%

Northeast
Midwest
South
West

1,499
2,794
3,770
1,727

3.7%
3.4%
4.1%
5.9%

0.8%
0.6%
0.5%
1.1%

1,867
2,782
4,185
2,177

17.5%
17.1%
17.5%
18.8%

1.6%
1.1%
1.0%
1.4%

2,156
2,570
4,747
2,479

37.3%
30.8%
34.7%
37.6%

1.9%
1.8%
1.2%
2.0%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

3,431
1,854
2,886
1,619

3.3%
5.5%
4.0%
4.9%

0.5%
1.1%
0.6%
0.8%

3,343
1,971
3,214
2,483

15.7%
16.9%
18.0%
20.9%

1.2%
1.4%
1.1%
1.3%

3,297
2,027
3,268
3,360

31.8%
32.5%
35.4%
39.7%

1.6%
2.0%
1.9%
1.5%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

204
477
2,965
2,594
1,264
2,286

0.6%
3.0%
4.0%
3.8%
4.9%
4.7%

0.4%
1.0%
0.6%
0.6%
1.3%
0.7%

290
568
3,302
3,152
1,621
2,078

13.9%
19.8%
16.6%
18.1%
18.1%
18.3%

2.8%
2.4%
1.1%
1.2%
1.4%
1.6%

256
535
3,337
3,040
1,724
3,060

59.1%
34.3%
36.4%
35.7%
33.4%
32.9%

5.9%
3.3%
1.5%
1.5%
2.2%
1.7%

Regular
Special education
Vocational
Alternative

9,428
108
92
162

4.2%
0.0%
4.1%
6.6%

0.4%
0.0%
2.9%
2.2%

10,514
149
130
218

17.8%
13.4%
15.2%
18.5%

0.7%
3.6%
5.3%
3.8%

11,389
117
119
327

34.6%
63.6%
23.5%
51.2%

0.8%
7.0%
6.1%
6.4%

Field Follow-up
Special district
Not special district

E

Early
Wave
Weighted
percent
5.0%
4.0%

Teacher Questionnaire Response Experience:
Detailed Tables

Early Wave
Unweighted
respondent
count
1,270
8,520

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-3B.

Percentage of teacher respondents with telephone/mail follow-up by teacher domain by teacher wave group,
NTPS 2015-16
Early
Wave
Weighted
percent

E-10

Early
Wave
Standard
error

Middle Wave
Unweighted
respondent
count

Middle
Wave
Weighted
percent

Middle
Wave
Standard
error

Late Wave
Unweighted
respondent
count

Late
Wave
Weighted
percent

Late
Wave
Standard
error

Primary - Missing
Primary - Special ed
Primary - General elem.
Primary - Math
Primary - Science
Primary - English
Primary - Social studies
Primary - Other

33
394
2,388
137
54
153
23
639

8.4%
1.6%
4.0%
4.1%
2.4%
5.1%
0.0%
3.1%

8.8%
0.6%
0.6%
2.1%
1.4%
1.9%
0.0%
0.8%

40
534
2,879
157
68
169
35
736

14.3%
13.2%
17.0%
23.1%
15.5%
17.5%
7.2%
15.4%

5.1%
1.7%
1.0%
3.9%
4.9%
3.5%
4.3%
1.5%

50
501
2,655
147
62
277
27
883

34.0%
33.1%
36.2%
30.3%
40.4%
35.0%
39.6%
37.0%

15.9%
2.4%
1.3%
4.8%
7.8%
3.4%
11.6%
2.1%

Middle - Missing
Middle - Special ed
Middle - General elem.
Middle - Math
Middle - Science
Middle - English
Middle - Social studies
Middle - Vo/Tech
Middle - Other

22
276
41
313
243
379
205
78
455

8.1%
0.3%
3.5%
3.7%
3.6%
4.1%
4.1%
6.5%
4.6%

8.9%
0.3%
2.6%
1.2%
1.4%
1.2%
1.5%
3.2%
1.4%

36
304
43
348
254
392
222
60
473

9.4%
17.3%
11.8%
22.2%
16.1%
19.1%
20.2%
13.6%
19.0%

10.1%
2.2%
5.7%
2.5%
2.6%
2.3%
3.0%
5.1%
2.1%

31
268
140
245
177
415
165
27
720

40.7%
32.2%
35.2%
37.9%
35.4%
34.9%
42.0%
29.0%
34.5%

15.7%
3.2%
5.1%
3.1%
3.5%
2.6%
4.2%
9.0%
2.3%

High - Missing
High - Special ed
High - Math
High - Science
High - English
High - Social studies
High - Vo/Tech
High - Other

15
353
379
374
433
333
287
680

0.0%
3.9%
4.0%
4.9%
4.7%
7.3%
6.7%
6.2%

0.0%
1.3%
1.0%
1.2%
1.2%
1.6%
1.8%
1.0%

39
356
415
349
429
343
308
734

37.5%
13.3%
19.7%
19.5%
19.3%
19.1%
20.7%
19.7%

10.5%
2.3%
2.2%
2.3%
2.2%
2.5%
2.7%
2.0%

98
460
509
448
648
390
222
1,084

38.9%
25.9%
40.3%
35.7%
30.8%
32.3%
34.8%
36.4%

11.0%
2.3%
3.6%
3.0%
2.3%
2.5%
4.2%
2.2%

Teacher Questionnaire Response Experience:
Detailed Tables

Field Follow-up

Early Wave
Unweighted
respondent
count

E

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-3B.

Percentage of teacher respondents with telephone/mail follow-up by teacher domain by teacher wave group,
NTPS 2015-16 (continued)

Field Follow-up

Combined - Missing
Combined - Special ed
Combined - General elem.
Combined - Math
Combined - Science
Combined - English
Combined - Social studies
Combined - Vo/Tech
Combined - Other

E-11

Status missing
Full-time teachers
Part-time teachers

Early Wave
Unweighted
respondent
count

Early
Wave
Weighted
percent

Early
Wave
Standard
error

Middle Wave
Unweighted
respondent
count

Middle
Wave
Weighted
percent

Middle
Wave
Standard
error

Late Wave
Unweighted
respondent
count

Late
Wave
Weighted
percent

Late
Wave
Standard
error

13
201
219
124
98
118
81
56
193

0.0%
2.2%
5.4%
3.2%
5.0%
6.9%
3.6%
0.0%
5.3%

0.0%
1.1%
1.7%
1.8%
2.2%
3.1%
2.1%
0.0%
1.8%

20
214
253
133
114
139
95
82
238

40.6%
15.1%
22.2%
16.5%
24.7%
16.6%
15.2%
20.8%
24.2%

17.4%
2.4%
3.3%
3.2%
4.2%
3.6%
3.8%
4.6%
3.5%

47
200
267
130
91
165
88
36
279

49.0%
42.1%
30.6%
31.6%
37.9%
31.3%
40.8%
28.4%
36.1%

12.0%
5.3%
4.4%
4.4%
6.2%
4.5%
5.6%
9.4%
3.5%

246
8,928
616

7.1%
4.1%
4.4%

2.5%
0.4%
1.0%

609
10,419
754

16.4%
17.9%
14.5%

3.2%
0.7%
1.5%

5,935
5,624
393

34.2%
36.2%
36.5%

1.2%
1.2%
3.4%
Teacher Questionnaire Response Experience:
Detailed Tables

E

Teacher Questionnaire Response Experience:
Detailed Tables

E

While special afternoon reminder calls do not appear to reduce the follow-up effort overall, the
percentages of teacher respondents who responded before field follow-up within school and teacher
domains is compared by experimental group in Table E-4. The special afternoon reminder calls had
some impact on a small number of domains: teachers from schools with enrollment 100-199, middle
school math teachers, middle school math teachers, and high school social studies teachers. All of
these domains have less than 300 teachers. The statistics for these domains are in bold and italic in
Table E-4. If there is any effect of special afternoon calls, it looks minimal.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

E-12

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-4.

Percentage of teacher respondents with telephone or mail follow-up by nonresponse follow-up experimental group by
school/teacher domain, NTPS 2015-16

Domain
Special district
Not special district

Teacher experimental group 1 special afternoon call
Unweighted
respondent
Unweighted
Weighted
Standard
count
percent
percent
error
952
26.2%
25.0%
1.8%
4,172
36.9%
36.4%
1.0%

Teacher experimental group 2 regular call schedule
Unweighted
respondent
Unweighted
Weighted
Standard
count
percent
percent
error
959
30.1%
29.3%
2.3%
4,045
35.6%
34.8%
1.0%

E-13

31.1%
35.4%

28.9%
34.2%

2.0%
1.0%

594
4,410

32.5%
34.8%

31.0%
33.7%

2.1%
0.9%

Primary
Middle
High
Combined

2,074
956
1,524
570

36.2%
35.4%
33.4%
33.2%

35.2%
33.1%
32.6%
32.9%

1.3%
1.9%
2.0%
2.6%

1,975
953
1,502
574

35.9%
35.6%
32.9%
32.4%

34.9%
33.6%
31.9%
31.8%

1.3%
1.8%
2.1%
3.0%

City
Suburban
Town
Rural

1,631
1,660
730
1,103

28.7%
36.4%
38.8%
39.2%

28.9%
35.5%
37.6%
38.9%

1.7%
1.5%
2.2%
1.9%

1,638
1,620
670
1,076

30.6%
36.2%
36.6%
36.7%

29.9%
35.2%
35.6%
37.1%

1.6%
1.6%
2.3%
1.8%

Northeast
Midwest
South
West

935
1,129
1,968
1,092

34.0%
38.9%
33.0%
34.9%

34.1%
37.5%
32.4%
33.3%

2.0%
1.8%
1.4%
2.4%

928
1,085
1,922
1,069

33.1%
38.8%
33.1%
34.1%

32.5%
37.1%
32.9%
32.9%

1.9%
1.9%
1.5%
2.5%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

1,430
881
1,410
1,403

39.9%
34.4%
35.7%
29.2%

39.9%
33.7%
33.7%
27.9%

1.9%
2.2%
1.9%
1.5%

1,367
841
1,395
1,401

38.0%
37.2%
34.3%
29.8%

38.0%
35.5%
32.9%
28.7%

2.0%
2.1%
1.8%
1.6%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

150
252
1,518
1,368
718
1,118

36.0%
42.1%
36.6%
35.1%
32.6%
32.0%

40.2%
41.0%
36.2%
33.7%
31.5%
31.3%

10.0%
3.8%
1.7%
1.5%
2.3%
1.7%

131
247
1,460
1,323
714
1,129

32.8%
32.8%
35.1%
35.4%
36.0%
32.4%

35.4%
30.3%
33.8%
34.1%
36.3%
31.2%

12.1%
3.5%
1.5%
1.6%
2.3%
1.9%

E

662
4,462

Teacher Questionnaire Response Experience:
Detailed Tables

Charter
Non-charter

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-4.

Response follow-up experience with teacher-interview final respondents by nonresponse follow-up experimental group by
school/teacher domain, NTPS 2015-16 (continued)

Domain
Regular
Special education
Vocational
Alternative

E-14

Primary - Missing
Primary - Special ed
Primary - General elem.
Primary - Math
Primary - Science
Primary - English
Primary - Social studies
Primary - Other

Teacher experimental group 1 special afternoon call
Unwgtd
respondent
Unwgtd
Wgtd
Standard
count
percent
percent
error
4,864
35.1%
33.9%
0.9%
82
34.1%
33.7%
7.8%
53
37.7%
56.9%
10.5%
125
26.4%
20.3%
5.2%

Teacher experimental group 2 regular call schedule
Unwgtd
respondent
Unwgtd
Wgtd
Standard
count
percent
percent
error
4,735
34.5%
33.5%
0.9%
72
44.4%
38.3%
9.9%
52
46.2%
59.8%
14.2%
145
26.2%
20.4%
5.7%

39.2%
37.4%
35.3%
27.8%
43.7%
27.4%
55.1%
35.5%

14.0%
3.8%
1.6%
5.5%
9.3%
4.6%
13.3%
2.5%

20
207
1,223
61
28
90
11
335

40.0%
38.2%
35.7%
44.3%
32.1%
34.4%
45.5%
34.0%

35.9%
36.4%
34.9%
42.9%
37.9%
32.6%
41.5%
32.7%

14.9%
4.0%
1.6%
7.4%
10.0%
5.1%
18.4%
2.9%

Middle - Missing
Middle - Special ed
Middle - General elem.
Middle - Math
Middle - Science
Middle - English
Middle - Social studies
Middle - Vo/Tech
Middle - Other

11
106
39
157
96
169
94
13
271

54.5%
34.0%
28.2%
37.6%
40.6%
30.8%
41.5%
30.8%
33.9%

41.8%
35.3%
24.1%
35.6%
38.2%
30.2%
37.7%
6.6%
32.1%

24.4%
4.8%
6.2%
4.0%
5.2%
3.7%
5.3%
6.9%
3.2%

15
121
35
115
100
177
92
19
279

33.3%
41.3%
28.6%
24.3%
45.0%
36.7%
29.3%
42.1%
36.2%

27.6%
40.8%
23.2%
21.6%
44.6%
32.8%
24.5%
39.6%
34.9%

16.5%
4.6%
7.2%
3.9%
5.4%
3.2%
5.1%
12.5%
2.9%

High - Missing
High - Special ed
High - Math
High - Science
High - English
High - Social studies
High - Vo/Tech
High - Other

38
141
217
164
237
161
144
422

23.7%
39.7%
28.6%
40.9%
30.8%
37.9%
38.2%
29.9%

21.2%
45.5%
26.0%
36.2%
32.0%
39.8%
35.4%
27.3%

9.7%
6.5%
3.3%
4.0%
3.7%
4.6%
4.7%
2.5%

29
158
193
208
212
169
109
424

27.6%
36.1%
32.1%
34.6%
33.0%
31.4%
33.9%
31.8%

30.0%
35.4%
29.3%
31.1%
32.4%
27.1%
35.0%
33.1%

11.3%
4.1%
3.8%
3.5%
3.5%
3.7%
4.8%
4.4%

E

40.9%
37.4%
35.9%
29.9%
45.9%
28.7%
50.0%
38.1%

Teacher Questionnaire Response Experience:
Detailed Tables

22
190
1,222
77
37
108
14
404

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table E-4.

Response follow-up experience with teacher-interview final respondents by nonresponse follow-up experimental group by
school/teacher domain, NTPS 2015-16 (continued)

Domain
Combined - Missing
Combined - Special ed
Combined - General elem.
Combined - Math
Combined - Science
Combined - English
Combined - Social studies
Combined - Vo/Tech
Combined - Other

E-15

Full-time teachers
Part-time teachers
Status missing

Teacher experimental group 1 special afternoon call
Unwgtd
respondent
Unwgtd
Wgtd
Standard
count
percent
percent
error
19
15.8%
22.9%
19.9%
96
36.5%
35.9%
7.1%
109
33.9%
32.3%
4.8%
60
31.7%
33.3%
7.0%
38
31.6%
27.9%
6.6%
58
39.7%
38.4%
6.5%
41
29.3%
32.5%
7.8%
29
41.4%
32.4%
9.7%
120
30.0%
30.7%
4.8%
1,336
3,535
253

19.3%
40.1%
43.5%

18.3%
39.8%
42.7%

1.3%
1.2%
3.5%

Teacher experimental group 2 regular call schedule
Unwgtd
respondent
Unwgtd
Wgtd
Standard
count
percent
percent
error
19
10.5%
8.1%
4.6%
97
37.1%
39.0%
9.2%
110
27.3%
26.8%
5.6%
50
46.0%
48.2%
8.3%
56
25.0%
24.7%
6.3%
65
33.8%
32.8%
6.4%
40
37.5%
35.9%
7.2%
19
47.4%
55.4%
13.0%
118
29.7%
25.4%
4.2%
1,260
3,497
247

19.1%
39.2%
46.6%

18.8%
38.7%
44.6%

1.4%
1.1%
3.5%
Teacher Questionnaire Response Experience:
Detailed Tables

E

Appendix F
Experimental Study Detailed Tables

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

F-1

Experimental Study Detailed Tables

F

Tables F-1 through F-3 show the response rate for the school questionnaire, principal questionnaire,
and teacher listing form by school domain. The difference in response rates is significant in twenty
one school domains out of thirty for the school questionnaire, twenty school domains for the
teacher listing form, and two school domains for the principal questionnaire at a significant level of
5%. The statistics in these domains are in bold and italic in Tables F-1 and F-3.

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

F-2

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table F-1.

School response rate comparison by experimental group and by school domain, NTPS 2015-16
Main Study
Sample
Size
8,300

Eligible
schools
8,029

Special district
Not special district

1,449
6,851

Charter
Non-charter

Experimental Group
Response
Completes
rate
288
29.4%

F-3

Sample
Size
1,000

Eligible
schools
980

1,421
6,608

400
2,678

28.1%
40.5%

1.2%
0.6%

164
836

163
817

19
269

11.7%
32.9%

2.5%
1.6%

1,173
7,127

1,094
6,935

355
2,723

32.4%
39.3%

1.4%
0.6%

142
858

141
839

36
252

25.5%
30.0%

3.7%
1.6%

Primary
Middle
High
Combined

3,708
1,441
2,054
1,097

3,626
1,420
1,981
1,002

1,410
544
705
419

38.9%
38.3%
35.6%
41.8%

0.8%
1.3%
1.1%
1.5%

439
175
250
136

435
175
243
127

136
53
68
31

31.3%
30.3%
28.0%
24.4%

2.2%
3.5%
2.8%
3.7%

City
Suburban
Town
Rural

2,507
2,585
1,201
2,007

2,395
2,520
1,161
1,953

700
932
534
912

29.2%
37.0%
46.0%
46.7%

0.9%
0.9%
1.4%
1.1%

301
312
145
242

297
309
139
235

59
91
55
83

19.9%
29.4%
39.6%
35.3%

2.3%
2.6%
4.1%
3.1%

Northeast
Midwest
South
West

1,352
1,924
3,200
1,824

1,332
1,849
3,087
1,761

435
868
1,111
664

32.7%
46.9%
36.0%
37.7%

1.3%
1.1%
0.8%
1.1%

159
235
388
218

157
232
379
212

41
78
121
48

26.1%
33.6%
31.9%
22.6%

3.5%
3.1%
2.4%
2.8%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

2,416
1,395
2,331
2,158

2,323
1,357
2,270
2,079

941
576
936
625

40.5%
42.4%
41.2%
30.1%

1.0%
1.3%
1.0%
1.0%

283
176
281
260

275
172
275
258

86
55
87
60

31.3%
32.0%
31.6%
23.3%

2.8%
3.5%
2.8%
2.6%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

404
551
2,820
2,151
1,035
1,339

342
512
2,733
2,098
1,022
1,322

145
228
1,101
819
360
425

42.4%
44.5%
40.3%
39.0%
35.2%
32.1%

2.5%
2.1%
0.9%
1.1%
1.5%
1.3%

50
69
360
232
132
157

44
68
352
231
132
153

11
23
108
69
38
39

25.0%
33.8%
30.7%
29.9%
28.8%
25.5%

6.1%
5.7%
2.4%
3.0%
3.9%
3.5%

Regular
Special education
Vocational
Alternative

7,641
158
111
390

7,485
125
99
320

2,899
49
39
91

38.7%
39.2%
39.4%
28.4%

0.6%
3.9%
4.6%
2.3%

939
19
14
28

927
18
10
25

278
4
4
2

30.0%
22.2%
40.0%
8.0%

1.5%
9.5%
13.1%
5.1%

Domain
All

Standard
error
1.4%

F

Standard
error
0.5%

Experimental Study Detailed Tables

Completes
3,078

Response
rate
38.3%

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table F-2.

Principal response rate comparison by experimental group and by school domain, NTPS 2015-16
Main Study
Sample
Size
8,300

Eligible
schools
8,025

Special district
Not special district

1,449
6,851

Charter
Non-charter

Experimental Group
Response
Completes
rate
380
38.9%

F-4

Sample
Size
1,000

Eligible
schools
978

1,420
6,605

409
2,760

28.8%
41.8%

1.2%
0.6%

164
836

163
815

30
350

18.4%
42.9%

3.0%
1.7%

1,173
7,127

1,093
6,932

372
2,797

34.0%
40.3%

1.4%
0.6%

142
858

140
838

46
334

32.9%
39.9%

3.9%
1.7%

Primary
Middle
High
Combined

3,708
1,441
2,054
1,097

3,622
1,420
1,981
1,002

1,453
546
729
441

40.1%
38.5%
36.8%
44.0%

0.8%
1.3%
1.1%
1.5%

439
175
250
136

434
175
242
127

166
71
93
50

38.2%
40.6%
38.4%
39.4%

2.3%
3.7%
3.1%
4.2%

City
Suburban
Town
Rural

2,507
2,585
1,201
2,007

2,394
2,519
1,159
1,953

724
952
558
935

30.2%
37.8%
48.1%
47.9%

0.9%
1.0%
1.4%
1.1%

301
312
145
242

295
309
139
235

82
117
69
112

27.8%
37.9%
49.6%
47.7%

2.6%
2.7%
4.2%
3.2%

Northeast
Midwest
South
West

1,352
1,924
3,200
1,824

1,331
1,849
3,087
1,758

450
897
1,140
682

33.8%
48.5%
36.9%
38.8%

1.3%
1.1%
0.9%
1.1%

159
235
388
218

157
231
379
211

62
94
149
75

39.5%
40.7%
39.3%
35.5%

3.9%
3.2%
2.5%
3.2%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

2,416
1,395
2,331
2,158

2,321
1,357
2,270
2,077

963
596
963
647

41.5%
43.9%
42.4%
31.2%

1.0%
1.3%
1.0%
1.0%

283
176
281
260

275
172
275
256

115
76
111
78

41.8%
44.2%
40.4%
30.5%

2.9%
3.7%
2.9%
2.9%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

404
551
2,820
2,151
1,035
1,339

343
512
2,731
2,096
1,022
1,321

147
229
1,138
844
372
439

42.9%
44.7%
41.7%
40.3%
36.4%
33.2%

2.5%
2.1%
0.9%
1.1%
1.5%
1.3%

50
69
360
232
132
157

44
68
352
229
132
153

18
26
147
89
46
54

40.9%
38.2%
41.8%
38.9%
34.8%
35.3%

7.0%
5.9%
2.6%
3.2%
4.1%
3.8%

Regular
Special education
Vocational
Alternative

7,641
158
111
390

7,480
125
99
321

2,988
48
39
94

39.9%
38.4%
39.4%
29.3%

0.6%
3.9%
4.6%
2.3%

939
19
14
28

925
18
10
25

362
8
6
4

39.1%
44.4%
60.0%
16.0%

1.6%
11.4%
13.1%
6.9%

Domain
All

Standard
error
1.5%

F

Standard
error
0.5%

Experimental Study Detailed Tables

Completes
3,169

Response
rate
39.5%

NTPS 2015-2016 Analysis of Response Rates and
Field Collection Experience

Table F-3.

Teacher listing form response rate comparison by experimental group and by school domain, NTPS 2015-16
Main Study
Sample
Size
8,300

Eligible
schools
8,025

Special district
Not special district

1,449
6,851

Charter
Non-charter

Experimental Group
Response
Completes
rate
291
29.7%

F-5

Sample
Size
1,000

Eligible
schools
980

1,421
6,604

381
2,718

26.8%
41.2%

1.2%
0.6%

164
836

163
817

21
270

12.9%
33.0%

2.6%
1.6%

1,173
7,127

1,093
6,932

366
2,733

33.5%
39.4%

1.4%
0.6%

142
858

141
839

35
256

24.8%
30.5%

3.6%
1.6%

Primary
Middle
High
Combined

3,708
1,441
2,054
1,097

3,624
1,420
1,979
1,002

1,424
535
714
426

39.3%
37.7%
36.1%
42.5%

0.8%
1.3%
1.1%
1.5%

439
175
250
136

435
175
243
127

136
49
69
37

31.3%
28.0%
28.4%
29.1%

2.2%
3.4%
2.9%
3.9%

City
Suburban
Town
Rural

2,507
2,585
1,201
2,007

2,393
2,517
1,161
1,954

710
920
546
923

29.7%
36.6%
47.0%
47.2%

0.9%
0.9%
1.4%
1.1%

301
312
145
242

297
309
139
235

62
92
54
83

20.9%
29.8%
38.8%
35.3%

2.3%
2.6%
4.0%
3.1%

Northeast
Midwest
South
West

1,352
1,924
3,200
1,824

1,332
1,850
3,085
1,758

449
867
1,137
646

33.7%
46.9%
36.9%
36.7%

1.3%
1.1%
0.9%
1.1%

159
235
388
218

157
232
379
212

45
77
119
50

28.7%
33.2%
31.4%
23.6%

3.6%
3.1%
2.4%
2.9%

FRPL 0-34%
FRPL 35-49%
FRPL 50-75%
FRPL 75-100%

2,416
1,395
2,331
2,158

2,318
1,358
2,270
2,079

937
568
953
641

40.4%
41.8%
42.0%
30.8%

1.0%
1.3%
1.0%
1.0%

283
176
281
260

275
172
275
258

85
60
88
58

30.9%
34.9%
32.0%
22.5%

2.7%
3.6%
2.8%
2.6%

Enrollment 0-99
Enrollment 100-199
Enrollment 200-499
Enrollment 500-749
Enrollment 750-999
Enrollment 1000+

404
551
2,820
2,151
1,035
1,339

342
512
2,731
2,097
1,022
1,321

146
228
1,124
821
355
425

42.7%
44.5%
41.2%
39.2%
34.7%
32.2%

2.5%
2.1%
0.9%
1.1%
1.5%
1.3%

50
69
360
232
132
157

44
68
352
231
132
153

12
22
109
68
38
42

27.3%
32.4%
31.0%
29.4%
28.8%
27.5%

6.3%
5.6%
2.4%
3.0%
3.9%
3.6%

Regular
Special education
Vocational
Alternative

7,641
158
111
390

7,480
125
98
322

2,914
48
40
97

39.0%
38.4%
40.8%
30.1%

0.6%
3.9%
4.7%
2.3%

939
19
14
28

927
18
10
25

278
5
5
3

30.0%
27.8%
50.0%
12.0%

1.5%
10.3%
13.4%
6.1%

Domain
All

Standard
error
1.4%

F

Standard
error
0.5%

Experimental Study Detailed Tables

Completes
3,099

Response
rate
38.6%


File Typeapplication/pdf
AuthorMinsun Riddles
File Modified2017-09-01
File Created2017-08-31

© 2024 OMB.report | Privacy Policy