Part B TFS PFS 2024-25 v13

Part B TFS PFS 2024-25 v13.docx

FOLLOW-UP SURVEYS TO THE 2023-24 NTPS: 2024-25 Teacher Follow-Up Survey (TFS) and 2024-25 Principal Follow-Up

OMB: 1850-0617

Document [docx]
Download: docx | pdf



FOLLOW-UP SURVEYS TO THE 2023-24 NATIONAL TEACHER AND PRINCIPAL SURVEY (NTPS):

2024-25 Teacher Follow-Up Survey (TFS) and

2024-25 Principal Follow-Up Survey (PFS)




PART B



OMB# 1850-0617 v.13



National Center for Education Statistics (NCES)

U.S. Department of Education






February 2024

revised May 2024

revised June 2024

revised January 2025












TABLE OF CONTENTS





PART B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS


This request is to conduct data collection for the two follow-up surveys to the 2023-24 National Teacher and Principal Survey (NTPS) – the 2024-25 Teacher Follow-up Survey (TFS) and the 2024-25 Principal Follow-up Survey (PFS).


Section B.1 of this document describes the universe, sample design, and estimation details for the 2024-25 TFS and PFS. Section B.2 describes the data collection procedures for both surveys. Section B.3 discusses methods to secure cooperation and mitigate nonresponse. In particular, it describes methods used to improve response rates in the 2023-24 NTPS and how those methods will be used in its two follow-up surveys. Section B.4 describes recent developments in a long history of tests of methods and procedures to improve data quality. Section B.5 lists the names of those involved in the design of the study and the development of these materials.


B.1 Respondent Universe and Sample Design

B.1.1 Background on the 2023-24 NTPS Sample

The 2023-24 National Teacher and Principal Survey (2023-24 NTPS) sample was a two-stage stratified sample. The schools were sampled first, and then teachers were selected within each sampled school.


The 2023-24 NTPS school sampling used implicit stratification based on proposed systematic sampling order, which uses a hierarchy of domains which vary between public and private schools. The 2023-24 NTPS oversampling stratification for public schools was based primarily on the following domains: charter status (charter, non-charter), school level (elementary, middle, secondary/high, combined/other), urbanicity (city, suburb, town, rural), and state. The private school oversampling domains are defined by affiliation (three levels), and school level (Elementary/Middle; Secondary/High; Combined/Other). The oversampling rates for public and private schools were as follows:


Table B1. NTPS 2023-24 public school oversampling domain and oversampling rates.

Charter type

School level

Urbanicity

School domain oversampling rate

Charter

All

All

2.91

Non-charter

Elementary

City

1.04

Non-charter

Elementary

Suburb

1.00

Non-charter

Elementary

Town

1.34

Non-charter

Elementary

Rural

1.12

Non-charter

Middle

City

1.33

Non-charter

Middle

Suburb

1.25

Non-charter

Middle

Town

1.81

Non-charter

Middle

Rural

1.45

Non-charter

Secondary/high

City

1.33

Non-charter

Secondary/high

Suburb

1.22

Non-charter

Secondary/high

Town

1.81

Non-charter

Secondary/high

Rural

1.46

Non-charter

Combined/other

All

4.69


Table B2. NTPS 2023-24 private school oversampling domain and oversampling rates.

School level

Affiliation

School domain oversampling rate

Elementary/Middle

Nonreligious

1.28

Elementary/Middle

Baptist

1.23

Elementary/Middle

Other religious

1.00

Secondary/High

Nonreligious

3.58

Secondary/High

Baptist

3.44

Secondary/High

Other religious

2.80

Combined/Other

Nonreligious

1.56

Combined/Other

Baptist

1.50

Combined/Other

Other religious

1.22

Elementary/Middle

Nonreligious

1.28

Elementary/Middle

Baptist

1.23

Elementary/Middle

Other religious

1.00

Secondary/High

Nonreligious

3.58

Secondary/High

Baptist

3.44



In addition to oversampling by characteristic, public schools in 31 states (generally smaller states, based on population size) and the District of Columbia had relatively higher sampling rates to realize lower bounds on precision for these states and to enable the calculation of reliable key state estimates.


Whenever possible, teacher rosters for the sampled schools were prepopulated using vendor-purchased teacher roster files, web scraped data, and the results of a clerical look-up operation to identify teacher lists located on school and/or district websites. Sampled schools were then asked to verify or provide (when the roster could not be prepopulated) the following information for all full- and part-time teachers in their school: name, email address, and subject matter taught (selecting from the following subjects: special education, general elementary, math, science, English/language arts, social studies, vocational/technical, world language, music or art, or another subject).


Within each sampled school, teachers were stratified by whether they were newly added to the teacher roster, and then sorted by subject taught (special education, general elementary, math, science, English/language arts, social studies, vocational/technical, world language, music/art, and everything else), and the teacher line number code, which is a unique number assigned to identify the teacher within the list of keyed teachers. When no new teacher was added to the roster for a sampled school, teachers from that school were considered to be in a single stratum, and sorted by subject taught and the teacher line number code without additional stratification. Within each teacher stratum in each school, teachers were selected systematically with equal probability.


Table B3 below shows a summary of the 2023-24 NTPS allocated sample sizes. Note that teachers are sampled for the NTPS throughout the 2023-24 school year, so sample sizes are an estimate at this time. The final sample allocations will be known at the conclusion of 2023-24 NTPS teacher sampling, in spring 2024.

Table B3. Summary of the 2023-24 NTPS sample sizes


School

Teacher

Public

10,100

59,713

Private

3,120

9,468

Total

13,220

69,181


B.1.2 2024-25 TFS Respondent Universe and Sample Design

The 2024-25 Teacher Follow-up Survey (2024-25 TFS) sample will be a subsample from the 2023-24 NTPS teacher sample, and as such, a probability sample similar to 2023-24 NTPS.


A planned total of 10,465 teachers will be selected according to a stratified design similar to earlier TFS collections and based on information obtained from the NTPS schools about the status of the 2023-24 NTPS respondent teachers during the 2024-25 school year, the school year following NTPS. The 2024-25 TFS sample will include former teachers (NTPS teachers who have left the teaching profession, or “leavers”), current teachers (NTPS teachers who have remained in the teaching profession, either still teaching at their base year school [“stayers”] or still teaching but at a different school since the base year ["movers"]), and teachers who have left their 2023-24 school, but for whom it is unknown whether they are still teaching at another school (the “unknown” group).


The current teachers group is further stratified into two groups: those who remained in the school they were teaching in 2023-24 (“stayers”) and those who moved to different schools (“movers”). This design provides analytic data on teachers who stay, move, or leave the teaching profession and teachers whose occupational status is unknown. The status for each TFS teacher is determined from Teacher Status Data Collection Operation (collection of the NTPS-5A/5B/5C from schools via web and paper as follow-up) during the fall of 2024. For the 2024-25 TFS, the movers, leavers, and unknown teachers will be sampled with certainty.


Stayers will be stratified by factors known about the teacher from the NTPS year, including sector (traditional public, public charter, and private), school level (elementary, middle, secondary/high, and combined/other)1, experience level (1-5 years, and 6+ years of teaching), and race/ethnicity (Hispanic, Black non-Hispanic, and other non-Hispanic).


The table below presents target sample sizes for important teacher categories. Movers, leavers, and teachers with unknown status will all be selected, which makes them overrepresented in the sample relative to other teachers. This will be done to increase the precision for analyses of these important subgroups. The breakdown of stayers, movers, leavers, and ‘unknown’ teachers in public, charter, and private schools is as shown in Table B4.


Table B4. Target Distribution of teacher types across school types

Status

Sector

Public

Charter

Private

Total

Stayer

2,625

321

997

3,943

Mover

1,962

287

172

2,421

Leaver

1,101

157

212

1,470

Unknown

1,939

395

297

2,631

Total

7,627

1,160

1,678

10,465



Within these target sample sizes, Hispanic and Black Non-Hispanic stayers will be oversampled by 60 percent to optimize the reliability of their comparison with other stayers (movers, leavers, and unknown are already sampled with certainty). The target sample sizes at the status/sector/race level are as seen in Table B5.


Using these target sample sizes for each of the main strata, the teachers will be sampled through probability proportional to size of the substrata, which are defined by the teacher’s sector, status, experience, school level, and race categories. This will be done proportional to the cumulative teacher measure of size within each main stratum. The teacher measure of size will be the 2023-24 NTPS teacher base weight. Within each substratum, there will be systematic sampling with a designated sort order.


The sort variables include measure of size (teacher base weight), teacher subject, state, urbanicity, school enrollment, teacher instructional grade level, teacher control number, and, for private school teachers, school affiliation. As the measure of size will be the 2023-24 NTPS teacher base weights, this essentially ‘reverses’ the probability of selection of the teacher in NTPS, and results in a self-weighting sample within each substratum. This will maximize the efficiency of the TFS sample.


Table B5. Distribution of teacher types across school types

Status

Sector

Total

Public

Charter

Private

Hispanic

Black Non-Hispanic

Other

Hispanic

Black Non-Hispanic

Other

Hispanic

Black Non-Hispanic

Other

Stayer

258

282

2,085

106

94

121

181

109

707

3,943

Mover

132

88

1,742

35

22

230

17

6

149

2,421

Leaver

62

42

997

24

12

121

18

10

184

1,470

Unknown

185

122

1,632

45

41

309

36

15

246

2,631

Total

637

534

6,456

210

169

781

252

140

1,286

10,465


B.1.3 2024-25 PFS Respondent Universe and Sample Design

The sampling frame for the 2024-25 PFS consists of all the traditional public and public charter school principals who completed a Principal Questionnaire and all private school principals who completed a Private School Principal Questionnaire during the 2023-24 NTPS. Any sampled NTPS principal who did not complete their questionnaire or was otherwise found to be out of scope for NTPS will not be included in the PFS frame. The 2024-25 PFS sample will include approximately 6,850 public and public charter school principals and 1,950 private school principals. The 2021-22 PFS included a sample of 6,508 public and public charter school principals and 1,753 private school principals.


B.2 Procedures for the Collection of Information

B.2.1 2024-25 TFS Procedures for the Collection of Information


B.2.1.1 Teacher Status Form (NTPS-5A/5B/5C)

Data collection for the TFS will begin with contacting schools in which at least one teacher completed a Teacher Questionnaire during the 2023-24 administration of NTPS. A knowledgeable person at the school will be asked to complete a teacher status form electronically via a Teacher Status Qualtrics web instrument. Specifically, the principal or other knowledgeable person will indicate the current teaching status of each NTPS sampled teacher, by selecting one of seven occupation status designations for each teacher, as well as indicating whether last year's principal, if information about last year's principal was collected on the NTPS, is still the principal of the school this school year. These data will be used to stratify the teachers for 2024-25 TFS sampling into groups of “stayers, movers, leavers, and teachers whose status is unknown. The principal status information will serve as a data point for a validation study conducted as part of the 2024-25 PFS.


Traditionally, this operation was conducted by paper questionnaire only; however, it will be conducted primarily via web starting with the 2024-25 TFS, with paper questionnaires offered in a follow-up mailing. There will be 3 different versions of the paper Teacher Status Form – the Teacher and Principal Status Form for public schools where the 2023-24 principal’s name is known (NTPS-5A), the Teacher and Principal Status Form for private schools where the 2023-24 principal’s/school head’s name is known (NTPS-5B), and the Teacher Status Form (NTPS-5C) for public and private schools where the principal’s/school head’s name is not known (excludes the principal status item).


TFS was designed from its inception to sample only from respondents to SASS/NTPS in the prior year. It is possible to sample teachers for TFS from the pool of teachers initially selected for NTPS rather than only from those who responded to NTPS. By not sampling teachers from the nonresponse stratum, potential biases may be introduced into TFS. However, when considering unit nonresponse for SASS, there was no evidence to point to a substantial bias in teacher response rates.


The main purpose of TFS is to measure various characteristics collected from NTPS teacher respondents of teachers who move, leave, or stay in the teaching profession the following year. A response rate of approximately 84% percent can be expected from the Teacher Status data collection operations with schools, and the expectation is that the overall Teacher Status response rate will be approximately 95% once clerical research operations are completed for non-responding schools.


B.2.1.2 Questionnaires for Former and Current Teachers (TFS-2 and TFS-3)

Using the teacher statuses provided by schools on the Teacher Status Form (NTPS-5A/5B/5C) or collected via clerical research, Census Bureau staff will select the sample of teachers for the 2024-25 TFS and prepare materials for data collection.


On the 2023-24 NTPS Teacher Questionnaire, teachers were asked to provide the following information:


Name

Home address

Home email address

Work email address

Cell phone number

An option to opt out of receiving text messages

Home phone number


Prior to the start of TFS data collection, Census Bureau staff will conduct research to obtain contact information (e.g., home address, email address, and/or telephone number) for teachers who provided incomplete or no personal contact information on their completed NTPS questionnaire. Staff will use partial address information provided on the NTPS questionnaire to attempt to complete the address of the sampled teacher. In addition, staff will use sources such as the white or yellow pages, internet searches, and other address and phone number databases to attempt to collect missing contact information.


All teachers with a “mover”, “leaver”, or unknown occupation status ("unknown" teachers) are included in the TFS. Data collection for movers, leavers, unknowns will begin in the fall with an early email invitation asking them to participate in the 2024-25 TFS, unless a valid home email address is unavailable. This email invitation will include text thanking the teacher for their participation in the NTPS, along with a link to the TFS survey and their login credentials. Teachers receiving this email will be promised a $20 incentive if they complete the TFS online within two weeks ("early bird incentive"). One day before the completion deadline, teachers who have not yet completed their TFS online will be sent a reminder email, extending the completion deadline one additional week. Therefore, movers, leavers, and unknown teachers who complete their TFS online within 3 weeks of the initial invitation email will receive a $20 incentive after they complete their TFS. This early contact strategy described above will be administered in two batches, with the first in late October and the second in early December. Nonrespondents will be included in the Main TFS Contact Operations, beginning in January 20252.


Teachers sampled as "stayers" will begin data collection in January 2025, once the TFS Main Contact Operations begin. The first contact will be an initial email invitation to participate (unless a valid email address is unavailable) with a link to the survey and their login credentials. For the "mover, "leaver,”, and "unknown" teachers who did not complete their TFS during the early contact operations, this email contact will serve as a reminder email rather than an initial email invitation.


A few weeks after this email, a letter will also be sent by mail inviting teachers to participate in the survey and providing credentials for them to log in and complete the TFS online. If a home address was provided on the 2023-24 NTPS Teacher Questionnaire or was obtained during the research operation conducted prior to TFS data collection, then the package will be mailed to the teacher’s home address; otherwise, the NTPS school address will be used as the mailing address. All teachers will receive a $10 cash incentive in this initial mailed package. This $10 cash incentive will be adhered to a piece of yellow cardstock using removable sticky glue and will be inserted into the package along with the letter. The cardstock will be ½ sheet rather than a full sheet and will include text thanking the teacher for their participation in the study. Adhering the cash to the cardstock insert will (1) increase the weight of the envelope, making it feel more “substantial” and important; (2) prevent the money from free-floating inside the envelope; and (3) help ensure that the respondent notices the cash.


Each letter and email sent throughout TFS data collection will be customized and will provide the sampled teacher with login information to access the web-based instrument.


In late January, Census will send nonresponding teachers a first email reminder, followed by a second mailout letter in early February. A second reminder will be sent by email only in mid-February.


A third reminder by email will be sent to nonresponding teachers in early March, followed by a third mailout letter in mid-March. A fourth reminder by email only will be sent in late March.


In mid-April, nonresponding teachers will be sent a fourth mailout letter with web instrument login credentials and a paper copy of the appropriate TFS questionnaire (if current or former teaching status is known) or both TFS questionnaires (if teaching status is not known). A contingency incentive may be offered alongside the fourth mailed package to teachers in low-responding subdomains, as identified jointly by Census and NCES. A fifth and sixth reminder by email will be sent to nonresponding teachers in mid- and late April, respectively.


A fifth mailout letter with another paper TFS questionnaire(s) will be sent to nonresponding teachers in mid-May, followed by a seventh reminder by email in late May and an eighth reminder by email in early June. Data collection will end in early July 2025. Throughout data collection, research will be conducted, as needed, to find current addresses and emails for sampled teachers whose letters and emails are returned as undeliverable.

From February through May, outbound telephone calls will be made to remind teachers who have not yet responded to the web-based collection to complete their form. While the call is intended to encourage teachers to log on and respond to the survey themselves, interviewers will be able to administer it over the phone at the request of the respondent. Throughout the data collection period, respondents who request a paper questionnaire will be mailed the appropriate paper questionnaire(s) along with a postage page return envelope.


A response rate of approximately 68 percent (base-weighted) can be expected from leavers after follow-up, and a response rate of about 81 percent (base-weighted) can be expected from stayers and movers after follow-up; this is based on results from the prior administration of TFS and also accounts for an expected increase in the response rate due to the early mover and leaver contact strategy planned for the 2024-25 TFS. Table B6 shows weighted unit response rates in the last administration of TFS, TFS 2021-22.


Table B6. Weighted unit and overall response rates of TFS teachers, using initial base weight, by sector and teaching status: 2021–22

Sector and teaching status

Weighted

2020-21 NTPS Teacher

Listing Form unit response rate

Weighted

2020-21 NTPS Teacher Questionnaire

unit response rate

Weighted 2021-22 TFS unit response rate

Weighted overall response rate1

Public

88.2

62.4

79.8

43.9

Current teachers

88.2

62.4

81.2

44.7

Stayers

88.2

62.4

82.1

45.2

Movers

88.2

62.4

75.4

41.5

Former teachers (Leavers)

88.2

62.4

65.8

36.2

Private

71.8

60.6

77.7

33.8

Current teachers

71.8

60.6

79.5

34.6

Stayers

71.8

60.6

81.7

35.5

Movers

71.8

60.6

77.1

33.5

Former teachers (Leavers)

71.8

60.6

64.9

28.2

1NTPS base weighted Teacher Listing Form response rate times NTPS base weighted teacher questionnaire response rate times TFS base weighted teacher questionnaire response rate.


B.2.2 2024-25 PFS Procedures for the Collection of Information


On the 2023-24 NTPS Principal Questionnaire, principals were asked to provide the following information:


Name

Home address

Home email address

Work email address

Work phone number

Cell phone number

An option to opt out of receiving text messages

Home phone number


PFS data collection will begin with the Principal Data Collection operation, which will involve contacting principals directly using the personal contact information they provided on the NTPS Principal Questionnaire; if incomplete or no contact information was provided, Census Bureau staff will conduct research to attempt to locate personal contact information. For principals who do not respond during the Principal Data Collection operation and principals for whom no personal contact information could be found, data collection will shift to contact the NTPS school as part of the School Data Collection operation. Each letter and email sent throughout PFS data collection will be customized and will provide login information to access the web-based instrument.


The Principal Data Collection operation will begin in January 2025 with either an email or a text message. Principals who provided a cellphone number on their completed NTPS Principal Questionnaire and who did not mark the checkbox3 to opt out of contact by text message will be included in the "text message" group; remaining principals with any personal contact information will be included in the "no text message group".


Within the "text message" group, there will be a further split into two experimental treatments to compare two different methods of completing the PFS when texting principals. Text message contacts for PFS will be either an interactive exchange with the principal or a texted web link. This experiment will help determine whether either of the text methods increase PFS response and whether one method performs better than the other. For additional details about testing the use of text messaging in the 2024-25 PFS, refer to section B.4.1 of this document.


There will be two text message treatment groups:


  1. A “text survey group” where interactive questions are sent via text message prior to contact by email or mail.

  2. A “text-to-web group” where text messages containing a direct link to complete the PFS online via a Qualtrics instrument are sent prior to contact by email or mail.


Both text groups will receive up to two text messages – an initial text and a reminder text (if they did not complete their PFS following the initial text). After the text messaging groups receive their first and second text messages in early and mid-January, respectively, nonresponding principals will then receive an initial email invitation with a link to the PFS and login credentials in late January. An initial contact letter with login credentials will be sent to nonresponding principals at their home address in early February, followed by another reminder email in mid-February. A second contact letter with login credentials and a paper copy of the PFS questionnaire (PFS-1(C/D)) will be mailed to nonresponding principals at their home address in late February.


The principals in the "no text message" group will be contacted on a similar schedule as those principals in the "text message" group, but with different types of contacts. If a home email address was provided on the NTPS questionnaire or was obtained from research, an initial email invitation with a link to the PFS and login credentials will be sent to that email address to collect their current occupational status. If a home email address is not available, the initial email invitation will be sent to the principal at their 2023-24 NTPS school email address. This email will be sent in early January, concurrent with the initial text message (for the "text message" groups).


In mid-January, Census will send nonresponding principals with a home email address a reminder by email to complete the PFS. If a home email address is not available, an initial contact letter with login credentials and a request to complete the PFS will be sent to the 2023-24 principal at their home address. In late January, nonresponding principals with a home email address will be sent a second reminder by email, while those without a home email address will be sent a first reminder by email to their NTPS school email address. These contacts are concurrent with the second text message and initial email invitation for principals in the "text message" group.


In early February, a first (if a home email address is available) or second (if a home email address is not available) mailout will be sent to nonresponding principals in the "no text message" group; the package will contain a letter with login credentials and a paper copy of the PFS questionnaire (PFS-1(C/D)4) and will be sent to their home address. This mailout is concurrent with the initial mail contact for principals in the "text message" group.


Principals who were nonresponsive during the Principal Data Collection operations described above will shift to the School Data Collection operation in mid-March. During this operation, the NTPS school will be contacted and asked to provide the current occupational status of the 2023-24 principal. The School Data Collection operation will have “early” and “late” collection dates for all contact materials. Principals for whom no personal contact information was initially available will be contacted at the NTPS school on the early dates, while nonrespondents from the Principal Data Collection operation will be contacted at the NTPS school on the late dates.


During School Data Collection, Census Bureau staff will mail an initial contact letter with a request to complete the PFS and login credentials to the 2023-24 NTPS school in early January. This web-based instrument will be slightly different than the instrument used for the Principal Data Collection operation as it will be tailored toward the school staff rather than the previous years’ principal. An initial email invitation with a link to the survey and login credentials will also be sent to the school in mid-January to collect the current occupational status of the 2023-24 principal.


Nonresponding schools will be sent a reminder letter with web login credentials along with a paper copy of the PFS-1 questionnaire (A/B)5 approximately three weeks after the initial mailing. A reminder email with a link to the PFS and login credentials will also be sent to nonresponding schools around the same time.


About a month after the second mailout and email, nonresponding schools will receive a telephone call during which telephone center staff will follow up to collect the occupational status of the 2023-24 principal. The telephone center staff will first ask school office staff for the occupational status of the 2023-24 principal. If the office staff is unable to provide the information, telephone center staff will ask to speak with the current school principal.


After the telephone follow-up operation, nonrespondents will be sent to the principal status research operation. During this operation, Census Bureau staff will conduct research via internet searches to attempt to obtain the current occupational status of the 2023-24 NTPS principal.


A response rate of approximately 90 percent is expected based on the prior administration of PFS.


B.3 Methods for Maximizing Response

This section describes the methods that NCES will use to secure cooperation, maximize response rates, and deal with nonresponse for the 2024-25 NTPS follow-up surveys. The entire survey process, starting with securing research cooperation from key groups and individual sample members and continuing throughout the distribution and collection of individual questionnaires, is designed to increase survey response rates. In addition, NCES believes that the following endorsements, stressing the survey’s importance in communication materials, and extensive follow-up with nonrespondents will further facilitate the overall success of the survey and enhance response rates.

B.3.1 Endorsements from Key Public and Private School Groups

The level of interest and cooperation demonstrated by key groups can often greatly influence the degree of participation of survey respondents. Endorsements are viewed as an important factor in soliciting cooperation from educators and education leadership.


NCES has secured endorsement for the 2023-24 NTPS and 2024-25 TFS and PFS from the following organizations or agencies:

AASA, The School Superintendents Association

American Association of School Librarians

American Federation of Teachers

American Montessori Society

American School Counselor Association

Association for Middle Level Education

Association for Supervision and Curriculum Development

Association of American Educators

Council of Chief State School Officers

Council of the Great City Schools

National Association of Elementary School Principals

National Association of Secondary School Principals

National Education Association

National Parent Teacher Association

American Association of Christian Schools

Association Montessori Internationale

Association of Christian Schools International

Association of Christian Teachers and Schools

Association of Military Colleges and Schools of the United States

Christian Schools International

Council for American Private Education

Council of Islamic Schools in North America

Education Department, General Conference of Seventh-Day Adventist Church

Evangelical Lutheran Church in America

Evangelical Lutheran Education Association

Islamic Schools League of America

National Alliance of Christian Schools

National Association for the Education of Young Children

National Association of Episcopal Schools

National Association of Independent Schools

National Association of Private Special Education Centers

National Catholic Educational Association

National Christian School Association

National Council for Private School Accreditation

National Independent Private Schools Association

Oral Roberts University Educational Fellowship

The Lutheran Church-Missouri Synod

United States Conference of Catholic Bishops

Wisconsin Evangelical Lutheran Synod



B.3.2 Stressing the Importance of the Survey and the Respondents’ Participation in it

Official letters will be used to motivate respondents to return the surveys. Respondent letters for both the 2024-25 TFS and PFS will be sent by the U.S. Census Bureau and signed by NCES senior leadership. Communications in the form of both letters and emails will be personalized with the teachers’ and principals’ names, which is expected to have positive effects on the survey response rates.


B.3.3 Offering a monetary incentive to current and former teachers

Results of previous administrations of the NTPS indicated that offering a prepaid teacher incentive significantly increased the final response rates for both public and private school teacher questionnaires (see OMB# 1850-0598 v.43, Appendix D). Due to these favorable results, NTPS 2023-24 offered all teachers a prepaid incentive at the first contact by mail. Additionally, some teachers in late sampled waves were also offered an additional incentive upon completion of their teacher questionnaire to include a teacher incentive for responding to the survey or a school-level incentive for achieving an overall sufficient rate of teacher response. As such, the 2024-25 TFS will include a monetary incentive for all current and former teachers, and these incentives will be offered nonexperimentally. Teachers identified as movers and leavers will be offered a promised $20 incentive for completing their survey online within 3 weeks of first being contacted by email; these teachers will receive their $20 incentive after they complete their TFS online. If these teachers do not complete the survey online within 3 weeks, both they and teachers identified as stayers will receive a $10 cash incentive in their first contact by mail.


Data on teacher retention and attrition are in high demand from researchers and policymakers. During the 2012-13 TFS, the response rate for private school teachers was problematically low, and NCES was unable to publish a TFS private school teacher data file. In 2016-17, after the 2015-16 NTPS, the overall unit response rate of the TFS 2016-17 was likely to be under 50 percent and therefore the entire collection was cancelled. In an effort to increase response rates for the 2021-22 cycle, a contingency incentive was used in later TFS mailings to combat low response, as identified in weekly status reports. Consistent with the prior cycle, the 2024-25 TFS may also offer an additional incentive, as a contingency, in later mailings to combat low response for teachers in specific subdomains, as identified jointly by Census and NCES.


During week 11 of TFS Main Contact Operations (3/19/2025), Census and NCES will jointly evaluate the response rates for the TFS-2 (former teachers) and TFS-3 (current teachers) and determine whether an additional monetary incentive should be offered to nonresponding teachers. Specifically, if the TFS-2 response rate is less than 58%, nonresponding former teachers will be eligible to receive the additional incentive, and if the TFS-3 response rate is less than 64.5%, nonresponding current teachers will be eligible to receive the additional incentive. Response rates will be evaluated both overall and by subdomain, for example, by school characteristic. Depending on whether response rates are problematic overall or only for certain subdomains, the additional incentive may be offered to all nonresponding teachers or to nonresponding teachers in select subdomain(s).


If executed, eligible teachers will be offered a promised $10 digital incentive, contingent on response to their TFS. It will be offered starting in mid-April at the time of the fourth mailout and in all contacts thereafter until the end of TFS data collection. The $10 digital incentive, an e-gift card, will be delivered to responding teachers by email upon Census’ receipt of their completed TFS, either via the TFS web instrument or paper questionnaire. The email will be sent to the email address entered by the respondent in the contact information section of the completed TFS questionnaire. It will thank respondents for their participation in the TFS and will include a personal URL to redeem the $10 e-gift card by selecting from a list of participating retailers.


B.3.4 Extensive Follow-up (by mail, email, and telephone) of Non-Respondents

For the 2024-25 TFS, the Census Bureau will use a variety of techniques to increase response levels including:


(a) Using multiple modes of contact and offering multiple modes of response— self-administered web-based, telephone interviewer-assisted web-based, and self-administered mail instruments, as needed, to maximize response levels.


(b) Allocating adequate time and resources to respondent tracking efforts to ensure that a high percentage of movers and leavers are successfully located and surveyed. A variety of techniques will be employed to locate survey respondents. Potential tracking sources include: (a) leads provided by school principals or their designees; (b) clerical operations using electronic databases and internet searches; and (c) post office for possible forwarding addresses.


To permit sufficient time to locate sampled teachers who left their teaching position in the previous year, including "movers" and "leavers," data collection will begin with an early TFS invitation by email during the fall of 2024. Additionally, to allow for ample contact opportunities and thorough follow-up for all TFS non-respondents, the Main TFS Contact Operations will begin in January 2025, allowing six months for all data collection activities related to teachers.


For the PFS, the Census Bureau will also utilize multiple survey modes – a web-based Qualtrics instrument, self-administered mail instruments, text messaging, and telephone, as needed, to maximize response levels. Nonrespondents will receive at least two reminders, either by mail, email, or text message (if applicable). Given the brevity of the instrument, we anticipate response rates at or above 90% with this approach, as with past cycles of the survey.


B.4 Tests of Procedures and Methods

NCES is currently considering options for tests of methods, materials, and procedures to be conducted as part of the 2024-25 NTPS Follow-up Surveys, with the goal to increase response in the 2024-25 collection. Each test is described in detail below.


B.4.1 Testing the use of text messages in PFS

For the 2024-25 data collection cycle of the PFS, NCES will explore two uses of text messaging for administering the PFS. This builds upon an experiment conducted in the previous NTPS and PFS collections. For the 2020-21 NTPS, about half of principals provided their cell phone number, about half of whom selected a checkbox to opt-in to future contact by text message. These principals were randomly assigned to different contact and survey administration modes for the 2021-22 PFS, and we found that response rates were higher when text surveys were added alongside paper questionnaires, and principals who were offered both modes completed the survey more quickly.

PFS data collection will begin with contacts to principals directly using the personal contact information they provided on the NTPS Principal Questionnaire. If the principal is not responsive, data collection will shift to contacts with the principal at the NTPS school.


To explore using text messaging as a contact method for PFS, the PFS data collection will include two treatment groups:

  1. A “text survey group” where interactive questions are sent via text message prior to contact by email or mail.

  2. A “text-to-web group” where text messages containing a direct link to complete the PFS online via a Qualtrics instrument are sent prior to contact by email or mail.


For group one, questions about current occupational status will be texted directly to the 2023-24 NTPS principal and their response will be collected via their text messaged reply. These interactive texts will implement a skip pattern where subsequent texts will ask follow-up questions based on the principal’s responses. For group two, the text message will invite the principal to complete the PFS online by providing the direct link to the Qualtrics web instrument in the text message. Results from this experiment will help determine whether a respondent is more likely to respond directly in an interactive text message exchange or via a texted survey web link.


In order to adhere to the necessary guidelines for texting per the Census Bureau’s Policy office, the 2023-24 NTPS Principal Questionnaires were updated to include a checkbox for opting out of contact by text message in the future alongside the respondent’s cell phone number. Principals who provided a cellphone number and who did not opt-out of receiving text messages are eligible for inclusion in the text messaging experiment. Based on preliminary data from the 2023-24 NTPS, we expect that approximately 25% of principals will meet these criteria. The treatment for each principal will be randomly assigned at the start of PFS data collection. As such, the random assignment should result in even sample sizes and a similar distribution of school types across the treatment groups. The experimental contact plan and sample sizes are displayed in Exhibit 1.



Exhibit 1: 2024-25 PFS Experimental Contact Strategy and Sample Sizes – Principal Data Collection Operation


Experimental Treatment

Estimated Sample Sizes

 

Planned Dates for Principal-level Contacts (PFS-1C/D)

1/7/2025

1/16/2025

1/27/2025

2/6/2025

2/13/2025

2/24/2025

Standard/Control Group (No Personal Email Available)

1,709 public school principals

380 private school principals

Mail Activity

 

1st Mail

 

2nd Mail

 

 

Email Activity

1st Email

 

2nd Email

 

 

 

Text Activity

 

 

 

 

 

 

 



 

 

 

 

 

 

Standard/Control Group (Personal Email Available)


4,882 public school principals

878 private school principals

Mail Activity

 

 

 

1st Mail

 

2nd Mail

Email Activity

1st Email

2nd Email

3rd Email

 

 

 

Text Activity

 

 

 

 

 

 

 

Text Survey Group

1,098 public school principals

210 private school principals

Mail Activity

 

 

 

1st Mail

 

2nd Mail

Email Activity

 

 

1st Email

 

2nd Email

 

Text Activity

Text 1

Text 2

 

 

 

 

 

Text-to-web Group

1,098 public school principals

210 private school principals

Mail Activity

 

 

 

1st Mail

 

2nd Mail

Email Activity

 

 

1st Email

 

2nd Email

 

Text Activity

Text 1

Text 2

 

 

 

 

Following data collection, analyses will be conducted at the treatment level (experimental group vs. control). The analyses examined include:

  • Response rate;

  • R-Indicators;

  • Average number of contacts; and

  • Median days to respond.

The response rate will be calculated for each treatment group and selected demographic domains and compared using significance tests for differences. To account for confounding variables, a model-based approach will also be calculated to determine what effect the text message(s) had on a case’s likelihood to respond, given that case’s unique characteristics.

Given the project sample sizes in the table above, a statistically significant difference between the no text message group with personal email available and any of the other two treatment groups will be found if the response rates between the two groups differ by:

  • 6.64% for public school principals (PFS)

  • 15.25% for private school principals (PFS)

  • 6.09% for all school principals (PFS)

R-indicators will be used to determine the overall balance of the respondent population, as well as within each experimental group. R-indicators will be calculated for the full sample, as well as variable-level and category-level partial R-indicators to determine which characteristics specifically are contributing to imbalance within the respondent population.

The average number of contacts and median days to respond across the experimental groups will be used as a proxy for timeliness of response. A reduction in the average number of contacts could be used to justify the use of one type of contact strategy over another. If cases within the experimental group receiving text messages respond in a more timely fashion, this could reduce the number of cases included in follow-up operations, allowing finite resources, to be spread across fewer cases.

B.4.2 Testing the impact of updated SPD 15

As noted in A.7, the NTPS collects data on teachers’ race and ethnicity. In order to minimize burden, the TFS does not typically include questions where respondents’ answers are unlikely to have changed since they completed the NTPS during the previous school year. The 2023-24 NTPS and earlier collections asked about race and ethnicity as defined at the time of collection.


In March 2024, the Office of Management and Budget (OMB) announced revisions to Statistical Policy Directive No. 15: Standards for Maintaining, Collecting, and Presenting Federal Data on Race and Ethnicity (SPD 15) and published the revised SPD15 standard in the Federal Register (89 FR 22182).


For the 2024-25 TFS, we will include the updated SPD 15 question on race and ethnicity. Specifically, as seen in Appendix B, the 2024-25 TFS will be using Figure 2 (89 FR 22194). The TFS race and ethnicity question will allow us to empirically measure how respondents answer the older and updated versions, and it will serve as a bridge in order to analyze trends in teachers’ race and ethnicity before and after the updated SPD 15 is added to the next NTPS collection. The TFS race and ethnicity question will solely be used for this empirical purpose; that is, the question(s) are not for production and will not be used to produce official statistics. The vast majority of TFS and NTPS respondents answer the surveys via the web instrument, and Figure 2 has better usability for web and mobile users than Figure 1 (89 FR 22193). In specific, on the 2023-24 NTPS, 87-89 percent of responses on the principal and school questionnaires, respectfully, came from the web and 94 percent of responses came from the web on the 2021-22 TFS. Further, because this is an addition to a previous data collection and we are not recalculating burden for this collection, Figure 2 has the advantage of providing exactly the level of data that will be of use for preliminary bridging calculations while not increasing burden to respondents through use of the more complex and potentially confusing Figure 1.


Note that when asking about race, the NTPS has previously listed response options in descending order of prevalence, rather than alphabetically. For the 2024-25 TFS, we plan to randomly assign teachers to either view the response options in alphabetical order or by descending prevalence. This will allow us to compare how teachers answer the older and updated versions, as well as whether this differs by the order in which response options are presented.



B.5 Reviewing Statisticians

Jinhee Hong and Alfred Meier of the U.S. Census Bureau, Carol Wan and Alex McMillan of the American Institute for Research (AIR), and Andy Zukerberg, Maura Spiegelman, and Julia Merlin of NCES reviewed and approved the TFS and PFS sample design and related matters for statistical quality, feasibility, and suitability to the overall objectives of the survey.

1 Elementary: school offers at least one of grades K to 4, and the number of elementary grades is greater than or equal to the number of middle grades; Middle: school offers at least one of grades 5 to 8, and the number of middle grades is greater than the number of elementary or secondary grades; Secondary/high: schools offer at least one of grades 9 to 11, with or without grade 12, and the number of secondary/high grades is greater than or equal to the number of middle grades; Combined/other – school includes all other combinations of grades, includes K–12 schools.

2 Nonresponding movers, leavers, and unknown teachers will be added to the main TFS contact operations in January 2025. They will receive all contacts described in this section, but the email number references for movers and leavers are advanced by two. That is, these teachers will receive their third email through the tenth email rather than their initial through eighth email, since these teachers will have already received two contacts by email during the early contact operations.

3 In order to adhere to the necessary guidelines for texting per the Census Bureau’s Policy office, the 2023-24 NTPS Principal Questionnaires included a checkbox for opting out of contact by text message in the future alongside the respondent’s cell phone number.

4 The “C” version is for public schools; the “D” version is for private schools.

5 The “A” version is for public schools; the “B” version is for private schools.

1


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File Titletfs omb supporting statement [Memorandum]
AuthorNCES
File Modified0000-00-00
File Created2025-01-15

© 2025 OMB.report | Privacy Policy