Impact Study to Inform the Teacher and School Leader Incentive Program

OMB Control No.:

Expiration Date:

Mathematica Reference No.:

Appendix B

U.S. DEPARTMENT OF EDUCATION

Impact Evaluation to Inform the Teacher and School Leader (TSL) Incentive Program

PRINCIPAL SURVEY

Spring [2021/2022]

This questionnaire is part of the Design of an Impact Evaluation to Inform the Teacher and School Leader (TSL) Incentive Program, which Mathematica is conducting for the U.S. Department of Education. The questionnaire asks about the types and frequency of coaching, mentoring, and professional development occurring at your school; strategies for recruiting and retaining teachers; activities of teacher leaders; and your satisfaction. If you prefer to complete this survey by telephone or would like a paper copy of the questionnaire mailed to you, please call 1-xxx-xxx-xxxx. If you have any questions about the study or your school’s participation, email us at [STUDY EMAIL]@mathematica-mpr.com.

We would like you to know the following:

• The survey takes about 30 minutes to complete. When you finish, we will send you a $30 Visa gift card as a thank you.

• Your answers will be completely confidential; no information that identifies you, your school, your district, or your teachers will be reported. Your responses are protected from disclosure per the policies and procedures required by the Education Sciences Reform Act of 2002, Title I, Part E, Section 183. Mathematica will present the information collected as part of this study in an aggregate form and will not associate responses to any people who participate. We will not provide information that identifies you, your school, or your district to anyone outside the study team except as required by law. Your responses will be used only for statistical purposes. Any willful disclosure of such information for nonstatistical purposes, without the informed consent of the respondent, is a class E felony.

• Participation in the principal survey does not pose any special risks to you as a respondent other than accidental disclosure of information. Mathematica has safeguards in place to ensure respondents; confidentiality, including restricting access to survey data and separating identifying information such as principal and school names from survey responses. All Mathematica staff sign a confidentiality pledge, and all staff with access to identifiable study data have received clearance from the U.S. Department of Education and are subject to severe legal consequences for any breach of confidentiality. Any data that identifies you will be destroyed at the end of the study. If you have any questions about your rights as a research volunteer, contact HML IRB toll free at 1-800-xxx-xxxx and reference IRB number xxxxxxxxxx.

Click here to proceed if you have read and understand the above statements and agree to participate in the survey.

If you would like a copy of this disclosure statement, please contact Eric Zeidman at [email protected] or xxx-xxx-xxxxx.

This evaluation is authorized by Title II sections 2001-2002 and Title VIII section 8042 of the Elementary and Secondary Education Act of 1965 (20 U.S.C. 6301 et. seq.) as amended by the Every Student Succeeds Act (ESSA).

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is 1850-0950. The time required to complete this information collection is estimated to average 30 minutes, including the time to review instructions, search existing data sources, gather the data needed, and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202. If you have comments or concerns regarding the content or the status of your individual submission of this form, write directly to: U.S. Department of Education, Institute of Education Sciences, 550 12th Street, SW, Washington, DC 20202.

A. TEACHER LEADER ROLE

ALL= Principals of schools who do and do not have teacher leaders providing individual support to teachers.

Initial Screen

Throughout the survey we have included definitions for some frequently used terms. If you wish to access the definition from a specific question, click on the blue hyperlinked text and it will open in a new window.

Next Screen

These first questions ask about support staff who were based at your school (meaning they were not district staff that might have periodically visited your school) and were not regular classroom teachers responsible for their own classroom.

Support staff might include individuals who provide in-class or pull-out support, such as Title I, ESL, or special education teachers.

A1. Please indicate whether your school had school-based staff, who were not regular classroom teachers responsible for their own classroom, in any of these support roles during this school year.

For a definition of a specific support role, click on the role title.

	SELECT ONE PER ROW
School-based staff in a support role	YES	NO	DON’T KNOW
a. Resource specialist/coach	1 	0 	d 
b. Instructional specialist/coach	1 	0 	d 
c. Curriculum specialist/coach	1 	0 	d 
d. Data specialist/coach	1 	0 	d 
e. Other school-based support staff	1 	0 	d 
(STRING (NUM))

EACH ITEM WILL HAVE A BLUE HYPERLINK WHICH WILL OPEN THE FOLLOWING DEFINITIONS WHEN SELECTED

a. (provides instruction and services to students who have an individualized educational program and who are assigned to regular classroom teachers for the majority of a school day)

b. (helps teachers implement effective teaching strategies)

c. (helps teachers understand content standards, implement the current curriculum, and develop, understand and implement shared student assessments)

d. (helps teachers understand, analyze, and use data to inform instruction)

ASK IF ANY A1 ITEMS A-E = 1, OTHERWISE GO TO A4

FILL WITH RESPONSE OPTIONS FROM A.1 IF ANY OPTIONS A-E = 1.

A2. Please enter the number of full-time equivalent school-based staff who were not regular classroom teachers responsible for their own classroom and had these support roles at your school during this school year.

	Number of full-time equivalent school-based staff with this support role
a. Resource specialist/coach
b. Instructional specialist/coach
c. Curriculum specialist/coach
d. Data specialist/coach
e. [FILL FROM A1e]

aSK IF ANY A1 ITEMS A-E = 1, OTHERWISE GO TO A4

FILL WITH RESPONSE OPTIONS FROM A.1 IF ANY OPTIONS A-E = 1.

A3. Please indicate whether the school-based staff, who were not regular classroom teachers responsible for their own classroom, listed below conducted classroom observations AND provided feedback to teachers based on those observations as part of their responsibilities during this school year.

	SELECT ONE PER ROW
	YES	NO
a. Resource specialist/coach	1 	0 
b. Instructional specialist/coach	1 	0 
c. Curriculum specialist/coach	1 	0 
d. Data specialist/coach	1 	0 
e. [FILL FROM A1e]	1 	0 

ALL

A4. Did your school have one or more teacher leaders during this school year?

By teacher leader, we mean regular classroom teachers responsible for their own classroom who take on additional administrative or support responsibilities in their school. They may or may not be compensated with a new job title, a reduction in their classroom teaching time, additional pay, or some combination of these items. They regularly engage in these administrative or support activities, in addition to their own classroom teaching.

 Yes 1

 No 0 GO TO D1

NO RESPONSE M GO TO D1

PROGRAMMER: TEACHER LEADER WILL HAVE A BLUE HYPERLINK WHICH WILL OPEN THE FOLLOWING DEFINITION WHEN SELECTED

Teacher leaders are regular classroom teachers responsible for their own classroom who take on additional administrative or support responsibilities in their school. They may or may not be compensated with a new job title, a reduction in their classroom teaching time, additional pay, or some combination of these items. They regularly engage in these administrative or support activities, in addition to their own classroom teaching.

A4 = 1

A4a. How many teacher leaders were there at your school during this school year?

NUMBER OF TEACHER LEADERS

(RANGE 1-20)

NO RESPONSE M

A4=1

A5. Did any of the teacher leaders perform any of the following activities at your school during this school year?

Select all that apply

 Provided individualized (one-on-one) coaching to teachers at your school. Coaching includes observing a class, providing feedback, and other types of instructional support such as co-teaching, modeling a practice, and providing resources to support a teacher’s instruction. 1

 Supported teachers in a small group or team setting with other teachers (e.g. professional learning community or PLC) 2

 Supported school or district level professional development 3

 Other roles and responsibilities 99

Specify (STRING (NUM))

NO RESPONSE M

PROGRAMMER: “INDIVIDUALIZED (ONE-ON-ONE) COACHING” WILL HAVE A BLUE HYPERLINK WHICH WILL OPEN THE FOLLOWING DEFINITION WHEN SELECTED:

Coaching includes observing a class, providing feedback, and other types of instructional support such as co-teaching, modeling a practice, and providing resources to support a teacher’s instruction.

PROGRAMMER SKIP BOX: IF A5 NE 1, SKIP TO D1, ELSE CONTINUE TO A6

A5 = 1

The remaining questions in this section relate to teacher leaders who provided individualized (one-on-one) coaching to teachers at your school during this school year.

A6. How many teacher leaders provided individualized (one-on-one) coaching to teachers at your school during this school year?

NUMBER OF TEACHER LEADERS

(RANGE 1-20)

NO RESPONSE M

A5 = 1

A7. For which grade levels did teacher leaders provide individualized (one-on-one) coaching to teachers at your school during this school year?

This question focuses on grades K–6, even if your school includes grades beyond 6.

Select all that apply

 Pre-kindergarten or kindergarten 1

 1st grade 2

 2nd grade 3

 3rd grade 4

 4th grade 5

 5th grade 6

 6th grade 7

NO RESPONSE M

A5 = 1

A8. Did any of the individualized (one-on-one) coaching that teacher leaders provided to teachers at your school during this school year plan to focus on a specific subject (for example, math, English language arts, reading, and so on)?

 Yes 1

 No 0 GO TO A10

NO RESPONSE M GO TO A10

A8 = 1

A9. Which subjects did you plan to have teacher leaders focus on during the individualized (one-on-one) coaching this school year?

Select all that apply

 English, language arts, or reading 1

 Mathematics 2

 Science 3

 Social studies or history 4

 Special instruction for English Language Learners (ELL) or Limited English Proficient (LEP) students 5

 English as a Second Language (ESL) 6

 Other subject 99

Specify (STRING (NUM))

NO RESPONSE M

A5 = 1

A10. During an average week, how would you describe the number of hours of regular classroom instruction provided by teacher leaders who provided individualized (one-on-one) coaching, compared to other classroom teachers at your school who teach the same grade and/or subject?

 Teacher leaders provided more or the same amount of classroom instruction as other classroom teachers 1 GO TO A13

 Teacher leaders provided fewer hours of classroom instruction than other classroom teachers to allow them more time to perform their duties as a teacher leader 2

NO RESPONSE M GO TO A13

A10= 2

A11. During an average week, how many fewer hours of regular classroom instruction did a typical teacher leader who provided individualized (one-on-one) coaching provide in comparison to other classroom teachers at your school who teach the same grade and/or subject?

FEWER HOURS PER WEEK

(RANGE 1-40)

NO RESPONSE M

A10 = 2

A11a. Did someone else cover instruction on a regular basis in the classrooms of teacher leaders who provided individualized (one-on-one) coaching?

 Yes 1

 No, teaching schedules were changed so that teacher leaders’ classrooms meet for fewer hours than other teachers’ classrooms 0 GO TO A13

NO RESPONSE M GO TO A13

A11a = 1

A12. Who covered classroom instruction on a regular basis for teacher leaders who provided individualized (one-on-one) coaching?

Select all that apply

 Other classroom teachers at the school 1

 Formal substitute teachers 2

 Paraprofessionals 3

 Teachers without a designated classroom (i.e. Title I, ESL, special education teacher) 4

 Someone else 99

Specify (STRING (NUM))

NO RESPONSE M

A11a = 1

ONLY SHOW RESPONSE OPTIONS FOR SELECTIONS IN A12

A12a. Did staff receive additional compensation for covering classroom instruction on a regular basis for teacher leaders who provided individualized (one-on-one) coaching?

	SELECT ONE PER ROW
	YES	NO
a. Other classroom teachers at the school	1 	0 
b. Formal substitute teachers	1 	0 
c. Paraprofessionals	1 	0 
d. Teachers without a designated classroom (i.e. Title I, ESL, special education teacher)	1 	0 
e. [FILL FROM A12]	1 	0 

A5 = 1

A13. Did any teacher leaders who provided individualized (one-on-one) coaching to teachers at your school receive additional pay for taking on the role during this school year?

 Yes 1

 No 0 GO TO A15

NO RESPONSE M GO TO A15

A13=1

A13a. How many teacher leaders who provided individualized (one-on-one) coaching to teachers at your school received additional pay for taking on the role during this school year?

NUMBER OF TEACHER LEADERS

(RANGE 1-20)

NO RESPONSE M

A13=1

A14. On average, how much additional pay did a teacher leader who provided individualized (one-on-one) coaching to teachers at your school receive per year? Only include teacher leaders who received additional pay.

TOTAL AMOUNT OF ADDITIONAL PAY

(RANGE $1-$50,000)

NO RESPONSE M

A5 = 1

A15. Were any teacher leaders who provided individualized (one-on-one) coaching to teachers at your school required to participate in training to prepare for the teacher leader role they held this school year?

• Include training when the teacher first began the teacher leader role, regardless of when that training occurred.

• Include any ongoing training received after the teacher began the teacher leader role.

• Do not include training received before the teacher joined the school.

 Yes 1

 No 0

NO RESPONSE M

A5 = 1

A16. Did any teachers have to complete a formal application process at any point in time to be a teacher leader who provided individualized (one-on-one) coaching during this school year?

 Yes 1 GO TO

 No 0 GO TO A18

NO RESPONSE M GO TO A18

A16=1

A17. Did the formal application process require applicants to…

	SELECT ONE PER ROW
	YES	NO	DON’T KNOW
a. submit a cover letter?	1 	0 	d 
b. submit a resume?	1 	0 	d 
c. provide evidence of improving student achievement?	1 	0 	d 
d. provide references?	1 	0 	d 
e. participate in an interview?	1 	0 	d 
f. respond in writing to a short-answer or essay prompt?	1 	0 	d 
g. participate in a role play exercise?	1 	0 	d 
h. some other activity? (Please specify)	1 	0 	d 
(STRING (NUM))

A5 = 1

A18. Thinking about teacher leaders who provided individualized (one-on-one) coaching to teachers at your school for this school year, what factors were used to select them?

Select all that apply

 Measures of student achievement growth 1

 The district’s teacher evaluation system 2

 Ratings based on the teacher leader application process 3

 Results from interviews of teacher leader candidates 4

 Years of teaching experience 5

 Administrator’s knowledge of and experience with the teacher 6

 Input from current teachers or other school staff 7

 Other factors used to select teacher leaders 99

Specify (STRING (NUM))

NO RESPONSE M

A5 = 1

A19. Which teachers at your school did the teacher leaders provide individualized (one-on-one) coaching to during this school year?

Select all that apply

 First-year teachers 1

 Early-career teachers (Second or third year of teaching) 2

 Teachers new to the school 3

 Teachers who would be receptive to or who requested additional coaching 4

 Teachers implementing a new curriculum 5

 Teachers in the same grade level 6

 Teachers teaching the same subject 7

 Teachers in the same grade level and same subject 8

 Teachers with a certain number of years of teaching experience regardless of grade level or subject taught 9

 Low-performing teachers 10

 Teachers of low-performing students 11

 Other teachers 99

Specify (STRING (NUM))

NO RESPONSE M GO TO B1

PROGRAMMER SKIP BOX: IF TWO OR MORE RESPONSES ARE SELECTED AT A19, GO TO A20, ELSE GO TO B1

a19=TWO OR MORE RESPONSES SELECTED

if a19=three or more responses selected, fill “three”

A20. Of the teachers you indicated, please select, in order, which [THREE] types of teachers the teacher leaders provided the most individualized (one-on-one) coaching to during this school year.

PROGRAMMER: DO NOT ALLOW MORE THAN ONE SELECTION IN EACH COLUMN. IF ONLY TWO RESPONSES SELECTED AT A19, DO NOT SHOW THIRD COLUMN

	SELECT ONE PER COLUMN
	Provided most support	provided second most support	Provided third most support
[FILL SELECTED A19 RESPONSES]	1 	2 	3 
	1 	2 	3 
	1 	2 	3 
	1 	2 	3 
	1 	2 	3 

B. TEACHER LEADER TRAINING AND SUPPORT

A5 = 1

The next questions are about the training you received as a school administrator to support teacher leaders who provide individualized (one-on-one) coaching to teachers at your school.

B1. In preparation for implementing this teacher leader role at your school, did you receive any training?

• Include training that you received when your school first implemented this teacher leader role, regardless of when that training occurred.

• Include any ongoing training.

• If your school was already implementing the teacher leader role when you joined the school, include training you received after being hired as the principal of the school.

• Do not include training you received before joining the school.

 Yes 1

 No 0 GO TO C1

NO RESPONSE M GO TO C1

B1 =1

B2. How many hours of training did you receive to prepare for implementing this teacher leader role at your school?

HOURS

(RANGE 1–40)

NO RESPONSE M

A5 = 1

B3. During this school year, how often on average did you meet with each teacher leader who provided individualized (one-on-one) coaching to teachers about that person’s teacher leader role? These meetings could include the teacher leader reporting on their activities or opportunities for you to provide feedback to them.

Select one only

 Never 1

 Once or twice this year 2

 Monthly or several times per year 3

 Weekly or several times per month 4

 Daily or several times per week 5

NO RESPONSE M

C. ATTITUDES ABOUT TEACHER LEADERS

A5 = 1

The next questions are about the benefits of and challenges to having teacher leaders who provide individualized (one-on-one) coaching to teachers at your school.

C1. During this school year, to what extent did having teacher leaders who provide individualized (one-on-one) coaching to teachers at your school result in each of the following benefits?

		SELECT ONE PER ROW
Benefit of having teacher leaders	NOT AT ALL	TO A SMALL EXTENT	TO A MODERATE EXTENT	TO A GREAT EXTENT	NOT APPLICABLE
a. Improved overall classroom instruction at the school	1 	2 	3 	4 	NA 
b. Allowed teachers to receive regular feedback about their classroom instruction	1 	2 	3 	4 	NA 
c. Enhanced collegiality among school staff	1 	2 	3 	4 	NA 
d. Provided career advancement opportunity for teachers	1 	2 	3 	4 	NA 
e. Allowed me more time to work on other principal duties	1 	2 	3 	4 	NA 
f. Provided additional support for new teachers	1 	2 	3 	4 	NA 
g. Provided additional support for struggling teachers	1 	2 	3 	4 	NA 
h. Built leadership capacity in the school or district	1 	2 	3 	4 	NA 
i. Helped to recruit effective teachers	1 	2 	3 	4 	NA 
j. Helped to retain effective teachers	1 	2 	3 	4 	NA 
k. Other benefit (Please specify)	1 	2 	3 	4 	NA 
(STRING (NUM))

A5 = 1

C2. During this school year, to what extent was each of the following a challenge to having teacher leaders who provide individualized (one-on-one) coaching to teachers at your school?

			SELECT ONE PER ROW
Challenge to having teacher leaders	NOT AT ALL	TO A SMALL EXTENT		TO A MODERATE EXTENT	TO A GREAT EXTENT	NOT APPLICABLE
a. Offering enough additional pay to attract qualified applicants	1 	2 		3 	4 	NA 
b. Covering teacher leaders’ designated time set aside from classroom instruction for teacher leader activities	1 	2 		3 	4 	NA 
c. Identifying quality candidates for the teacher leader position	1 	2 		3 	4 	NA 
d. Other teachers’ receptiveness to the teacher leader	1 	2 		3 	4 	NA 
e. Balancing a teacher leader’s teacher and leadership roles	1 	2 		3 	4 	NA 
f. Ensuring that the process of selecting teacher leaders appeared fair to other teachers	1 	2 		3 	4 	NA 
g. Burnout among teacher leaders	1 	2 		3 	4 	NA 
h. Turnover among teacher leaders	1 	2 		3 	4 	NA 
i. Scheduling teacher leader’s activities (such as observing other teachers’ instruction)	1 	2 		3 	4 	NA 
j. Providing teacher leaders with support for effective coaching	1 	2 		3 	4 	NA 
k. Other challenge (Please specify)	1 	2 		3 	4 	NA 
(STRING (NUM))

D. TEACHER RECRUITMENT AND RETENTION

all

The next questions are about strategies to recruit and retain ALL teachers (not just teacher leaders) for your school.

D1. Did you recruit or interview teachers to work this year at your school?

 Yes 1

 No 0 GO TO D2

NO RESPONSE M GO TO D2

D1=1

D1a. When recruiting or interviewing teachers to work this year at your school, to what extent did each of the following features help you recruit teachers you believe will be highly effective to your school?

	SELECT ONE PER ROW
Feature of working at school	NOT AT ALL	TO A SMALL EXTENT	TO A MODERATE EXTENT	TO A GREAT EXTENT	NOT APPLICABLE
a. Opportunities to earn performance-based pay	1 	2 	3 	4 	N.A 
b. Opportunities for career advancement	1 	2 	3 	4 	N.A 
c. Opportunities for professional development	1 	2 	3 	4 	N.A 
d. The level of teacher involvement in school decision making	1 	2 	3 	4 	N.A 
e. Collegiality of teaching staff	1 	2 	3 	4 	N.A 
f. The school culture	1 	2 	3 	4 	N.A 
g. Opportunities to receive individualized (one-on-one) coaching from a teacher leader	1 	2 	3 	4 	N.A 
h. Other feature 1 (Please specify)	1 	2 	3 	4 	N.A 
(STRING (NUM))
i. Other feature 2 (Please specify)	1 	2 	3 	4 	N.A 
(STRING (NUM))

all

D2. During this school year, to what extent did each of the following features help you retain highly effective teachers at your school?

	SELECT ONE PER ROW
Feature of working at school	NOT AT ALL	TO A SMALL EXTENT	TO A MODERATE EXTENT	TO A GREAT EXTENT	NOT APPLICABLE
a. Opportunities to earn performance-based pay	1 	2 	3 	4 	N.A 
b. Opportunities for career advancement	1 	2 	3 	4 	N.A 
c. Opportunities for professional development	1 	2 	3 	4 	N.A 
d. The level of teacher involvement in school decision making	1 	2 	3 	4 	N.A 
e. Collegiality of teaching staff	1 	2 	3 	4 	N.A 
f. The school culture	1 	2 	3 	4 	N.A 
g. Opportunities to receive individualized (one-on-one) coaching from a teacher leader	1 	2 	3 	4 	N.A 
h. Other feature 1 (Please specify)	1 	2 	3 	4 	N.A 
(STRING (NUM))
i. Other feature 2 (Please specify)	1 	2 	3 	4 	N.A 
(STRING (NUM))

E. YOUR JOB SATISFACTION AND BACKGROUND

all

The next questions are about you.

E1. How would you describe your overall job satisfaction for this school year?

Select one only

 Very satisfied 1

 Satisfied 2

 Neither satisfied nor dissatisfied 3

 Dissatisfied 4

 Very dissatisfied 5

NO RESPONSE M

all

E2. How many years have you served…

Please round up to the nearest whole number and include the current school year.

PROGRAMMER: RANGE FOR GRID IS 0-99

	years of experience
	A.	B.
	In this school	In any school (including this school)
a. as a principal?
b. in any administrative position other than principal?
	In any elementary school	In any school (including elementary)
c. as a classroom teacher?

F. FEEDBACK AND CONTACT INFORMATION

all

F1. Please share any additional comments or feedback you have related to this survey here.

all

END. Thank you for completing this questionnaire. Please provide the mailing address to where we should send your $30 Visa gift card. If you do not provide an address, we will send it to you at your school address.

First name:

Middle initial:

Last name:

Street address 1:

Street address 2:

City:

State:

Zip: