Districts Surveys and Interviews

OMB Control No.: 1850-0876

Expiration Date: 09/30/2014

TEACHER INCENTIVE FUND

2012-2013

DISTRICT SURVEY

This survey is part of the national evaluation of the Teacher Incentive Fund (TIF), a federally funded initiative on performance pay for educators. The survey includes questions about features of your program, changes made to your program after you received the TIF grant, and communication with program stakeholders.

We would like you to know that:

1. Your response is critical for producing valid and reliable data. You may skip any questions you do not wish to answer; however, we hope that you answer as many questions as you can. Your answers to questions will not affect your grant—now or in future—and will not be shared with anyone else other than the researchers. Participation in the district survey will not impose any risks to you as a respondent. If you have any questions about your rights as a research participant, you can contact New England IRB at 1-800-232-9570.

2. Responses to this data collection will be used only for statistical purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific district or individual. We will not provide information that identifies you or your district to anyone outside the study team, except as required by law. Additionally, no one in your district will see your responses.

Thank you very much for your help with this survey.

Please return the completed form to: Mathematica Policy Research 707 Alexander Road Building 3, Suite 304 Princeton, NJ 08540 ATTN: Lauren Greczyn

If you have questions about completing this form or to request another copy, please contact: Lauren Greczyn Survey Operations Center at 866-275-6052 (toll free) Fax: 202-863-1763 [email protected]

This survey is authorized by law: The ARRA, Division A, Title VIII, Pub. L. 111–5 and Departments of Labor, Health and Human Services, and Education, and Related Agencies Appropriations Act, 2010, Division D, Title III, Pub. L. 111–117.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 1850-0876. The time required to complete this information collection is estimated to average 25 minutes per respondent, including the time to review instructions, gather the data needed, and complete and review the information collected. If you have any comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: U.S. Department of Education, Washington, DC 20202. If you have comments or concerns regarding the status of your individual submission of this form, write directly to: U.S. Department of Education, Institute of Education Sciences, 555 New Jersey Avenue, N.W., Washington, DC 20208.

DISTRICT SURVEY

This survey is about activities supported by the 2010 Teacher Incentive Fund grant, also called TIF 3. Please answer only for the activities supported by the 2010 TIF 3 grant. Do not provide information for TIF grants received before 2010.

If you are responding for a TIF 3 program for a group of charter schools, for a single charter school, or a single-building program, please answer only for the school or schools that are part of your TIF 3 program.

If your district is NOT participating in the 2010 TIF 3 grant this school year, please check the box below and indicate the reason your district decided not to participate in TIF 3. Then proceed to Section D, provide your contact information, and return the form in the envelope provided. All TIF 3 grant participants should go to Question A1 below and complete the entire questionnaire.

1  Check this box if your district is NOT participating in the TIF 3 grant.

Please indicate the reason your district decided not to participate indicating whether your district ever participated, then GO TO SECTION D

_______________________________________________________________________ ________________

_______________________________________________________________________________________

_______________________________________________________________________________________

_______________________________________________________________________________________

_______________________________________________________________________________________

_______________________________________________________________________________________

A1.How do you normally refer to the TIF 3 performance-based compensation system (PBCS) program in your site?

1  The Teacher Incentive Fund program, the TIF program or TIF

2  The Teacher Advancement Program or TAP

3  Other (specify):________________________________________

A2. For each level listed, please provide the number of schools covered by the TIF 3 grant this school year.

	number of schools
a. Elementary schools	| | | |
b. Middle schools	| | | |
c. High schools	| | | |
d. Other (Specify)	| | | |
TOTAL NUMBER OF TIF 3 SCHOOLS (sum of rows A through D)	| | | |

A3. Besides TIF 3, are there other sources of funding from federal, state, or privately sponsored programs to provide performance-based bonuses in any of your TIF 3 schools?

1  Yes

0  No

Note: Questions A4-A11 focus specifically on bonuses based solely on performance. Performance-based bonuses do not include the additional pay that may be available for roles, responsibilities or other activities.

A4. Did any teachers in your TIF 3 schools receive bonuses based solely on their performance in the 2011-2012 school year?

1  Yes  GO TO A5

0  No  GO TO A6

A5.   Which of the following methods did you use to inform teachers and other stakeholders about the bonus amounts that were given based solely on performance in the 2011-2012 year?

	MARK ONE PER ROW
	yes	no
letter or email
a. Letter or e-mail to each teacher with his or her individual bonus amount	1 	0 
b.  Letter or e-mail with school- or district-level information about bonus amounts......................	1 	0 
in-person meetings
c.  Individual meeting(s) with each teacher to discuss his or her individual bonus amount	1 	0 
d.  Group presentation(s) to teachers with school- or district-level information about bonus amounts................................................	1 	0 
e.  Presentation(s) to school board	1 	0 
other
f. Press release or press conference about bonus amounts for individual teachers...............................................	1 	0 
g.  Press release or press conference with school- or district-level information about bonus amounts.....................	1 	0 
h. District website.....................................................................................	1 	0 
i. Other (Specify).....................................................................................	1 	0 
______________________________________________________

A6. Did you revise your TIF 3 program after the 2011-2012 school year to change any aspect of your bonuses based solely on performance?

1  Yes  GO TO A7

0  No  GO TO A12

A7. Did you make revisions to your TIF 3 program after the 2011-2012 school year to change who is eligible to earn a bonus based solely on performance?

1  Yes, to expand the pool of staff who are eligible to earn a performance-based bonus

2  Yes, to shrink the pool of staff who are eligible to earn a performance-based bonus

0  No

A8. Did you make revisions to your TIF 3 program after the 2011-2012 school year to change how performance is assessed or how the evaluation criteria affect bonuses based solely on performance?

1  Yes

0  No

A9. Did you make revisions to your TIF 3 program after the 2011-2012 school year to change the amount of bonuses based solely on performance?

1  Yes

0  No

A10. For which of the following reasons did you revise your bonuses based solely on performance after the 2011-2012 school year?

	MARK ONE PER ROW
	yes	no
a. To obtain or maintain principals’ support	1 	0 
b. To obtain or maintain support from a teachers union or association	1 	0 
c. To obtain or maintain support from teachers in participating schools	1 	0 
d. To obtain or maintain support from the school board	1 	0 
e. To stay within budget constraints	1 	0 
f. To simplify the criteria for earning a bonus	1 	0 
g. To improve perceived fairness	1 	0 
h. At the request of the U.S. Department of Education	1 	0 
i. Others (Specify)	1 	0 

A11. Did you revise the following aspects of your bonuses based solely on performance after the 2011-2012 school year?

1. Difficulty of earning a performance-based bonus

mark one only

1  Yes, so more teachers are likely to earn a performance-based bonus

2  Yes, so fewer teachers are likely to earn a performance-based bonus

0  No

2. Amount of the average bonus

mark one only

1  Yes, to have larger performance-based bonuses on average

2  Yes, to have smaller performance-based bonuses on average

0  No

3. Distribution of the bonus

mark one only

1  Yes, to have a larger difference between the average and maximum performance-based bonuses

2  Yes, to have a smaller difference between the average and maximum performance-based bonuses

0  No

A12. Tell us about the professional development activities related to your TIF 3 program that were planned for teachers in TIF 3 schools during the 2012-2013 school year.

These activities include courses they may take for recertification or advanced certification, workshops sponsored by your school(s) or district, conferences, or any other training that is relevant to their teaching at your school(s).

In each row, check the box if the activity is the focus of professional development (PD), then write the percent of teachers expected to receive PD in that topic.

	focus of pd	percent of teachers
a. Understanding components of your TIF 3 program other than performance measures	1 □	| | | |
b. Understanding performance measure used as part of your TIF 3 program	1 □	| | | |
c. Direct feedback based upon individual performance ratings as measured through your TIF 3 program	1 □	| | | |
d. Differentiated instructional strategies based on student assessments	1 □	| | | |
e. Instructional techniques and strategies	1 □	| | | |
f. Aligning curricula to state or district standards	1 □	| | | |

Questions in this section refer to activities funded by your TIF 3 grant.

Teacher Additional Pay and Performance Measures

B1. For which of the roles or activities below can a teacher in your TIF 3 schools earn additional pay this school year?

Mark Yes or No for each eligible role or activity listed. For each Yes answer, write the maximum amount of additional pay that a teacher could earn for that role or responsibility.

	can earn additional pay?		maximum additional amount
roles and responsibilities	yes	no
a. Mentor teacher	1 	0 	$ | | |,| | | |
b. Master or lead teacher	1 	0 	$ | | |,| | | |
c. Department chair or head	1 	0 	$ | | |,| | | |
d. Lead curriculum specialist	1 	0 	$ | | |,| | | |
e. Serving on a school-wide committee or task force	1 	0 	$ | | |,| | | |
f. Serving on a leadership team in some capacity	1 	0 	$ | | |,| | | |
other activities
a. Teaching in a hard-to-staff school (e.g., schools serving large proportions of economically disadvantaged or low-performing students)	1 	0 	$ | | |,| | | |
b. Teaching high-need subjects (e.g., math, science, or special education)	1 	0 	$ | | |,| | | |
c. Attending professional development activities or enrolling in graduate-level courses	1 	0 	$ | | |,| | | |

B2. Are any teachers in tested grades and subjects (grades/subjects that administer state or district assessments) in your TIF 3 schools eligible for bonuses based solely on their performance this school year?

• Performance-based bonuses do not include the additional pay that may be available for roles, responsibilities or activities listed in B1.

1  Yes

0  No

B3. For teachers in tested grades and subjects, which of the following performance measures are used to evaluate teachers or to determine their bonuses based solely on performance?

In column A, indicate whether the performance criterion is used to evaluate teachers.

In column B, indicate whether the performance criterion helps determine which teachers receive performance-based bonuses or the amounts of the bonuses that teachers receive.

	column a		column b
	used to evaluate teachers?		affects bonuses?
	yes	no	yes	no
student achievement
a. Student test scores at a point in time (e.g., average scores, proficiency rates, or Adequate Yearly Progress – AYP based on 2012-2013 scores only)	1 	0 	1 	0 
b. Growth in student test scores between the current and prior school year(s) at the school level (e.g., value-added, average gains)	1 	0 	1 	0 
c. Growth in student test scores between the current and prior school year(s) in certain student groups (e.g., grade level, team, subject area	1 	0 	1 	0 
d. Growth in student test scores between the current and prior school year(s) in teachers’ individual classes	1 	0 	1 	0 
direct observation
e. Classroom observations	1 	0 	1 	0 
f. Professional judgment of principal or other administrator, other than classroom observations	1 	0 	1 	0 
attendance
g. Teacher attendance	1 	0 	1 	0 
h. Student attendance	1 	0 	1 	0 
stakeholder input
i. Reviews from other teachers	1 	0 	1 	0 
j. Parent input	1 	0 	1 	0 
k. Student input	1 	0 	1 	0 
l. other (Specify)	1 	0 	1 	0 

B4. Are any teachers in non-tested grades and subjects (grades/subjects that do not administer state or district assessments) in your TIF 3 schools eligible for bonuses based solely on their performance this school year?

Performance-based bonuses do not include additional pay that may be available for roles, responsibilities or activities listed in B1.

1  Yes

0  No

B5. For teachers in non-tested grades and subjects, which of the following performance measures are used to evaluate teachers or to determine their bonuses based solely on performance?

In column A, indicate whether the performance criterion is used to evaluate teachers.

In column B, indicate whether the performance criterion is used to determine which teachers receive performance-based bonuses or the amounts of the bonuses that teachers receive.

	column a		column b
	used to evaluate teachers?		affects bonuses?
	yes	no	yes	no
student achievement
a. Student test scores at a point in time (e.g., average scores, proficiency rates, or Adequate Yearly Progress – AYP based on 2012-2013 scores only)	1 	0 	1 	0 
b. Growth in student test scores between the current and prior school year(s) at the school level (e.g., value-added, average gains)	1 	0 	1 	0 
c. Growth in student test scores between the current and prior school year(s) in certain student groups (e.g., grade level, team, subject area	1 	0 	1 	0 
d. Growth in student test scores between the current and prior school year(s) in teachers’ individual classes	1 	0 	1 	0 
direct observation
e. Classroom observations	1 	0 	1 	0 
f. Professional judgment of principal or other administrator, other than classroom observations	1 	0 	1 	0 
attendance
g. Teacher attendance	1 	0 	1 	0 
h. Student attendance	1 	0 	1 	0 
stakeholder input
i. Reviews from other teachers	1 	0 	1 	0 
j. Parent input	1 	0 	1 	0 
k. Student input	1 	0 	1 	0 
l. other (Specify)	1 	0 	1 	0 

B6. Are the TIF 3 schools conducting formal teacher classroom observations using a rubric or checklist this school year?

1  Yes GO TO B7

0  No GO TO B12

B7. Were observers trained on using the observation tool?

1  Yes GO TO B8

0  No GO TO B9

B8. Please indicate whether the observation training included the following activities.

	MARK ONE PER ROW
activity	yes	no
Discussed examples of performance at each rating level on the observation tool……………………………………………………………	1 	0 
Practiced using the observation tool using a video or in-person observation………………………………………………………………….	1 	0 
Compared observer ratings to an expert rater or gold-standard rating before conducting observations for TIF3…………………………………	1 	0 
Compared observer ratings to an expert rater or gold-standard rating while conducting observations for TIF3 …………………………………	1 	0 
Required additional training for observers who did not have similar ratings to the expert or a gold-standard rating. …………………………	1 	0 

B9. On average, how many formal classroom observations (based on a rubric or checklist) will be conducted per teacher this school year?

| | |OBSERVATIONS PER TEACHER

B10. What is (or will be) the approximate length of one typical formal classroom observation based on a rubric or checklist?

| | | | MINUTES PER OBSERVATION

B11.Who conducts or will conduct the formal classroom observations in TIF 3 schools this school year?

	MARK ONE PER ROW
	yes	no
a. The principal at the teacher’s school	1 □	0 □
b. Other administrator (e.g., assistant principal, department head) at the teacher’s school	1 □	0 □
c. Mentor, master, or lead teacher at the teacher’s school	1 □	0 □
d. Mentor, master, or lead teacher from a different school in the district	1 □	0 □
e. Content specialist	1 □	0 □
f. Other observer (Specify)	1 □	0 □

Principal Additional Pay and Performance Measures

B12. For which of the roles or activities listed below can principals in your TIF 3 schools earn additional pay this school year?

• Mark Yes or No for each role or activity listed. For each Yes answer, write the maximum additional amount that a principal could receive.

	can earn additional pay?		maximum additional amount
	yes	no
a. Working in a hard-to-staff school (e.g., schools serving large proportions of economically disadvantaged or low-performing students)	1 	0 	$ | | |,| | | |
b. Assuming additional roles or responsibilities in the school or district	1 	0 	$ | | |,| | | |
c. Attending professional development activities or enrolling in graduate-level courses	1 	0 	$ | | |,| | | |

B13. Are any principals in your TIF 3 schools eligible for bonuses based solely on their performance this school year?

• Performance-based bonuses do not include the additional pay that may be available for roles, responsibilities or activities listed in B12.

1  Yes

0  No

B14. Which of the following performance criteria are used to evaluate principals or to determine their bonuses based solely on performance?

In column A, indicate whether the performance criterion is used to evaluate principals.

In column B, indicate whether the performance criterion helps determine which principals receive performance-based bonuses or the amounts of the bonuses that principals receive.

	A		B
	used to evaluate principals?		affects bonuses?
	yes	no	yes	no
student achievement
a. Student test scores at a point in time (e.g., average scores, proficiency rates, or Adequate Yearly Progress – AYP based on 2012-2013 scores only)	1 	0 	1 	0 
b. Growth in student test scores between the current and prior school year(s) at the school level (e.g., value-added, average gains)	1 	0 	1 	0 
c. Growth in student test scores between the current and prior school year(s) in certain student groups (e.g., grade level, team, subject area)	1 	0 	1 	0 
direct observation
d. Observations by trained observers	1 	0 	1 	0 
e. Professional judgment of supervisors (e.g., superintendent), other than classroom observations	1 	0 	1 	0 
f. Teachers’ assessment of principal performance	1 	0 	1 	0 
attendance
g. Teacher attendance	1 	0 	1 	0 
h. Student attendance	1 	0 	1 	0 
stakeholder input
i. Parent input	1 	0 	1 	0 
j. Student input	1 	0 	1 	0 
k. other (Specify)
__________________________________________________

B15. Are trained observers conducting formal observations of TIF 3 principals this school year?

1  Yes GO TO B16

0  No GO TO B19

B16. On average, how many observations by trained observers will be conducted per principal this school year?

| | | OBSERVATIONS PER PRINCIPAL

B17.What is (or will be) the approximate length of one typical observation by a trained observer?

| | | | MINUTES PER OBSERVATION

B18.Which of the following types of trained observers will observe principals?

	MARK ONE PER ROW
	yes	no
a. Superintendent	1 	0 
b. Other central office administrator from the same district	1 	0 
c. Administrator from another district	1 	0 
d. Other observer (Specify)	1 	0 

B19. Please indicate which of the school staff below are eligible for bonuses based solely on their performance.

• Mark Yes or No to indicate whether staff in any of your TIF 3 schools are eligible to receive a performance-based bonus.

• Check NA for any staff category that does not exist in the schools in your TIF 3 program.

SCHOOL STAFF	eligible for bonuses?
	yes	no	n/a
a. Assistant/vice principal	1 	0 	n 
b. Other school administrators	1 	0 	n 
c. Other teaching staff (e.g., part-time teachers, substitutes, teacher aides)	1 	0 	n 
d. Non-teaching staff (e.g., counselors, librarians, custodial staff)	1 	0 	n 

C. Potential Challenges implementing your TIF 3 program

C1. For the listed activities associated with teacher classroom observations, please indicate the degree to which they were challenging for your district to implement.

	MARK ONE PER ROW
	not a challenge	minor challenge	major challenge
a. Choosing a classroom observation tool	0 	1 	2 
b. Finding a classroom observation tool that is ready for implementation	0 	1 	2 
c. Hiring observers for classroom observations Collecting and storing classroom	0 	1 	2 
d. Training observers to use a classroom observation tool	0 	1 	2 
e. Scheduling and/or conducting classroom observations Collecting and storing classroom	0 	1 	2 
f. Providing useful and/or timely feedback from observations	0 	1 	2 
g. Collecting and storing classroom observation data Collecting and storing classroom	0 	1 	2 

C2. For the listed activities associated with principal observations, please indicate the degree to which they were challenging for your district to implement.

	MARK ONE PER ROW
	not a challenge	minor challenge	major challenge
a. Choosing a principal observation measure/tool	0 	1 	2 
b. Finding a principal observation tool that is ready for implementation	0 	1 	2 
c. Hiring observers for principal observations Collecting and storing classroom	0 	1 	2 
d. Training observers to use a principal observation tool	0 	1 	2 
e. Scheduling and/or conducting principal observations Collecting and storing classroom	0 	1 	2 
f. Providing useful and/or timely feedback from observations	0 	1 	2 

C3. For the listed activities associated with incorporating student achievement growth into teacher evaluations, please indicate the degree to which they were challenging for your district to implement.

	MARK ONE PER ROW
	not a challenge	minor challenge	major challenge
a. Calculating student achievement growth	0 	1 	2 
b. Attributing student achievement growth to individual teachers	0 	1 	2 
c. Explaining student achievement measures to educators	0 	1 	2
d. Providing useful and timely feedback on student achievement measures to educators	0 	1 	2 
e. Collecting and storing data linking teachers to student achievement data	0 	1 	2

C4. For the issues listed, please indicate the degree to which they were challenging to your district when it was designing and implementing the TIF program.

	MARK ONE PER ROW
	not a challenge	minor challenge	major challenge
a. Defining the criteria for earning a performance-based bonus or the amount of the bonus	0 	1 	2 
b. Calculating performance-based bonuses	0 	1 	2 
c. Distributing performance-based bonuses	0 	1 	2 
d. Choosing educators for additional roles and responsibilities	0 	1 	2 
e. Sustainability of the TIF program	0 	1 	2

C5. To what degree were the following communication activities challenging for your district?

	MARK ONE PER ROW
	not a challenge	minor challenge	major challenge
a. Communicating the TIF program to educators	0 	1 	2 
b. Communicating bonus payouts to educators	0 	1 	2 
c. Communicating with other stakeholders	0 	1 	2 

C6. To what degree was it challenging for your district to obtain or maintain support for your TIF program from the following stakeholders?

	MARK ONE PER ROW
	not a challenge	minor challenge	major challenge
a. Teachers or teachers’ union or association	0 	1 	2 
b. Principals or principals’ union or association	0 	1 	2 
c. Superintendent	0 	1 	2
d. School board	0 	1 	2 
e. Parents or the broader community	0 	1 	2

C7.Please use the space below to provide any additional thoughts or clarifications you would like to share about your TIF 3 program or this questionnaire.

___________________________________________________________________________________

___________________________________________________________________________________

___________________________________________________________________________________

___________________________________________________________________________________

___________________________________________________________________________________

___________________________________________________________________________________

___________________________________________________________________________________

Please provide your contact information below. We will use it to reach you in case we need to clarify any of your responses.

Providing this information is voluntary.

Name:

Title:

Street address:

City: State: Zip Code:

Best telephone to reach you: ________________________________________________

Cell phone: _______________________________________________________________

Best Email to reach you: _____________________________________________________

Other Email: _______________________________________________________________

Best time to reach you by phone (day and time): _________________________________

THANK YOU VERY MUCH FOR COMPLETING THIS QUESTIONNAIRE