DISTRICT INTERVIEW PROTOCOL
TIF DISTRICT INTERVIEW PROTOCOL
Introduction
This is _________ from xxxxxxxxxxxx. As you know, The U.S. Department of Education has contracted with xxxxxxxxxx to conduct the national evaluation of the Teacher Incentive Fund. The information we will obtain from this interview is extremely important for us to fully understand your TIF program, your implementation experiences and challenges, how you met those challenges, and to learn about other district improvement efforts. This interview should take about 45 to 60 minutes. Before we proceed, I would like to confirm that this is still a good time for you.
(If a good time, proceed with text below; if not, ask: when would be a good time to call? On what date? Then repeat time and date to respondent.)
We would like you to know that:
As a requirement of receiving the TIF grant, your district has agreed to be included in the TIF evaluation. You are being asked to participate in the district interview because it is a component of the evaluation. While cooperation with this activity is a condition of your grant, we appreciate your time.
Your responses are critical for producing valid and reliable data. You may not respond to any questions you do not wish to answer; however, we hope that you answer as many questions as you can.
Your responses will not affect your grant—now or in future.
We will keep your responses confidential. The data you provide will be used only for statistical purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific district or individual. We will not provide information that identifies you or your district to anyone outside the study team, except as required by law. Additionally, no one in your district will see your responses.
Participation in the district interview will not impose any risks to you as a respondent other than accidental disclosure of information. Safeguards are in place to ensure individual respondent’s confidentiality including restricted access to interview data and separating identifying information such as name and school from interview responses. All <contractor> staff sign a confidentiality pledge and all staff with access to study data have received clearance from the U.S. Department of Education and are subject to severe legal consequences for any breach of confidentiality.
If you have any questions about this study, please contact xxxxxxxxxxxxxx.
In preparation for the interview, we have refered to your district survey or other program documents. We may ask for confirmation of this information during the interview.
Program name:
District Name:
Date:
Person interviewed:
Name:
Title:
Phone number:
Email:
Interviewer name:
Background and Context
First, I’d like to talk about any existing or additional performance pay programs in your district.
Before the 2010 TIF grant, did your district have performance pay program(s) for teachers or principals?
If yes, ask the following questions.
Is the program still in operation?*
If no: When was it discontinued?
Why was it discontinued?
During the current school year, are there federal, state, district, or privately sponsored programs, OTHER than TIF, that offer additional compensation to teachers and/or principals in your district?
If yes, ask the following questions.
What are the names of the programs?
What are the sources of funding?*
For what reasons or factors can educators receive additional compensation? (e.g., high effectiveness, extra roles and responsibilities, teacher in high-need schools, etc.)
Who is eligible for additional compensation? (e.g., teachers, principals, non-teaching staff)
How much extra compensation can teachers and/or principals receive?
Are schools required to participate, or can they opt out?*
How many schools participate?*
Are any of these schools participating in the 2010 TIF (or TIF 3) grant?*
If yes: What are the names of the schools?
Have any schools stopped participating in TIF during the period of the grant?
If yes, ask the following questions.
How many schools discontinued their participation in TIF?
What were the primary reasons for discontinuing participation in TIF (lack of principal support? Lack teacher support? Other?)
Characteristics of TIF Program
I’d like to learn about the eligibility criteria for schools, teachers, and principals in the TIF program.
Do teachers or principals have to pass any eligibility requirements to be part of the performance-based compensation program? (e.g., teach tested subject, full-time teacher)
If yes: What are these requirements? (Ask about anything besides what is listed above)
What grade levels/subjects are eligible?
Are non-tested grades/subjects eligible?
Do teachers or principals have to apply to be considered for a performance award?]
If yes: What are the application procedures and requirements?
Are teachers or principals required to do anything additional to receive a performance award (e.g., open their classrooms to observers)?
Next, let’s talk about classroom observations.
Are classroom observations used to evaluate teacher performance?
If yes: Describe the observation tool or rubric that is used. Is it commercially published or district created?
Describe the observations. How many formal observations are conducted annually? Informal? How long are the observations? Who conducts the observations?
Who trains the classroom observers?
Are observers required to meet reliability standards? If so, what is the process for meeting reliability (i.e. training to the gold standard, frequency of reliability assessment, quality assurance procedures)?
Are there any differences between the treatment and control schools for teachers in terms of the number of observations or feedback provided?
Are teachers able to review the rubric prior to observations?
Are teachers shown their rating after an observation?
Next I’d like to ask about evaluation criteria.
How do you measure effectiveness for teachers in tested grades/subjects?
What are the components of the evaluation system? (e.g. student achievement, observations, portfolio/teaching artifacts)
How many performance levels comprise the evaluation system? What is the definition of each level? (e.g. satisfactory, unsatisfactory)
What growth model is used? What assessment scores are used for elementary? Secondary?
Do you have different ways of evaluating teachers who teach in untested grades or subjects?
If yes: How do you evaluate teachers in untested grades or subjects?
Are there any differences between the treatment and control schools for teachers in terms of how effectiveness was measured?
How are principals evaluated in your program?
What are the components of the evaluation system? (e.g. student achievement, observations, portfolio/teaching artifacts)
Describe the principal observation process. Is there are tool or rubric that is used? How many formal observations are conducted annually? Informal? How long are the observations? Who conducts the observations?
Have there been any challenges about principal observations? If so, what?
Are there any differences between the treatment and control schools for principals in terms of the number of observations or feedback provided?
Are the performance levels for principals the same as those for teachers?
If No: What are the performance levels for principals?
Is the growth model used for principals the same as that used for teachers?
If No: What growth model is used for principals?
Are there any differences between the treatment and control schools for principals in terms of how effectiveness was measured?
Now let’s discuss performance pay that is based on the evaluation criteria.
Will performance-based bonuses or awards for teachers be based on one overall performance rating, separate ratings for each criterion, or something else?
If yes for overall rating: (focusing only on teachers in tested grades and subjects)
What criteria are included in this overall rating?
How do you weight the different criteria?
What is the maximum expected performance award you expect a teacher could receive?
What is the average, or “typical,” expected performance award you expect a teacher could receive?
If yes for separate awards for different criteria:
What are they?
What is the maximum expected performance award you expect a teacher could receive?
What is the average, or “typical,” expected performance award you expect a teacher could receive?
If yes for something else:
How are awards determined? (Could be the threshold model or something else)
What is the maximum expected performance award you expect a teacher could receive?
What is the average, or “typical,” expected performance award you expect a teacher could receive?
Will performance-based bonuses or awards for principals be based on one overall performance rating, separate ratings for each criterion, or something else?
If yes for overall rating:
What criteria are included in this overall rating?
How do you weight the different criteria?
What is the maximum expected performance award you expect a principal could receive?
What is the average, or “typical,” expected performance award you expect a principal could receive?
If yes for separate awards for different criteria:
What are they?
What is the maximum expected performance award you expect a principal could receive?
What is the average, or “typical,” expected performance award you expect a principal could receive?
If yes for something else:
How are awards determined? (Could be the threshold model or something else)
What is the maximum expected performance award you expect a principal could receive?
What is the average, or “typical,” expected performance award you expect a principal could receive?
What is the maximum salary a teacher in the district can make? Principal?
Can a teacher or principal appeal or dispute his or her performance pay or award amount?
If yes: Is there a formal appeals process for teachers or principals have such a dispute?
If yes: Please briefly describe this process.
How are bonuses for teachers handled if the teacher…
|
Circle one |
Notes (cut off dates, differences in award amounts, etc.) |
Gets hired after the beginning of the year at a TIF group 1 school? |
Bonus No Bonus |
|
Leaves a TIF group 1 school before the end of the year? |
Bonus No Bonus |
|
Transfer schools within the district from a TIF to a non-TIF school, or vice versa? |
Bonus No Bonus |
|
Transfer schools within the district from a TIF group 1 school to a group 2 school? |
Bonus No Bonus |
|
Is on extended leave? |
Bonus No Bonus |
|
Now let’s discuss roles and responsibilities in the TIF program.
What opportunities do you provide teachers and principals to earn additional pay through the TIF program? (e.g. leadership roles, teaching a high-need subject)
How much additional pay is available?
What criteria must be met to receive the additional pay?
What is the selection process for participation in these opportunities? (Probe on each as appropriate. Ask specifically if selection is based all or in part on whether the applicant has been rated highly effective by the current performance-based evaluation system.)
Are there any differences between the treatment and control schools for teachers or principals in terms of opportunities for additional pay, such as criteria or the number of opportunities?
I’d like to ask a little about professional development as part of the TIF program.
Do you provide professional development opportunities for teachers and/or principals to explain the measures that are used to determine effectiveness?
If yes: Who provides it, and briefly describe how it is done.
Are teachers compensated for professional development opportunities?
If yes: How much?
Are there any differences between the treatment and control schools for teachers or principals in terms of professional development opportunities?
22. Do you use the prior year’s (2013-14) performance evaluation results to select this year’s (2014-15) professional development (PD) offerings?
If yes: Briefly describe the process. Do PD offerings vary by school or for individual staff based on their performance results?
Recruitment, Hiring, and Transfer Policies
I’d like to learn about your district’s recruiting, hiring, and transfer policies.
What policies and practices does the district have for recruiting and hiring teachers to work in the district?
What policies and practices does the district have for transferring teachers within the district?
What is the last day in the school year that a transfer to another school is allowed?
How much input do principals have in recruitment, hiring, or transfer decisions?
Are there any differences between the treatment and control schools for teachers or principals in terms of:
Policies and practices the district has for recruiting and hiring teachers?
Policies and practices the district has for transferring teachers within the district?
Turnover of teachers or principals in treatment or control schools?
If yes: Was turnover more prevalent in T or C schools?
Was turnover last year different that previous years?
Which grade levels? Subject areas?
Did you conduct exit interviews to learn why these positions were vacated?
If yes: Why were these positions vacated?
If no: In your opinion, why were the positions vacated?
Were any of these vacancies because of dismissal?
Communication and Participant Support
I’d like to turn to talk about how your district has refined the design and implemented of your TIF program.
What modifications have been made to the original TIF program design this year (2014-15)?
Why were these modifications made?
Next I’d like to talk about communication about the TIF program.
When were school staff first made aware that they could receive bonuses for 2013-2014? [Month, Year] How was this communicated?
What techniques do you use to communicate information about the program and its benefits? Have you found a particular technique to be more effective than others? Why?
Is there a process for program participants to provide feedback about the program?
Are there any differences between the treatment and control schools for teachers or principals in terms of communication activities about the TIF program?
Technical Assistance
I’d like to discuss support you have received for implementing the TIF program.
What have been the two most useful technical assistance activities the district received as part of its participation in the TIF program?
For each activity: Who provided this TA? [If not Mathematica, ask about relationship with TA provider]
How often was this TA provided?
Challenges and Experiences
Let’s discuss the experiences and challenges your district has experienced.
What key successes did the district experience during the 2013-14 period of the TIF program?
What has been challenging so far about implementing the program? [Probe specifically on resistance to participation, communication, calculating awards amounts, determining effectiveness measures (explicit VA model or observation tool to use)]
Based on your experiences, would you have done anything differently?
If yes: Please describe.
Plans for 2015-16
Let’s discuss your plans for TIF policy or policy building off of your TIF experience in your district.
Do you plan to continue additional pay for teachers based solely on their performance?
If no, why not?
if yes – do you plan to retain the policy in a subset of targeted schools or expand the policy to more or all of your schools? Please discuss how TIF informed this choice.
Related to teacher performance evaluation measures:
Do you plan on using student achievement growth to measure teacher performance?
if yes – are there particular benefits or reasons for wanting this policy? Please discuss how TIF informed your choices related to the growth measure you selected (same or different from that used with TIF)
If no – are there particular concerns with or challenges to using this policy?)? Please discuss how TIF informed this choice.
Do you plan on using observations to measure teacher performance?
if yes – are you using the same rubric you used for TIF? Please discuss how TIF informed your choice related to the rubric you selected (same or different from that used with TIF). Also, please discuss how TIF informed your choice related to number and nature of observations you will be conducting.
If no – are there particular concerns with or challenges to using observations to measure teacher performance?
Do you plan on using measures other than student achievement growth or observations to measure teacher performance? Please discuss how TIF informed your choice.
Related to principal performance evaluation measures:
Do you plan on using student achievement growth to measure principal performance? Please discuss how TIF informed your choice.
Do you plan on using observations to measure principal performance? Please discuss how TIF informed your choice.
Do you plan on using measures other than student achievement growth or observations to measure principal performance? Please discuss how TIF informed your choice.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | District Interview |
| Author | Libby Makowsky |
| File Modified | 0000-00-00 |
| File Created | 2021-01-27 |