OMB memo requesting change

u. s. dEPARTMENT OF eDUCATION

institute of education sciences

National Center for Education Evaluation and regional assistance

to: OMB

from: Betsy Warner

subject: Request For Survey Revisions for: An Impact Evaluation of the Teacher Incentive Fund (TIF), (OMB# 1850-0876)

date: 2/27/13

As you may know, the Impact Evaluation of the Teacher Incentive Fund (TIF) Program is a congressionally-mandated, high profile randomized trial of the TIF program for cohort 3 (TIF3). Participating districts include grantees who proposed under the TIF3 grant competition in 2010. Data collection includes general program information and program implementation and outcome information. Specifically, the study includes data on students, teachers, and principals using the following approved data instruments: i) a district survey administered to all TIF3 grantees, ii) principal and teacher surveys as well as a district interview administered in schools/districts participating in the randomized control trial. IES obtained OMB approval (#1850-0876) for the data collection on 9/16/2011. IES is currently in the process of analyzing data from the first round of data collection and preparing the first report.

Based on responses from the first year of data collection and conducting analyses of those data, we think it is important to modestly revise the surveys in order to get the best program information possible for this evaluation in subsequent data administrations. In particular, respondents found it difficult to answer questions about the performance-based pay aspect of the program given how TIF grantees ended up structuring their programs. I am writing to request replacing the current approved surveys with modest changes that we believe will streamline the surveys and improve the information that we end up collecting. We also believe that these changes are consistent with the spirit and burden of the originally approved instruments (the district survey is slightly reduced burden). We argue that these are revisions to clarify and improve data collection, and do not constitute a new data collection effort.

I have included copies of the surveys in track changes and comment fields to provide detailed information about the proposed changes. In addition, below provides an overview of and justification for the proposed changes.

Overview of Proposed Revisions

Principal Survey. It is fairly easy to see in the track changes that revisions to the principal survey are fairly minimal. The most substantive changes are to elicit information about the various ways in which teachers can earn additional pay under TIF3 in a manner more understandable and consistent with the programs designed and implemented by the grantees. See section C, revised questions C6-C11 and D3-D12. The number of questions remain similar, thus we expect no change in burden.

Teacher Survey. The revisions in the teacher survey are also modest. Analogous to the principal survey, we propose revising the additional pay questions to be more consistent with how TIF programs are being implemented. We think that revised questions E1-E10 will better elicit information about the various ways in which teachers can earn additional pay under TIF3 in a manner more understandable and consistent with the programs. We also alter questions about the classroom observation aspect of the TIF program (C14-C17) to provide more useful information. The number of questions remain similar, thus we expect no change in burden.

District Survey. We greatly streamlined the district survey by deleting information about initial program design and initial implementation that was only relevant for the first survey administration (most of previous sections A and B). Instead we altered a subset of these questions to address analogous issues related to early implementation experiences and combined into now section A. We greatly streamlined prior sections C and D primarily to improve the clarity of the various additional pay questions, analogous to the principal and teacher survey revisions. Although these revisions involved similar content and questions, the most efficient sequence and structure for the questions differs from the prior survey. For this reason we simply deleted the entire sections (C and D) and show with comment fields the similarity of these new questions now in section B. Finally, we have included a minimal number of questions in the now section C to better understand some of the early implementation experiences. The number of questions have been reduced by about 1/3, and we anticipate the burden for a single administration to be reduced from a 30 minute survey to a 25 minute survey.

We needed to analyze results from the first round of data collection to know that revisions would be prudent. Unfortunately, the timing of learning what revisions are desirable relative to the timing of needing to administer the next round of data collection does not allow for a new data collection clearance. I feel strongly that our proposed survey revisions will lead to much better information going forward, and I hope you will agree that the content and burden associated with these instruments is consistent with existing clearance and respectfully request that these revised instruments be approved.

I would be happy to have a conversation with you or to answer any questions. In order to have sufficient time to conduct survey reprogramming (since the teacher and principal surveys are web-based), field the surveys, and have adequate time for respondent follow-up during this school year, we request approval by March 8, 2013. I apologize for the quick turnaround that this request requires, and I would appreciate hearing back from you as soon as possible.

3