Att_Revised TIF OMB Supporting Statement A response

This document has been prepared to support the clearance of data collection instruments for the evaluation of the Teacher Incentive Fund (TIF) Program. The Policy and Program Studies Service (PPSS) of the U.S Department of Education (ED) is conducting this evaluation. PPSS has contracted with SRI International and their subcontractors Berkeley Policy Associates and The Urban Institute to conduct the evaluation. In the introduction to the supporting statement, we provide a description of the TIF program and a description of the evaluation questions and study design. The current request is limited to telephone interviews and case study site visits. We also plan to develop a survey for teachers and principals, which we anticipate will be submitted in late 2009 or early 2010 as a revision to this collection. Finally, the TIF evaluation design includes a feasibility study, at the end of which we may decide to have SRI pursue an outcomes analysis. If we elect to conduct an outcomes analysis, we will submit the appropriate additional revision to OMB.

The Teacher Incentive Fund Grants Program

The TIF program, first funded in fiscal year (FY) 2006 with an appropriation of $99.0 million, was designed to spur an increase in the number and quality of educator performance pay compensation systems. The Teacher Incentive Fund is authorized in PL 109-149, the Appropriations Act of 2006. Discretionary grants are made on a competitive basis by the TIF program office, which is located in the Office of Elementary and Secondary Education (OESE) (See Appendix A).

Local education agencies (LEAs) and State education agencies (SEAs), either alone or in partnership with non-profit organizations, were eligible to apply for competitive grants in FY 2006 to develop and implement innovative performance-based compensation systems for public school teachers and principals in high need schools (including charter schools). Grantees were required by the 2006 appropriations legislation to “develop and implement performance-based teacher and principal compensation systems in high-need schools.” These systems were to “consider gains in student academic achievement as well as classroom evaluations conducted multiple times during each school year among other factors and provide educators with incentives to take on additional responsibilities and leadership roles.” High-need schools were defined as those with more than 30 percent of their enrollments from low-income families (based on free and reduced lunch eligibility or other State-approved poverty measures). The competitive priorities of the 2006 grant competition also stressed a commitment to sustaining the projects and recruiting and retaining qualified teachers in hard-to-staff schools and subject areas.

TIF has funded 34 grantees across the Nation, primarily LEAs but also include State agencies, individual schools, and non-profits (such as charter schools or charter school networks). Grantees vary in the number of eligible educators, from fewer than 100 to more than 10,000, and in the demographic composition of participating schools —although they generally serve schools with high concentrations of minority students and high proportions of low-income students (in order to be eligible for a TIF grant the school must have met the high-need threshold based on poverty measures).

In addition to the TIF program office at ED, TIF grantees are supported by the Center for Educator Compensation Reform (CECR). CECR is an ED-funded organization that raises national awareness about alternative and effective strategies for educator compensation reform and provides technical assistance to TIF grantees on a variety of issues including sustainability, implementation, and evaluation. CECR also hosts an annual grantee conference.

Teacher Incentive Fund Evaluation

The evaluation of the TIF program consists of: (1) an implementation study of the local TIF projects; (2) a study of the feasibility of conducting a rigorous assessment of the relationship between the TIF projects and the outcomes of interest (strengthening the education workforce and increasing student learning); and (3) an outcomes analysis of the program’s impact. The full outcomes analysis is proposed as an option, which will be exercised only if the projects have indeed been implemented and appropriate data is available to conduct the analyses.

The implementation study will include descriptions of: (1) the central features of the local TIF performance-pay projects, and (2) the extent to which the projects are actually being implemented. This study will help explain the relationships among project characteristics and system supports, and project outcomes. Data collection activities will be iterative, beginning with a telephone interviews with of key stakeholders in all 34 of the TIF sites, followed by two rounds of more in-depth case studies in a sample of sites. These activities are the focus of this OMB package.

In addition to the telephone interviews and case studies discussed in this request, we plan to develop a survey of teachers and principals that would provide data representative of these populations among TIF grantees. The survey would provide quantifiable implementation data and provide TIF participants’ and nonparticipants’ knowledge, beliefs, attitudes, and motivations critical to the success of compensation reform. The surveys would be distributed to a random sample of eligible principals and teachers and would represent a full range of program knowledge and experiences in each grantee program. We anticipate a survey component for the implementation study will be submitted as a revision to this collection in late 2009 or early 2010.

The feasibility study will determine if it is possible to conduct the necessary rigorous analyses for the outcome analysis. That question will be answered in part through the implementation study, which will assess the extent to which projects have actually been put in place. The feasibility study will also include an analysis of local grantee evaluations to ascertain the degree to which they meet certain selection criteria for quality and suitability for a rigorous meta-analysis/empirical synthesis report. In addition, study team members will be describing the pros and cons of a regression discontinuity and difference-in-difference design and assessing the availability of appropriate data at TIF sites and, where necessary, in comparison sites for each design option.

Based on the results of the feasibility study, the study team will propose a research design (or designs) to conduct two related outcomes analyses. Both of these optional tasks are dependent on the results of the feasibility and implementation studies. The first analysis will address project effects—the degree to which there is evidence that the performance-pay systems established by TIF grantees have had their intended effects on teacher and principal quality and supply and student achievement in high-need schools. The second analysis will describe grantee and program features that are correlated with effects and will help shed light on practices related to positive outcomes. If we decide to have SRI undertake the optional analyses of the effects of TIF, we will submit a revision to OMB describing the proposed data collection and analysis.

Finally, if exercised, the evaluation will conclude with an optional synthesis report that provides a thorough analysis of all quantitative and qualitative data gathered and fully addresses all evaluation questions. The audience for the report will be policy-makers and practitioners.

Evaluation Questions and Data Sources

The evaluation questions for this study include the following:

What are the main characteristics or components of local TIF performance pay plans in terms of strategies (rewarding educators who increase student learning, attracting effective educators to specific schools or content areas, and/or rewarding educator knowledge and skills), targets (principals and academic teachers; principals only, schoolwide and/or classroom), and sizes of awards (in absolute terms as well as percentage of teacher salaries and expenditures)?
To what extent are grantees implementing performance-pay systems as planned?
What system supports (planning and buy-in, clear communication, project and funding stability, adequate data systems, and alignment with other human resource policies) and broader contextual factors impede or enhance implementation of performance pay systems?
What evidence exists that the performance pay systems are being established in the local grantee sites? In terms of stakeholder satisfaction as well as the percentage of a district’s personnel budget that is used for performance-pay, what are the prospects for sustainability beyond the life of the grant?
What is the feasibility of assessing the outcomes (both educator and student-level) of incentive systems?
How successful are TIF grantees at attracting effective principals and/or teachers to high-need schools and hard-to-staff subjects? Are they more successful than similar schools or districts without performance-pay programs?
How effective are TIF grantees at retaining effective principals and/or teachers? Are these rates higher than those of similar schools or districts without performance-pay programs?
Do teacher and principal performance evaluations indicate improved performance relative to teaching and other key job-related tasks (in the case of principals, as measured by supervisor ratings and increased recruitment and retention of effective teachers in the schools)?
What are the direct or indirect effects of performance pay systems on student performance? If incentive systems increase achievement, does the effect occur because of changes in teacher behaviors (attendance, retention, classroom instruction, some combination) and, if so, which behaviors?
Do certain kinds of TIF systems appear to warrant further investigation, including research to improve the design and implementation of any promising projects at the local level?

Data Collection Activities

In this section, we describe the data collection activities related to the implementation study for which we are seeking OMB approval. The evaluation employs multiple methods, gathering both qualitative and quantitative data. Wherever possible, we rely on secondary data sources to reduce burden on project, state, district, and school personnel, and any new data collection will be planned with other data collection efforts in mind to avoid overlap. Because the outcomes analyses are optional tasks, those efforts are not included in this OMB package. Protocols for the telephone interviews and site visits are semi-structured to allow for the study team to appropriately customize the questions for each respondent. They have been developed to maximize understanding of the unique issues and reform strategies in each site and minimize any redundancy. Depending on the precise roles and responsibilities of each respondent and the data already available from each grantee, interviewers will adjust the protocols to ask only those questions which are appropriate to each respondent and for which researchers do not already have verified data.

Telephone Interviews

We intend to conduct telephone interviews in fall 2009 with key informants at all 34 TIF sites. The purpose of grantee telephone interviews is to verify grantees’ project descriptions and address questions about the implementation of the TIF program from a variety of perspectives. Respondents will include: TIF project staff, educators, senior administrative staff, and other stakeholders. Protocols for these semi-structured interviews have been developed so as to ask objective questions in a manner that is least burdensome for interviewees. Because TIF projects are designed to motivate educators, some questions ask for respondents’ opinions. These subjective data are critical because participants’ beliefs about the projects will shape the extent to which projects do or do not motivate participants.

Before beginning data collection, interviewers will receive a manual containing all materials relevant to case study data collection (e.g., lists of types of respondents, selection criteria for respondents, protocols, and available background documents for the grantees). The training will help team members develop a common understanding of the conceptual framework driving the evaluation, the purposes of the data collection, the protocol questions, and the analyses in which interview study data will be used. The training will also include a discussion in the variation amongst grantees (which range in size, urbanicity, organization type, and scope of projects) and the implications of that variation for selecting respondents and using the protocols consistently across grantees. This is discussed in more detail below.

Telephone interviews will last approximately 60 minutes and will be conducted by teams of two researchers. We will gather information from all sites on the following topics: context of participation in TIF; planning and project design; implementation barriers, challenges and successes; as well as a discussion of perceived outcomes (e.g. teacher practice, principal leadership, and student learning). Through interviews we will begin to learn some of the outstanding questions about implementation that can later be answered at the local level through site visits. The protocols for these interviews are included with this OMB package.

We will make digital recordings of the interviews for completeness and consistency in reporting, and will communicate with interviewees as needed to obtain complete data and address inconsistencies in responses if they occur. The interviews will then be coded using both the evaluation questions and themes detected across grantees and question categories to create site summaries. These summaries will be used for a cross-site analysis. The storage of electronic and hard copy interview materials will be secure according to guidelines described in the Assurances of Confidentiality section of this document.

Case Study Site Visits

We intend to conduct two sets of site visits to collect in-depth information on grantee practices. The first set of site visits will be conducted in year two of the evaluation (spring of 2010) and the second set of site visits will be conducted in the fourth year (spring of 2012). Twelve sites will be selected for each set of site visits and each visit will include three days of interviews by a two-person team of researchers to determine progress in implementation. Some of the same sites may be visited in both years of the site visits based on the criteria for inclusion outlined in Supporting Statement B. Depending on the sample, it would be possible to visit anywhere from 12 (if the same sites are selected in both samples) to 24 (no sites are included in both samples) different grantees over the course of the case study site visits. Similar to the informants for the telephone interviews, we will arrange in-person interviews to be conducted during these site visits with representatives from the local TIF project office, principals, a cross-subject and cross-grade sample of teacher participants, and other stakeholders as necessary to get a broader picture of grant activities.

The purpose of the site visits is to: (1) verify data gathered through the telephone interviews and program documents, and (2) to complement the interviews with information about stakeholder satisfaction and potential for sustainability that can best be gathered through direct observation and in-person discussion. Researchers will learn about the format and content of the funded activities, the types of activities teachers are participating in, the work teachers are expected to perform, and the objectives and goals teachers are expected to reach. In particular, site visits will be crucial in the event that subsequent outcome analyses reveal a significant influence of TIF on outcomes such as retention of effective teachers in high-need schools. By discussing stakeholder satisfaction and observing school climate in the TIF environment, we will gain tools for potentially explaining the correlation of TIF projects and improvements with teacher outcomes.

The grantees selected for the site visits will be selected so as to represent the diversity of TIF grantees’ approaches to developing and implementing performance-based compensation systems Depending on availability, at least four of the sites will be implementing projects where 50 percent or more of the TIF grant activities award is allocated to performance pay for improved student achievement; four sites primarily implementing broader forms of differentiated compensation (e.g., increased pay for teaching in specific schools or subjects); and four sites implementing the most comprehensive systems (combining awards based on improved student achievement with various compensated teacher professional development activities, e.g., new teacher career ladder projects; Teacher Advancement Program/TAP.

Sites will be selected using methods described in detail in Supporting Statement B.

Similar to the material review preceding the telephone interviews, site visitors will review information regarding the grantee before going into the field. Site visitors will receive a manual containing all materials relevant to case study data collection (e.g., lists of types of respondents, selection criteria for respondents, protocols, consent forms, and structured debriefing forms). The training will help team members develop a common understanding of the conceptual framework driving the evaluation, the purposes of the data collection, the protocol questions, and the analyses in which the case study data will be used. Before going into the field, site visitors will also review results of the telephone interviews and extant documents, including grantee applications, annual reports, research reports, and background materials on the district(s) and schools to be visited. The protocols for these site visits are included with this OMB package.

As with the telephone interviews, we will make digital recordings of the site visit interviews for completeness and consistency, and will communicate with interviewees as needed to obtain complete data and address inconsistencies in responses if they occur. The interviews will then be coded using both the evaluation questions and themes detected across grantees and question categories to create site summaries. These summaries will be used for cross-site analysis. The storage of electronic and hard copy interview materials will be secure according to guidelines described in the Assurances of Confidentiality section of this document.

Planning and Accounting for Grantee Variation

Based on the uniqueness of each TIF grantee as well as the dynamics of the districts or States in which they operate, our study methodology and protocols have been designed to maximize the quantity and quality of relevant information to be collected from each individual while minimizing the burden on their time.

As mentioned above, the 34 TIF grantees vary widely along a number of attributes—including the recipient (State, local education agency, or nonprofit), the geography and size (whole States, single schools, urban areas, rural areas) that have implications for the design of the study protocols. Our flexible protocols—partnered with highly trained interviewers—will allow for interviews that are tailored and appropriate for a given grantee’s experiences and project structure.

Both the number and nature of key informants interviewed and the positions they hold will also vary somewhat across projects based on the specifics of each grantee. For example, in a TIF site where the grantee is a State entity and the project includes multiple districts and many schools, there will likely be roles filled by a number of individuals in the project office. In other sites, such as in the case of a small network of charter schools, a single person may represent the project office. Likewise, the individual representing the grantee leadership may be a district superintendent if the grantee is a district or a principal if the grantee is a school. The breadth of involvement will also vary by site making the flexible protocols essential to the smooth running of the interviews. For example, the technical assistance providers may be integrally involved in the projects of some grantees while playing a smaller role in others, local unions have been more or less involved in specific sites, the scope and stage of evaluations vary, and in some sites there may have been little or no media coverage of the program. Protocols contain all of the questions that might be asked of any respondent with each role. Based on a given individual’s responsibilities, the interviewer will ask only the questions that apply to a particular individual.

Key informants at each site and for each role will be identified with assistance from the TIF program office, along with the grantee’s project leadership (most often the project director).

Before data collection begins, interviewers will review extant documents that have been submitted to the TIF program office, including grantee applications, annual reports, research reports, and background materials on the grantees. Interviewer training will focus on providing the interviewers the necessary background and information to customize each interview to account for variation in grantee characteristics as well as prepare interviewers to respond to and pursue unanticipated or interesting findings.

Instruments to be Cleared through this Submission

ED is requesting clearance for the following data collection instruments which are included with this submission.

Letters and Information Requests

1. Letter of Data Collection Notification

2. Award Structure and Payout Form

Telephone Interview Protocols

3. Telephone Interview Protocol: TIF Project Staff

4. Telephone Interview Protocol: Grantee-level Senior Administrative Staff

5. Telephone Interview Protocol: Educators in TIF Sites

6. Telephone Interview Protocol: Stakeholders in TIF Sites

Site Visit 1 Protocols

7. Site Visit 1 Protocol: TIF Project Staff

8. Site Visit 1 Protocol: Grantee-level Senior Administrative Staff

9. Site Visit 1 Protocol: Educators in TIF Sites

10. Site Visit 1 Protocol: Stakeholders in TIF Sites

Site Visit 2 Protocols

11. Site Visit 2 Protocol: TIF Project Staff

12. Site Visit 2 Protocol: Grantee-level Senior Administrative Staff

13. Site Visit 2 Protocol: Educators in TIF Sites

14. Site Visit 2 Protocol: Stakeholders in TIF Sites

Consent Form

15. Consent Form

II. SUPPORTING Statement For Paperwork Reduction Act Submission

A. Justification for the Evaluation of the Teacher Incentive Fund (TIF) Program

1. Necessity of Information Collection

The evaluation for which OMB clearance is requested is the first systematic study of the TIF grants program. Because TIF represents a substantial investment in teacher performance pay programs, this evaluation is crucial for (1) establishing whether the program is working as intended by Congress and (2) identifying which elements of TIF projects are most effective.

In addition to the telephone interviews and case studies requested at this time, it is anticipated that a survey component for the implementation study will be submitted as a revision to this collection in late 2009 or early 2010. Additionally, if the optional outcomes study is deemed feasible and exercised, we will submit the proposed study design as a revision to this collection in late 2010.

2. Use of Information

The ED will use the results of this data collection to inform a variety of stakeholders regarding the nature and outcomes of the TIF program. More specifically, the information will be used to:

Describe the main characteristics or components of local TIF performance plans;
Ascertain grantee implementation and prospects for project sustainability beyond the life of the grant;
Describe system supports and broader contextual factors that impede or enhance implementation of performance pay systems;
Inform Congress and other policy-makers with a general interest in the impact of alternative compensation on the educator and student outcomes; and,
Inform future reauthorizations of the TIF program.

The audience for this evaluation includes the ED, Congress, education policy-makers, K-12 administrators and teachers, and researchers.

3. Use of Information Technology

No online surveys are being conducted as part of the data collection activities covered in the current application. We anticipate that project directors in most grantees will send us an electronic file, stripped of all identifying information, in lieu of completing the Award Structure and Payout Form. Aside from that one data file from grantee project directors, technology will be used only by project staff in recording, analyzing, and reporting. We anticipate applying for a revision to the study within the next year for the proposed teacher and principal surveys and we may apply for a revision for outcomes analyses. We will discuss those separately if the proposal for surveys is approved or we elect to undertake the optional outcomes analyses.

4. Efforts to Identify Duplication

At each phase of the evaluation, efforts will be made to minimize and reduce the burden on respondents. Wherever possible, we rely on secondary data sources to reduce burden on project, State, district, and school personnel. These data collection activities will be designed to avoid redundancy and to build on the work of other researchers. We are familiar with ongoing research on pay-for-performance, as well as the specific evaluations of the TIF grantees, and propose to complement—rather than duplicate—existing data activities.

To minimize respondent burden we will employ three strategies to build on existing evaluations of TIF grantees. First, we will include local evaluators in our telephone interviews and gather existing evaluation reports in order to understand exactly what kind of data they are collecting and from whom. Second, we will work with the TIF program office and local evaluators to acquire data collection instruments used in the local evaluations. Third, we will review local evaluation reports to refine our interview protocols for each grantee in order to streamline our data collection with a focus on confirming findings without repeating questions already asked of informants. We will not request that local evaluators turn over raw and identifiable interview notes or transcripts because this would likely violate the evaluators’ confidentiality agreements.

We believe that the sponsorship by ED of the grantees and their local evaluations offers a special opportunity to avoid redundancy and reduce burden. As a condition of the program, all TIF Grantees are conducting local evaluations the results of which are reported to the TIF program office along with annual performance reviews. These data will be accessed by researchers prior to conducting original data collection. Because our evaluation includes both the documents and interviews with local evaluators, we have the opportunity to build on, rather than replicate, this existing work. We will also work closely with researchers from the Center for Educator Compensation Reform (CECR)—the technical assistance provider for the grantees. We will review reports produced by CECR. We will also interview CECR technical assistors as part of data collection for each grantee.

5. Methods to Minimize Burden on Small Entities

TIF grantees are generally local school districts, but also include State agencies, charter schools, and school networks. The evaluation strategy—outlined in other parts of this document—is to minimize the burden on respondents in evaluation activities by collecting as much relevant information as possible prior to conducting interviews and site visits, to refine protocols in advance of interviews to avoid redundancy, and streamline the data collection for each respondent.

6. Consequences if Information is Not Collected or is Collected Less Frequently

Failure to collect this information will prevent Congress and ED from obtaining evaluation data necessary to assess the impact of the TIF program on educator recruitment and retention and student achievement. Failure to collect this data would also impede progress in determining the components of effective educator compensation reform initiatives.

All participants are only involved in a limited number of data collection activities—at most, three rounds of data collection over the next four years of the evaluation: (1) telephone interviews with all grantees in fall 2009, (2) Site Visit 1 with a sample of 12 grantees in 2010, and (3) Site Visit 2 with a sample of 12 grantees in 2011. All 34 grantees will be included in the first round of data collection (Telephone Interviews). In the second round (Site Visit 1) 12 grantees will be selected. Twelve grantees will also be selected for the third round (Site Visit 2). Depending on the sample, it would be possible to visit anywhere from 12 grantees (if the same sites are selected in both samples) to 24 grantees over the two rounds of site visit data collection. A detailed discussion of the sampling plan for Site Visit 1 and Site Visit 2 is included in Supporting Statement B.

7. Special Circumstances

Not applicable.

8. Federal Register Comments and Persons Consulted Outside the Agency

A request for comment on the original proposed data collection activities and instruments was published in the Federal Register Volume 74, page 16,190 on April 9, 2009.

Federal Register Comments. No public comments have been received.

Consultation Outside the Agency and with Respondent Representatives. We have assembled a Technical Working Group (TWG) of individuals with expertise in implementation of teacher and principal incentive systems, including performance pay, teacher profession development, administrator (principal) compensation reform, and/or a strong quantitative research background and expertise in regression discontinuity and other quasi-experimental designs/methodology. The TWG is advising the evaluation on all matters related to study design, sample selection, instrumentation, data collection, and data analysis. The members of the TWG, and their titles and professional affiliations, are:

Teacher professional development

Suzanne Wilson

Director of the Center for the Scholarship of Teaching and Chair of the Department of Teacher Education
Michigan State University

Jennifer King Rice
Associate Professor, Department of Education Policy and Leadership
University of Maryland

Performance pay

Patricia King

Director of School Improvement

Minnesota Department of Education

Matthew G. Springer

Research Assistant Professor of Public Policy and Education

Vanderbilt University

Administrator Compensation Reform

Dan D. Goldhaber

Research Professor

University of Washington

Ellen B. Goldring

Professor of Educational Policy and Leadership

Vanderbilt University

Regression discontinuity or other quasi-experimental designs/methodology

Peter Schochet

Senior Fellow

Mathematica Policy Research, Princeton, NJ

Susanna Loeb

Director of the Institute for Research on Education Policy and Practice

Stanford University

In addition to the Technical Working Group, we consulted with TIF program officers from the U.S. Department of Education.

9. Respondent Gifts

We will not make any monetary gifts or payments to participants in data collection activities.

10. Assurances of Confidentiality

We have established a set of standards and procedures to safeguard the privacy of participants and the security of data as they are collected, processed, stored, and reported. In an initial letter of invitation from ED, we will tell respondents that participation is voluntary and that we will assure the following:

Responses to this data collection will be used to summarize findings in an aggregate manner (across groups of grantees), or will be used to provide examples of program implementation in a manner that does not associate responses with a specific site or individual. In the report, pseudonyms will be used for each grantee.

The study team may refer to the generic title of an individual (e.g., "project director," or "high school teacher") but neither the grantee name nor individual name will be used. The contractor will not provide information that associates responses or findings with a subject to anyone outside the study team. We will aggregate data across respondent types (e.g., project directors, teachers) for most of the study analyses. In no case will we disaggregate data to such a degree that it would be possible to identify individual grantees or other entities.
We will acknowledge the cooperation of participating institutions in the final report, but will not identify them in the text of any report.
We will inform participants of the purposes of the data collection and the potential uses of the data collected. The risks and benefits of participating in this study, which are expected to be nominal, will be explained to potential participants before we begin the interviews. Informants’ consent will be actively requested and documented with a consent form.

We will educate project team members on respondents’ confidentiality assurances and how to handle sensitive materials and data. We will caution all persons assigned to the study not to discuss confidential data.
During training, we will reemphasize the need for interviewers and other data collection personnel to protect the privacy of respondents. We will caution such personnel not to discuss interview data with others outside the evaluation, and emphasize that they must restrict discussions within the evaluation team to the essential needs of the data collection activity.
We will disassociate names and addresses from the data as they are entered into the research team’s database and will use this information for data collection purposes only. As we gather information on individuals or sites, we will assign each a unique identification number, which we will use for raw data, print-out listings that display the data, and analysis files. We will also use the unique identification number for data linkage. We will not use any names, addresses, or other information that could connect the individual on any interview or case study protocols. The researchers will not provide information that associates responses or findings with a subject or district to anyone outside the study team.
We will store all electronic recordings of interviews, interview notes, and other project-related documents in secure areas that are accessible only to authorized staff members.
We will shred all interview protocols, forms, and other hard-copy documents containing identifiable data as soon as the need for this hard copy no longer exists.
We will duplicate all basic computer files on computer-based backup servers to allow for file restoration in the event of unrecoverable loss of the original data. We will store these backup files under secure conditions in an area separate from the location of the original data.

As the lead in the data collection of the evaluation, the research team will adhere to the Multiple Projects Assurance with the Office of Protection from Research Risks (OPRR) maintained by SRI. SRI's Assurance number is M-1088. SRI’s Human Subjects Committee is its official Institutional Review Board (IRB) charged with responsibility for the review and approval of all research involving human subjects. SRI clears all data collection protocols through its internal Human Subjects Committee as a safeguard to protect the rights of our research subjects. The National Evaluation of the Teacher Incentive Fund has already gained approval from SRI's Human Subjects Committee.

11. Questions of a Sensitive Nature

There are no questions of a sensitive nature included in data collection instruments or procedures. Participation in the study is voluntary and all data collection activities will be conducted as described above to assure respondent confidentiality. We will collect the names of participants only as needed to identify interview respondents internally.

12. Estimate of Information Collection Burden

Respondent burden for these interviews and site visits consists of the time spent participating in interviews. Respondents will not incur any equipment, postage, or travel costs. Project directors will spend limited additional time providing lists of teachers and partners and helping to arrange interviews.

Exhibit A1 displays estimates of the total respondent burden in hours and dollars. These time estimates are based on prior experience with site visits of this nature. On average, interviews will last approximately one hour depending on the participant. The annual reporting and record keeping burden over the three years of the study is with 393 respondents and 393 hours.

Telephone interviews with personnel at each of the 34 TIF project sites will average approximately 10 interviews per site, for a total of 340 participants. We will interview approximately 3 TIF project staff members (one project director at each site as well as other project staff as necessary based on the configuration of the grantee), 4 educators, 2 senior administrative staff and 1 stakeholder (e.g. representatives of partner organizations, representatives of parent organizations, teacher association officials).

Two sets of site visits will occur during the course of the study. In the first set of site visits, to be conducted in 2010, 12 grantees will be selected. Within each approximately 30 to 40 respondents will be interviewed per grantee for a total of approximately 420 participants.

In the second set of site visits, to be conducted in 2011, 12 grantees will be selected and within each approximately 30 to 40 respondents will be interviewed for a total of approximately 420 participants. Some of the same sites may be visited in both years site visits based on the criteria for inclusion outlined in Supporting Statement B. It would be possible to visit anywhere from 12 grantees (if the same sites are selected in both samples) to 24 grantees over the two rounds of site visit data collection.

Some participants (for example, project directors) will have participated in the preceding telephone interviews. During the site visits, interviews will be conducted with a variety of key informants including: TIF project staff (the project director at each site as other staff, technical assistance providers and evaluators), educators (approximately 15 teachers and 3 principals), 5 senior administrative staff (e.g., personnel directors, data management experts, superintendents of instruction), and 7 stakeholders in TIF sites (e.g. representatives of partner organizations, representatives of parent organizations, teacher association officials, and local education reporters). All numbers are approximate as respondents will be selected as needed to fully understand the unique experience of each grantee.

The Award Structure and Payout Form will be sent to project directors at each of the grantees prior to conducting telephone interviews and site visits to provide the least burdensome method for providing detailed award information. Based on our conversations with the TIF program office, we understand that grantees have this type of information as part of their regular reporting and the format in which we are requesting it should be relatively easy for them to provide.

Exhibit A1

Respondent Burden

	Number of Respondents	Time per Response (hours)	Total Hours	Average Hourly Wage	Total Cost
Telephone Interviews –2009
TIF Project Staff*	102	1	102	$40	$4,080
Educators in TIF Sites	136	1	136	$40	$5,440
Grantee-level Senior Administrative Staff	68	1	68	$60	$4,080
Stakeholders in TIF Sites	34	1	34	$40	$1,360
Site Visit 1—2010
TIF Project Staff*	60	1	60	$40	$2,400
Educators in TIF Sites	216	1	216	$40	$8,640
Grantee-level Senior Administrative Staff	60	1	60	$60	$3,600
Stakeholders in TIF Sites	84	1	84	$40	$3,360
Site Visit 2—2011
TIF Project Staff*	60	1	60	$40	$2,400
Educators in TIF Sites	216	1	216	$40	$8,640
Grantee-level Senior Administrative Staff	60	1	60	$60	$3,600
Stakeholders in TIF Sites	84	1	84	$40	$3,360
Total	1,180		1,180		$50,960

Note: The annual average number of respondents is 393. The annual average burden hours is 393.

*Project directors will be sent the Award Structure and Payout Form prior to their interview, however, we believe that most project directors (>90%) will elect to send us existing data files (that the program office says they are required to maintain for monitoring) instead of completing this form. Because we anticipate that approximately three project directors will complete the form, we estimate no additional burden for this data collection.

13. Estimate of Total Annual Cost Burden

There are no additional respondent costs associated with this data collection other than the hour burden estimated in item 12.

14. Estimates of Annualized Costs

The estimated annual cost to the federal government for the entire TIF evaluation, as specified in the contract, is displayed below. The study began in October 2008. The final report will be due in July 2013 (Month 57 of the program).

	Total Evaluation	Data Collection Costs	Telephone Interviews	Site Visits
Year One: FY 2009	$775,997	$122,778	$90,361	$32,417
Year Two: FY 2010	$638,003	$435,136	$152,573	$282,563
Year Three: FY 2011	$277,241	$33,279		$33,279
Year Four: FY 2012	$924,573	$313,865		$313,865
Year Five: FY 2013	$170,169	No data collection activities in year 5
Total	$2,785,983	$905,058
Average annual cost over three years of data collection		$301,686

We estimate that the average annual cost to the federal government for the activities covered under this OMB submission is $301,686. These activities will occur in FY 2009 thorough FY 2012 and will include collecting and analyzing interview and case study data, and reporting on results.

15. Change in Annual Reporting Burden

This request is for new information collection.

16. Project Time Schedule

We will conduct the case study tasks according to the schedule shown in Exhibit A2.

Exhibit A2

Schedule of Data Collection Tasks and Deliverables

Data Collection Activity	Data Collection Tasks and Deliverables	Time Period
Telephone Interviews	Conduct Telephone Interviews	October 2009- January 2010
	Draft Telephone Interview Results Memo	March 2010
	Final Telephone Interview Results Memo	April 2010
Case Study I	Conduct site visits for Case Study I	September – November 2010
	Draft Cross-Case Analysis Memo I	November 2010
	Final Cross-Case Analysis Memo I	January 2011
Case Study II	Conduct site visits for Case Study II	September – December 2011
	Draft Cross-Case Analysis Memo II	February 2012
	Final Cross-Case Analysis Memo II	April 2012

17. OMB Expiration Date

We will inform respondents about the OMB expiration date when they are notified about the study.

18. Exceptions to Certification Statement

No exceptions are requested.

Appendix A1. Legislation Authorizing the Teacher Incentive Fund Program

The Teacher Incentive Fund is authorized in P.L. 109-149 -- the Departments of Labor, Health and Human Services, and Education, and Related Agencies Appropriations Act, 2006, Title V, Part D.

Provided further, That $260,111,000 shall be available to carry out part D of title V of the ESEA, of which $100,000,000 of the funds for subpart 1 shall be for competitive grants to local educational agencies, including charter schools that are local educational agencies, or States, or partnerships of (1) a local educational agency, a State, or both and (2) at least one non-profit organization to develop and implement performance-based teacher and principal compensation systems in high-need schools:
Provided further, That such performance-based compensation systems must consider gains in student academic achievement as well as classroom evaluations conducted multiple times during each school year amount other factors and provide educators with incentives to take on additional responsibilities and leadership roles:
Provided further, That five percent of such funds for competitive grants shall become available on October 1, 2005 for technical assistance, training, peer review of applications, program outreach, and evaluation activities and that 95 percent shall become available on July 1, 2006 and remain available through September 30, 2007 for competitive grants.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	REQUEST FOR CLEARANCE OF DATA COLLECTION INSTRUMENTS FOR THE EVALUATION OF THE TEACHING AMERICAN HISTORY GRANTS PROGRAM
Author	Policy Division
File Modified	0000-00-00
File Created	2021-02-03