Response to Public Comments from the 60-day Federal Register Notice

Appendix D

Appendix D.

Evaluation of Teacher Residency Programs: District Perspective

Response to Public Comments from the 60-day Federal Register Notice

This attachment summarizes the responses to public comments received during the 60-day public comment period for the Evaluation of Teacher Residency Programs: District Perspective.

A 60-day notice to solicit public comments was published in the Federal Register, Volume 89, No. 191, pages 80232-80233 on October 2, 2024. Three public comments were received, and two were substantive. The first substantive comment was from the National Council for Teacher Residencies, and the second was from the National Education Association.

The full text of the two comments is available through www.regulations.gov by searching the Docket ID number ED-2024-SCC-0115-0001 or at the following link: https://www.regulations.gov/document/ED-2024-SCC-0115-0001.

The responses below describe revisions made to the District Data Form and TQP Grant Staff Data Form.

First substantive public comment, from the National Council for Teacher Residencies

The document with comments from the National Council for Teacher Residencies (NCTR) provides feedback on the study design and data collection. Below, we provide responses to each of NCTR’s points separately.

Use a shared vision and definition of teacher residency programs for the study. NCTR commented that a “shared vision and definition” of residency programs will be important for evaluating them. The study is grounded in a conceptual model of core recommended features of residency programs that will guide data collection and analyses. That conceptual model is based on two main sources: the publication by Pathways Alliance referenced by NCTR (“Towards a National Definition of Teacher Residencies”) and required features for residency programs included in the Teacher Quality Partnership (TQP) notice inviting applications. As a result, the study’s conceptual model for residency programs is aligned with the leading voices in the field and the TQP program.

Provide districts a more user-friendly way to submit the requested information. NCTR raised a concern about the burden of asking districts to separately report counts of new hires from residency programs. We are only requesting data on new hires from residency programs if districts have existing administrative data that can be used to produce these counts. We pilot tested the data workbook with four districts and found that it required one to two hours to complete. If a district cannot provide counts of new hires from residency programs, we will request that information from the TQP grantee. This request should not be an excessive burden for TQP grantees because they already track information on program completers for purposes of reporting the Government Performance and Results Act (GPRA) measures.

Align terminology with Title II reporting. NCTR raised a concern about whether the study’s definitions of different teacher preparation pathways align with definitions used for Title II reporting. This data collection differs from Title II because we are only asking districts and programs to provide information about teacher residency programs. We are not asking for data separately by other types of preparation programs. Because we are requesting data on a pathway that is directly tied to a federal funding source (that has its own reporting requirements), preparation programs and districts did not raise any concerns about the study's definition of residency programs during the pilot.

Additional burden of requesting data for TQP-funded program completers. Federal Title II data generally do not separately identify teacher residency programs with the same institution as other types of preparation programs, so it is not possible to use those data to address the study’s research questions. As noted above, TQP grantees already track information on TQP program completers for purposes of reporting the GPRA measures. The GPRA measures require TQP programs to track whether program completers are hired and retained by districts. Although the GPRA measures differ from the measures used for this study, the TQP grantees can use the same data to report the counts being requested for this study. We piloted the data workbook with four grantees and found that providing the counts required no more than two hours.

Provide information on effectiveness of different fundings sources for residency programs. NCTR noted that the information requested for this study may help assess the effectiveness of programs funded through different sources. This aspect of the study is descriptive and will provide information on the extent to which partner districts hire teachers from residency programs who expand diversity and fill hard-to-staff positions. This information will be valuable for understanding descriptively how these programs support district hiring. However, it is beyond the scope of this study to provide information on the impact of these programs on teacher hiring or student achievement.

Produce study results in a timely way. NCTR noted that providing study results in a timely way is important for ensuring that they can inform continuous improvement efforts. We agree about the value of providing timely results and we will immediately begin conducting the analysis and preparing the study report once the data are collected.

Second public comment, from the National Education Association

The second comment is from the National Education Association (NEA). The document provides suggestions for revising the study’s data collection instruments. Below, the study team addresses each of these points separately.

Expanding study to include mentor teachers. NEA suggested that we consider adding an additional research question that addresses the role of mentor teachers. We agree about the importance of mentors, however this request is beyond the scope of the current study and would add burden to the data request.

Extending data collection to three years. NEA suggested expanding data collection to cover three years (the length of the residency program’s teaching commitment) instead of one year. We focused data collection on one year (the 2024-2025 school year) to minimize the burden of data collection on districts and grantees. We also think it is more feasible for districts to provide information on new hires than to provide historical information on the retention of teachers over time. While additional years of data could provide valuable insight into the retention of residency program completers, that is beyond the scope of the current study.

Specifying time to complete form and how long data will be stored. NEA suggested adding language to specify how long respondents have to complete the form and how long their data will be maintained. We revised the email notification for respondents (Appendix C) to request that they provide the data within three weeks, if possible. However, we will remain flexible and allow respondents to take more time if needed. Additionally, we added language to the “Instructions” tab of the data workbook to clarify that the data will be included in the restricted-use file produced by IES for the study. The new text explains that the file will be maintained by the National Center for Education Statistics and will be accessible only to approved researchers.

Concerns about a district-defined versus common definition of “hard-to-staff.” NEA expressed concerns about allowing districts to define which subject areas are hard-to-staff versus using a common definition. We agree with this concern and have revised our main analysis to focus primarily on the hard-to-staff subject areas required by the 2019 TQP grants: science, technology, engineering, and math (STEM). However, given that residency programs are expected to help address the hiring needs of partner districts, it will also be useful to also have information on whether the TQP programs provided new hires in the subject areas that districts viewed as hard-to-staff. To simplify, we revised the question to ask whether it has been difficult to fill vacancies in each subject area.

Proposal to collect information about extent to which programs provide teaching pathways in multiple certification areas. NEA suggested collecting data to assess the extent to which teacher residency programs prepare program participants for all content areas and grade levels. The study is collecting information on whether residency programs provided new hires who were more likely to teach each subject area. This information would allow us to examine whether residency programs are contributing new hires across multiple subject areas. However, because the 2019 TQP grant specifically called for preparing teachers in STEM subject areas, the study’s primary analysis will focus on the residency programs’ role in filling hard-to-staff subject areas including STEM. The analysis can also look at whether residency programs’ helped fill subject areas that are not viewed as hard-to-staff as well.

Mathematica^® Inc. 2