Case Studies of Current and Former Grantees Under the Title III National Professional Development Program (NPDP)
Supporting Statement for Paperwork Reduction Act Submission
Contents |
Introduction |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
References |
Exhibits |
1. Research Questions |
2. Map of Information to be Collected, by Research Topic and Respondent |
3. Expert Reviewers |
4. Estimate of Time Burden, by Respondent and Type of Grantee |
Introduction
The purpose of the National Professional Development Program (NPDP), administered by the Office of English Language Acquisition (OELA), is to support pre-service education and professional development activities to improve instruction for English Learners (ELs). Recipients of the grants are Institutions of Higher Education (IHEs) in consortia with states or school districts. Funded projects are designed both to increase the pool of highly-qualified teachers prepared to serve EL students and to increase the skills of teachers already serving this population.
The purpose of this study is to examine how a sample of NDPD grantees is implementing their grants with respect to four areas: (1) the content and structure of the education they provide to current and prospective teachers of English Learners; (2) the nature of changes they attempt to make to the full teacher education program at their institutions; (3) the efforts grantees make to institutionalize their projects so that they can be sustained after the grant ends; and (4) the efforts of grantees to follow up with former program participants. The research team will use information gathered on these four topics to identify issues that could be investigated in a larger, more representative study.
This study will consist of a literature review and case studies of 15 purposively-selected current grantees and 9 purposively-selected former grantees. The case study sites, selected from among the list of 2007 grantees (“current grantees”) and 2002 and 2004 grantees (“former grantees”), will provide information on the pre-service and in-service teacher training approaches that current grantees are using, as well as on the strategies that former grantees have used to follow up with newly-minted teachers after program completion and to plan for continuing program services after the federal grant period. The literature review will document the evidence available on characteristics of effective programming in teacher education for instructing ELs and will be used to inform continued planning for the case studies.
The study will collect data from the current grantees through site visits and from the former grantees through telephone interviews. The research questions guiding the case studies are shown in Exhibit 1.
Research Topics |
Research Questions |
|
Current Grantees |
Former Grantees |
|
Content and structure |
|
No questions. |
Overall program outcomes |
|
|
Efforts to institutionalize |
|
|
Efforts to follow up with participants |
|
|
Supporting Statement for Paperwork Reduction Act Submission
The purpose of this effort is to collect data on the implementation of the National Professional Development Program (NPDP). The NPDP is authorized under the Elementary and Secondary Education Act (as amended, Title III, Sec. 3131, 20 U.S.C. 6861) and provides grants to Institutions of Higher Education (IHEs), working in partnership with states and districts, to support projects designed to increase the pool of teachers highly qualified to work with English Learners (ELs) and to increase the skills of teachers already working with these students. The goal of the NPDP, ultimately, is to improve instruction to ELs, who represent a fast-growing, geographically-expanding, and diverse group of students with language-related and other educational needs.
The data to be collected will shed light on the approaches and strategies being used in pre-service and in-service teacher education through the NPDP, what approaches are promising, in what areas additional support may be required, how programs are addressing sustainability, and the extent to which programs are following up with participants and what types of outcomes are being realized. These data are much needed by the Department of Education (ED) and the EL field in general.
Data show that pre-service and in-service teacher education related to instruction for ELs are still catching up to the fast-growing and geographically expanding EL population. However, meeting the demand for more and better teacher education remains challenging, as teacher education providers cope with a limited evidence base on effective instructional practices for the EL population and respond to evolving state policies for licensure (e.g., new requirements for coursework or practical experiences for pre-service teachers for instruction of ELs).
For example, most IHEs report that they offer coursework related to ELs, and most also report that they have recently improved, or are planning to improve, coverage of topics related to ELs in their pre-service curricula. However, still only 20 percent of IHEs offer stand-alone coursework focused on the EL population, and less than one-third require their graduates to have field experience with ELs. Moreover, for those IHEs that do have EL-focused courses, the emphasis tends to be on how to communicate with families and apply cultural sensitivity, with less emphasis placed on English language acquisition. (GAO, 2009)
Also, in-service teachers and administrators report a need for additional training in instructing ELs. Preliminary results from a survey conducted through AIR’s study of Title III implementation (Tanenbaum et al., 2012) showed that while 75% of districts receiving Title III funds in 2009-10 reported offering support on the use of state standards for English language proficiency, that support varied substantially in amount and type. Furthermore, data collected in 2006-07 through the National Longitudinal Study of No Child Left Behind revealed that principals from one-third of schools nationwide reported needing technical assistance to meet the needs of EL students (Taylor et al., 2007). A 2004 study of California’s teachers showed that even those teachers with at least half their students as ELs received no more than one in-service focused on instruction for ELs, and nearly one-third reported significant concerns with the quality and usefulness of that training (Gándara, Maxwell-Jolly, and Driscoll, 2005). National data from teachers showed that, two years prior to the California study, there was a 28-point gap in the percentage of teachers who reported working with ELs and those who had received related professional development (PD) (NCES, 2002, as cited in NWREL, 2008).
Moreover, there is a limited knowledge base on what works for EL students: only 12 of 32 student interventions reviewed through the What Works Clearinghouse (WWC) met high standards of research quality, 11 of which suggested positive or potentially positive effects on ELs from the studied intervention. This lack of research base was cited by IHEs as a challenge to improving their instruction of EL educators (GAO, 2009). Moreover, while there have been numerous studies of PD generally, there has been less systematic review of the available literature on effective pre-service and continuing PD focused on instructing ELs. The literature review that is being prepared as an accompaniment to these case studies is expected to confirm the paucity of research-based evidence in this area.
Thus, the planned study will attempt to fill in some of these knowledge gaps. In particular, the data collected in the case studies can be used to provide much needed information to the field about existing approaches to preparing teachers to work with ELs, as well as to inform planning for future competitions of the NPDP grants and planning for more rigorous testing of particular approaches or more comprehensive study of the grant program.
Purposes and Uses of Data
The planned data collection will examine how a sample of grantees is implementing their grants with respect to:
The content and structure of the education they provide to current and prospective teachers of ELs;
The nature of changes they attempt to make to the full teacher education program at their institutions;
The efforts grantees make to institutionalize their projects so that they can be sustained after the grant ends; and
Their efforts to follow up with former program participants.
To address these areas, the study will collect information from 15 grantees from the 2007 cohort (“current grantees”) and 9 grantees from the 2002 and 2004 cohorts (“former grantees”). The information will be collected from the current grantees through in-person site visits and from the former grantees through telephone interviews. The site visits will include interviews with the director of the funded project, other IHE staff involved in implementing the project, and staff from partner organizations (i.e., district staff, principals) responsible for implementing the project, as well focus groups with teachers involved in delivery and with pre- and/or in-service teachers or other groups (i.e., IHE faculty) participating in the project. The telephone interviews conducted for the case studies of former grantees will target the project directors of formerly funded projects and one other staff at the IHE who was involved in delivery of the program.
Exhibit 2 presents a map of the information that will be sought, by respondent group.
Exhibit 2: Information to be Collected, by Research Topics and Respondent Group
Research Areas |
Subtopics
|
Project directors |
IHE staff |
Staff from partner entities*
|
Teachers involved in delivery |
Participants in grant activities** |
Content and structure |
Goals, history, and context |
x |
x |
x |
|
|
Content and structure |
x |
x |
x |
x |
x |
|
Outcomes |
Program |
x |
x |
x |
x |
|
Participants |
|
|
|
x |
x |
|
Efforts to institutionalize |
-- |
x |
|
x |
|
|
Efforts to follow up |
-- |
x |
x |
x |
|
|
*This respondent group includes district staff and principals.
** This respondent group includes pre-service teachers, in-service teachers, and other groups like IHE faculty.
Additionally, the study will collect and review course descriptions, syllabi, and other relevant program materials and artifacts, and will review pertinent information available on the IHE and partner district websites.
The contractor will use a variety of information technologies to maximize the efficiency and completeness of the information gathered for this evaluation and to minimize the burden that the evaluation places on respondents at the IHE and district levels:
To streamline the interview process and reduce burden on respondents, the research team will use IHE and district websites to gather extant information to prepare for interviews and to obtain background on the grantees and interview questions. (For more information on the use of extant sources, see the Data Collection Procedures section in Part B.)
A toll-free number and e-mail address will be available during the data collection process to permit respondents to contact interview staff with questions or requests for assistance. The toll-free number and e-mail address will be included in all communication with respondents.
The study also will involve the use of recording devices during telephone, in-person interviews, and/or focus groups. This will help reduce errors in data collection by capturing verbatim responses from interviewees, thereby requiring few exchanges of case study drafts between respondents and the research team.
Only one federal study has been identified that is examining pre-service or in-service education programs focused specifically on preparing teachers to work with English Learners, and it was determined not to be duplicative with the planned case studies of NPDP. This study, the Institute for Education Science’s (IES) Feasibility and Design Work for an Impact Evaluation of STEM Instruction for English Language Learners, is to examine the feasibility of conducting a national random assignment evaluation in the area of STEM instruction for ELs and part of the scope is to identify possible interventions for study. These interventions may be a professional development intervention and/or curricula program in math or science. While it is not impossible that such an intervention would be administered through one of the NPDP sites, and that that site would be in the case studies’ sample, the risk is quite small and the much more narrow scope of information to be collected by the feasibility study would not interfere with the broader purposes of the case studies.
Most other studies underway in the field of education for ELs are focused on testing the impacts of strategies or curricula targeted directly to ELs on their academic outcomes and therefore not related to the planned study. Studies related to studying the impacts of teacher pre- or in-service education more generally, such as the Department’s Teacher Quality Partnership Grants Impact Study, are not duplicative with the planned study because they do not focus on strategies aimed at improving teachers’ preparation to work with ELs.
The entities participating in this data collection effort will include IHEs and school districts. No small businesses will be involved. Should any of the grantees chosen for the case studies include a partner school district in a community with a population less than 50,000, these districts will benefit from the general efforts being made to minimize burden on respondents. For example, the research team will work in the months preceding the scheduled visits to capture as much existing data about specific IHEs and partner districts rather than requiring school personnel to gather such data. Additionally, the research team will identify a point of contact prior to the visits within the IHE to serve as liaison and help coordinate the site visits and component interviews and focus groups. The research team will work with this individual to make certain that the visit is as efficient and non-disruptive as possible. Finally, a pilot test of the interview protocols will be completed with 2 people in similar roles to those who will be interviewed in the planned study to ensure that the time burden is as minimal as possible.
Consequences of Not Collecting the Information
In the absence of these case studies, the federal government will be missing some helpful information to guide the further development of the grant program and provide technical assistance to grantees; to plan badly-needed, rigorous studies of particular approaches; or to disseminate to providers for improving pre- and in-service education for preparing teachers of ELs.
Special Circumstances
This information collection will not be conducted in a manner that will require using any special circumstances.
Federal Register Comments and Persons Consulted Outside the Agency
A 60-day notice was published in the Federal Register (Vol. 77, No. 55, page 16541-16542) on March 21, 2012. No known public comments have been received.
At the outset of the study, the contractor drew upon the experience and expertise of three expert reviewers who provide a diverse range of experiences and perspectives. The members of this team, their affiliations, and areas of expertise are listed in Exhibit 3. Additionally, as described in Part B, the instruments will be pilot tested with staff from 2 grantees not participating in the case studies.
Expert Reviewer |
Professional Affiliation |
Area(s) of Expertise |
Dr. Maria Torres-Guzman |
Teacher’s College at Columbia University |
Dr. Torres-Guzman is a professor of bilingual education and has extensive recent experience related to professional development activities for teachers of ELs. |
Teddi Predaris |
Fairfax County Public Schools |
Ms. Predaris brings an on-the-ground perspective from a large, suburban district with a growing EL population. Strongly-regarded in the field, she is a past coordinator of ESOL services for the district and has also served as a resource specialist for NCELA and other organizations on issues related to EL students and professional development for EL education. |
Dr. Robert Jimenez |
Vanderbilt University |
Dr. Jimenez is a professor of language, literacy, and culture in the teacher education program at Vanderbilt. He teaches courses in research methods, second language literacy, and issues related to the education of Latino/Latina students. |
No payments or gifts are planned for the course of this study.
As a research contractor, the research team is concerned with maintaining the confidentiality and security of its records and will protect the confidentiality of the data to the extent possible through a variety of means. The contractor’s project staff has extensive experience collecting information and maintaining confidentiality, security, and integrity of interview and survey data. The team has worked with the Institutional Review Board (IRB) at the American Institutes for Research to seek and receive approval of this study and the measures used to protect confidentiality, including the following:
Project team members will be educated about the confidentiality protections given to respondents and about the sensitive nature of materials and data to be handled. Each person assigned to the study will be cautioned not to discuss confidential data.
All electronic data will be protected using several methods. AIR’s internal network is protected from unauthorized access by using defense-in-depth best practices, which incorporate firewalls and intrusion detection and prevention systems. The network is configured so that each user has a tailored set of rights, granted by the network administrator, to files approved for access and stored on the local area network (LAN). Access to AIR’s computer systems is password protected, and network passwords must be changed on a regular basis and conform to a strong password policy. All project staff assigned to tasks involving sensitive data will be required to provide specific assurance of confidentiality and obtain any clearances that may be necessary. All staff will sign a statement attesting to the fact that they have read and understood the security plan and ED’s security directives. A copy of this statement is featured in Appendix E.
For district and IHE interviews, respondents’ names and contact information will be used for data collection purposes only and will be disassociated from the data as they are entered into the database. As information is gathered from respondents or from sites, each will be assigned a unique identification number; this number will be used for printout listings on which the data are displayed and for analysis files. The unique identification number also will be used for data linkage. Data analysts will not be aware of any individual’s identity.
The contractor will shred all hardcopy documents containing identifiable data as soon as the need for the hardcopy documents no longer exists. They also will destroy any data tapes or disks containing sensitive data.
Participants will be informed of the purposes of the data collection and the uses that may be made of the data collected. All respondents will be asked to sign an informed consent form (see drafts in Appendix B). Consent forms will be collected and stored in secure file cabinets at the contractor’s office in Washington, DC.
In informing participants and obtaining their consent, the research team will provide explanations regarding confidentiality. They will explain that while the list of sites selected for case studies will be identified, no names of individual IHE respondents, district respondents, participating pre-service or in-service teachers will be identified in reports or presentations, nor will any quotes be attributed to specific individuals. Responses to the interviews and protocols will be used primarily to summarize findings in an aggregate manner (e.g., across types of programs) and secondarily to provide examples of program or strategy implementation. When providing such examples, the research team will do so in a manner that does not associate responses with a specific individual, school, or district.
No questions of a sensitive nature will be included in the study.
Estimates of Hour Burden
Exhibit 4 provides information on the burden of the collection of information, by type of respondent and type of grantee. This is a simplified version of the table provided in the IC Burden Analysis Table. The total burden of information collection is a maximum of 450 hours. Estimates for interviews with IHE staff and focus groups with participating pre- and in-service teachers are calculated based on the maximum number of participants, since there will be a range depending on the size of the individual programs and the need to collect data from additional respondents.
Exhibit 4: Estimate of Time Burden, by Respondent and Type of Grantee
|
Interview with Project Director |
Interviews with IHE staff |
Interviews with partner organization staff (including district staff, principals) |
Focus group with teachers involved in delivery |
Focus groups with participants (including pre-service teachers, in-service teachers, others)* |
90 minutes |
60 minutes |
60 minutes |
60 minutes |
60 minutes |
|
Current grantees |
1 per 15 sites |
Up to 3 per 15 sites |
Up to 2 per 15 sites |
1 group per 15 sites with up to 6 participants |
2 groups per 10 sites with up to 6 participants Up to 4 groups per 5 sites with up to 6 participants |
Former grantees |
1 per 9 sites |
1 per 9 sites |
– |
– |
– |
Total burden hours |
36 hours |
54 hours maximum |
30 hours maximum |
90 hours maximum |
240 hours maximum* |
* Size dependent: For most grantees, there will be two focus groups with the relevant category or categories of respondent groups. For grantees with over 100 participants (pre-service teachers, in-service teachers, or other) or more than two potential categories within the respondent group, there will be up to 4 focus groups. One-third of the sites (5) are expected to be this size or have more than two respondent categories. The estimated maximum burden is based on this expectation and the participation of 6 individuals in each focus group.
There are no direct monetary costs to respondents for this activity. At an average of $30 per labor hour, the overall cost burden for information collected through the interviews and focus groups is estimated to be $13,500.
There are no additional respondent costs associated with this data collection beyond the hour burden estimated in item A12.
The estimated cost to the federal government of conducting these data collection activities is based on the government’s contracted cost of the data collection and related study activities, along with personnel cost of government employees involved in oversight and/or analysis. For the data collection activities for which OMB approval is currently being requested, the overall cost to the government is $514,371. This includes activities of the contractor to develop the instruments, pilot test instruments, identify participating sites, design and conduct site visit training, and collect and analyze the data. This encompasses the planning, implementation, and analysis/reporting tasks. This estimate includes the required labor and associated administrative costs. This estimate also includes the preparation, training, travel, and logistical costs for the site visit teams to visit 15 sites and conduct phone interviews with an additional 9 sites. The site visit team will include two staff members and they will be in the field for approximately 2 days per trip.
This request is for a new information collection.
Data will be collected between July 9, 2012 and November 23, 2012.
This study will generate two products. The first product, for internal use by the Department, will be a memorandum of preliminary findings. This will be due December 12, 2012 and will serve as a basis of discussion between the contractor and the Department in outlining the final report.
The second product, the study report, will present the full findings from the case studies of current and former grantees. In addition to examining cross-cutting themes that emerge as the study questions are addressed, this report will profile each funded project included in the case studies. Finally, the study report will locate key findings and conclusions in the larger context of empirical research discussed in the literature review.
The first draft of the study report will be due January 25, 2013 and up to 5 additional drafts may be requested. Pending the Department’s review and approval, the final draft of the study report will be completed by August 26, 2013.
All interview and focus group instruments will include the OMB expiration date.
No exceptions are requested.
Gándara, P., Maxwell-Jolly, J. and Driscoll, A. (2005). Listening to teachers of English Learners. Santa Cruz, CA: Center for the Future of Teaching and Learning.
General Accounting Office (GAO) (2009). Multiple Federal Education Offices Support Teacher Preparation for Instructing Students with Disabilities and English Language Learners, but Systematic Departmentwide Coordination Could Enhance This Assistance (GAO 09-573). Washington, D.C.: Author.
Northwest Regional Education Lab (NWREL) (2008). Look @ Equity: Email news from NWREL’s Equity Center, October 2008. Accessed December 27, 2011 from: http://educationnorthwest.org/webfm_send/119.
Tanenbaum, C., Boyle, A., Soga, K., Carlson Le Floch, K., Golden, L., Petroccia, M., Toplitz, M., Taylor, T., and O’Day, J (2012). National Evaluation of Title III Implementation—Report on State and Local Implementation. Washington, D.C.: U.S. Department of Education.
Taylor, J., Stecher, B., O’Day, J., Naftel, S., Carlson Le Floch, K. (2010). State and Local Implementation of the No Child Left Behind Act, Volume IX—Accountability Under NCLB: Final Report. Washington, D.C.: U.S. Department of Education.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | mstephens |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |