Att_1875 NEW 4488 rev OMB Supporting Statement A_RC Edits Mar 24 2011_NTC

Language Instruction Educational Programs (LIEPs):

Lessons from the Research and Profiles of Promising Programs

Office of Policy and Program Studies Service

U.S. Department of Education

Supporting Statement for Paperwork Reduction Act Submission

Part A

Project Officer: Andrew Abrams, Analyst Policy and Program Studies Service U.S. Department of Education (202) 401-1232 [email protected]

Project Director Rhonda Crenshaw, Senior Program Associate Synergy Enterprises, Inc. 8757 Georgia Avenue, Suite 1440 Silver Spring, MD 20910 (240) 485-1700 Ext. 235 [email protected]

INTRODUCTION

This document requests approval for data collection activities to study Language Instruction Educational Programs (LIEPs) serving English learners (ELs) in kindergarten through grade 12. In September 2010, the U.S. Department of Education awarded a contract to Synergy Enterprises, Inc. (SEI) and its subcontractor edCount, entitled "Language Instruction Educational Programs (LIEPs): Lessons from the Research and Profiles of Promising Programs." LIEPs are program models that consist of a comprehensive set of services including curriculum, instructional strategies, and other essential services (e.g., counseling, family support) that facilitate English proficiency and academic content gains for English Learners (ELs). This contract involves an exploratory study of LIEP characteristics that may influence the quality of programs delivered to English Learners (EL) in grades K through 12. The major purpose of this study is to gather data from the field that yield an initial portrait of descriptive characteristics of well-designed and implemented LIEPs, and to provide practical guidance to local educators on selecting, designing, implementing, and evaluating LIEPs. The study is scheduled for completion in September 2012. This study is authorized under Title III, Sec. 3111(c)(1)(C).

The study has two phases: (1) a review of the literature on characteristics of high-quality LIEPs, and (2) case studies of twenty promising schools and/or school districts. To collect the case-study data, the research team will use two primary data collection instruments: (a) structured individual or group interview protocols for a range of respondents including teachers, administrators, district personnel, counseling staff, families and family support liaisons, other instructional staff, and State Title III directors; and (b) an observation instrument to capture qualitative, descriptive data on specific school-based activities and classroom practices that represent features of a better designed and implemented LIEP, as determined by a literature review and case studies of programs with solid outcomes for students. LIEPs. LIEP programs included in this study were selected based on nominations by Title III State Directors and AYP/AMAO data. Site visits to 20 school districts in 10 states will take place August 2011 through November 2011. Districts were nominated by State Title III Directors through an informal process. Specifically, the research team was invited by the ED Program Office to provide an overview of the study and its intended outcomes during an ED-sponsored monthly webinar with State Title III Directors. The research team indicated that the nominated sites should represent a range of LIEP types (i.e., English immersion, bilingual, newcomer). The sites should represent variations in EL student diversity (i.e., various language groups and countries), have a significant EL population, and include urban, rural, and suburban schools and varied geographic locations. Additionally, the nominated sites should be implementing a promising LIEP, have adequate resources and supports in place to sustain their program, and have a system in place for evaluating program success. During the webinar, the research team requested that nominated districts/schools be communicated to the research team director or via the Program Office Listserv. Once nominations were received, student outcome data were gathered by the research team to validate nominated sites. The site selection method is discussed also in Supporting Statement Section B.

The study will produce a review of the relevant literature and a Program Implementation and Evaluation Guide (the “Guide”) on LIEPs for local education practitioners. Content in the Guide will be based upon the literature review as well as case study data. It will outline best practices and challenges identified during case study site visits; discuss facilitators to overcoming barriers that prevent successful implementation; and provide guidance on how schools and districts should integrate and use evaluation as a key component of their programs. This study is intended to provide a foundation to support further research and informed dialogue among local educators, state coordinators, and the U.S. Department of Education.

This exploratory study will provide key descriptive information on LIEPs serving children in kindergarten through grade 12, but will not provide information on program outcomes or impacts. Three main questions (with related sub-questions) guide the study:

1. What are the research-based components of established LIEPs? How are these LIEPs designed to address specific student or context needs, for example, different levels of home-language and EL proficiency and different policy goals?

2. How are these LIEPs and related strategies implemented in various settings at the elementary and secondary levels (including middle and high school)? What are the key characteristics of promising implementation?

3. How can States and districts monitor and evaluate the effectiveness of these programs?

PART A: JUSTIFICATION

A1. Circumstances That Make Collection of Data Necessary

The review of the literature (which is Part 1 of this study) confirms the paucity of research-based evidence on the characteristics of LIEPs in K-12 public education. Specifically, there have been few systematic attempts to determine the characteristics of promising LIEPs for ELs in kindergarten through grade 12 and to describe contextual factors that contribute to their effectiveness. This study will provide information needed by the field and government to help support improved practices. In particular, ED, state education agencies (Title III State Directors), local education agencies, and school-based personnel will gain baseline descriptive data about LIEPs that will allow them to identify technical assistance needs in LIEPs and engage in program improvement. It also will identify schools/districts implementing promising LIEP elements and share best practices with the LIEP community. Among other benefits, this study can support the development of a specific set of policy recommendations to enhance program practices across LIEPs and the larger school community.

The proposed information collection will be conducted as part of the LIEP study. Specifically, the study will gather this data from 20 school districts in 10 states that were nominated by State Title III Director as having promising LIEPs. Public Law 107-110, Title III, Sec. 301 of the Elementary and Secondary Education Act (ESEA) is known as Language Instruction for Limited English Proficient and Immigrant Students. Through formula grants to State education agencies, Title III provides funds to support school districts in teaching English to limited English proficient students and in helping these students meet the same challenging state standards required of all students.

A2. Purposes and Uses of the Data

Data collection activities are designed to yield valuable information about the characteristics and context of LIEPs, ELs and their family’s needs, practitioners’ needs, and priorities for program effectiveness. The information collected will be used to inform program policy and technical assistance in the coming years.

Exhibit 1 below lists each of the instruments, along with the mode of administration, information to be gathered, time needed, and estimated timeline for administration.

Exhibit 1. Data Collection Instruments (See Appendices A-F)

Instrument/ Respondent Group	N	Mode of Administration	Information Sought	Time	Timeline
Phase 2
Interview Protocol/State Title III Directors Interview	10	Telephone	Document the program from State perspective, including: overall program description, curriculum, standards, academic support activities, effectiveness, contextual considerations, staffing considerations, and challenges	60 minutes	June 2011 – August 2011
Group Interview Protocol/Families and Family Support Liaisons Focus Group	100 individuals	Audio-taped (back-up to on-site note-taking) focus group by research team. We estimate up to 5 focus group participants per site consisting of 4 individuals representing families and one family support liaison.	Description of parent involvement and partnership with schools concerning EL students’ learning, social development, etc. Communication with families about EL needs, community relations, meeting parent needs, overall rating of satisfaction among parents with LIEP programming	60 minutes	September 2011 – November 2011
Group Interview Protocol/ Teachers Interview	100 individuals	Audio-taped (back up to on-site note-taking) group interview with up to 5 teachers per site by research team.	Description of overall program; policies and procedures in place; interactions with ELs; communication with families and community; content area and EL Standards, EL assessment tools; resources and professional development; collaborative partnerships within school building, community, and district; general overview of classroom-based components of the LIEPs	60 minutes	September 2011 – November 2011
Interview Protocol/School Administrative Team Interview	60 individuals	Audio-taped (back up to on-site note-taking) group or individual interviews by research team. We estimate up to 3 administrators per site.	Description of program; policies and procedures in place; interactions with EL teachers; communication with EL families and community; school alignment to district goals; EL assessment tools, resources and professional development	60 minutes	September 2011 – November 2011
Individual Interview Protocol/District Headquarters Team Interview	60 individuals	Audio-taped (back up to on-site note-taking) individual interviews by research team. We estimate interviewing up to 3 District HQ personnel per site.	Document the program from district perspective, including overall program description; curriculum Standards and academic support activities, effectiveness, contextual considerations, staffing issues, and challenges	60 minutes	September 2011 – November 2011
School-Based Activities Observation Instrument	120 events observed	Observation of up to six events at 30-45 minutes each by two-person research team	Document the application of best practices in a variety of school-based events and services that comprise the LIEP. Such events/services could include: classroom instruction, teacher planning periods, district planning or professional development events, family support events, school administrative planning events. . Narrative descriptions are captured on the observation form and key characteristics of known best practices are included to serve as cues to help observers during site visits.	45 minutes	September 2011 – November 2011

A3. Use of Information Technology to Reduce Burden

This study will involve accessing extant demographic and assessment/accountability data from web-based state, district and/or local school web portals. The data will be used to inform the selection of sites and to prepare for site visits. In addition, the study will involve the use of recording devices during in-person interviews and/or focus groups. This will help reduce errors in data collection by capturing verbatim responses from interviewees, thereby requiring few exchanges of case study drafts between respondents and the research team.

A4. Efforts to Identify and Avoid Duplication

There are no existing studies that describe in detail the comprehensive set of services associated with well-implemented LIEPs with the intent of using such data to develop a guide on how to select, design, implement and evaluate such programs. Most existing studies have focused narrowly on strategies related to curriculum and instruction (e.g., studies of specific curricula used). Few address the other key components that make the LIEP a comprehensive program or model (e.g., counseling services, family programs, transportation, and other services). The lack of existing data was determined through internet and literature searches and validated by expert members of the study’s Advisory Panel. Currently no other methods exist that would enable us to systematically capture information about the specific practices, strengths, needs, challenges, and concerns of such programs in the LIEP community.

A5. Efforts to Minimize Burden on Small Businesses or Other Entities

Small businesses are not respondents in this study. The research team will work in the months preceding scheduled site visits to capture as much existing data (e.g., assessment information, AYP status, via resources such as EDFacts, school and district websites) about specific schools and/or districts rather than requiring school personnel to gather such data. Additionally, the research team will identify a point of contact prior to the site visits. The research team will work with this individual to make certain that our visit is as efficient and non-disruptive as possible. There will only be one interview per respondent or respondent group. Respondents for these interviews, as listed in Exhibit 1, are staff members of the SEA, local educational agency, and school and parents whose children attend targeted schools/districts. Finally, a pilot test with each of these instruments will be completed with up to nine people in similar roles to those who will be interviewed to ensure that the time burden is as minimal as possible.

A6. Consequences if the Information is not Collected or is Collected Less Frequently

In the absence of these interviews, the EL field will not have information to help them improve program practices. Moreover, the federal government will not be able to provide enhanced guidance and technical assistance to practitioners, or fully address policy concerns among education practitioners who serve English learners. Furthermore, it will remain challenging for ED to provide guidance on LIEP program monitoring to State Title III directors.

A7. Special Circumstances Requiring Collection of Information in a Manner Inconsistent with Section 1320.5(d)(2) of the Federal Regulations

This information collection will not be conducted in a manner that will require using any special circumstances.

A8. Federal Register Comments and Persons Consulted Outside of the Agency

A 60 day Federal Register notice was published (VOL 76 page 4643) on January 26, 2011. No known public comments have been received. Lead researchers for these surveys have consulted on both the content and form of data collection with experts in interview protocols, observations, and other data collection instrument design; ESL and EL research experts; and state and local level experts in Title III and delivery of educational programming to ELs and their families.

A9. Payment to Respondents

No payments to respondents will be offered. No direct incentive to respondents is planned.

A10. Assurance of Confidentiality Provided to Respondents

Responses to this data collection will be used only for research purposes. The case studies prepared for this study will summarize findings across the sample and will not associate responses/quotes with a specific person from a district or school. Synergy Enterprises Inc. (SEI) and edCount only will provide information that districts, schools, or state officials have approved as a result of their execution of the informed consent document. We will not release any information to anyone outside the study team, except as required by law. All respondents will be made aware of confidentiality and will execute an informed consent document.

The organizations that are part of the research team will follow procedures for assuring and maintaining confidentiality and security of its records. As such, the research team will protect the confidentiality of the data to the extent possible through a variety of means. The research team staff has extensive experience collecting information and maintaining confidentiality, security, and integrity of interview data. The following safeguards are routinely employed to carry out confidentiality assurances:

• All staff members associated with this study at Synergy Enterprises Inc. (SEI) and edCount have signed agreements or have written policies regarding confidentiality and privacy. These agreements affirm each individual's understanding of the importance of maintaining data security and confidentiality and of abiding by the management and technical procedures that implement these policies. Members of the study team also are in the process of obtaining ED Security Clearances. Five members of the team have existing clearances with ED and/or the Department of Homeland Security.

• Research team members will participate in training and be educated about the confidentiality protections given to respondents and about the nature of materials and data to be handled. Each team member will be cautioned not to discuss confidential data.

• All data will be protected using several methods. All data, including paper files, audio files, and computerized files, will be kept in secure areas. Paper and audio-taped files will be stored in locked storage areas with limited access on a need-to-know basis. Computerized files will be managed via password control systems to restrict access and to physically secure the source files, which will be located on secure servers in other locations. The internal network is protected from unauthorized access. All members of the research team involved in collecting data will be required to provide specific assurance of confidentiality and obtain any clearances that may be necessary by ED. Research team members will sign a statement indicating that they have read and understood the research confidentiality and security plan and ED’s security directives.

• The contractor will shred all hardcopy documents containing identifiable data as soon as the need for the hardcopy documents no longer exists. They also will destroy any data tapes or disks containing information, once such data has been used. At the time of the interview, efforts to maintain confidentiality will be reviewed with the respondent.

• Participants will be informed of the purposes of the data collection and the uses that may be made of the data collected. All respondents will be asked to sign an informed consent form. Consent forms will be collected and stored in secure file cabinets at the contractor’s office.

• Merged data sources will have specific individual identification data stripped from the individual records or will be encoded with unique identifiers to preclude overt identification of specific individuals. For district and state interviews, respondents’ names and contact information will be used for data collection purposes only and will be disassociated from the data as they are entered into the database. As information is gathered from respondents or from sites, each will be assigned a unique identifier that will be used for printout listing on which the data are displayed for analysis. The unique identifier also will be used for data linkage.

• All reports, tables, and printed materials will be limited to the presentation of aggregated numbers.

• Compilations of individualized data will not be provided to participating schools, districts, or SEAs.

• Confidentiality agreements will be executed with any participating research subcontractors and consultants who must obtain access to detailed data files.

An explicit statement describing the study, the data collection, and confidentiality will be sent to all potential participants who are invited to participate in the interviews.

A11. Justification for Questions of a Sensitive Nature

The questions on the instruments do not address sensitive topics.

A12. Estimate of Information Collection Burden

As indicated earlier, the interview data collection and observation will occur only once. Exhibit 2, below, shows that the estimated annual respondent burden for this data collection is 165 hours. Total respondent burden for the study is 330 hours.

Exhibit 2. Respondent Hour Burden Estimate

Data Collection Activity	Hour Burden per Respondent (in hours)	Annual Expected Number of Respondents	Annual Hour Burden (in hours)
State Title III Director Interview	1.0	5	5
EL and Content Teachers	1.0	50	50
Families and Family Support Liaisons	1.0	50	50
School Administrative Team	1.0	30	30
District Headquarters Team	1.0	30	30
School-Based Activities Observation*	.75	60 events	45
ANNUAL TOTAL	5.00	165	165
TOTAL OVER 24 MONTHS		330	330

*Note: School-based activities observations are strictly observations and will not involve interviews or questioning respondents. No data is being requested at the time of observation. As such, the time represented above is NOT being included as part of the total annual hour burden estimate.

The estimated annual/ cost burden for all data collection is presented in Exhibit 3 below.

Exhibit 3. Respondent Cost Burden Estimate

Data Collection Activity	Annual Respondents	Annual Hour Burden	Hourly Rate	Annual Cost Burden
State Title III Directors	5	5	$50	$250.00
EL and Content Teachers	50	50	$15	$750.00
Families and Family Support Liaisons	50	50	$12	$600.00
School Administrative Team	30	30	$35	$1,050.00
District Headquarters Team	30	30	$40	$1,200.00
ANNUAL TOTAL	165	165	152	$3,850.00

A13. Estimate of Total Annual Cost Burden to Respondents

There are no direct costs to participants, with the exception of the time required by respondents to participate in interviews as provided in Exhibit 3 at the end of item A12.

A14. Estimates of Annualized Costs

The estimated cost to the federal government of conducting these data collection activities

is based on the government’s contracted cost of the data collection and related study activities along with personnel cost of government employees involved in oversight and/or analysis. For the data collection activities for which OMB approval is currently being requested, the overall cost to the government is $494,180. This includes activities of the prime contractor and subcontractors to develop the instruments, pilot test instruments, identify participating sites, design and conduct site visit training, and collect and analyze the data. This two-year study will encompass the planning, preparation, analysis, and reporting tasks. This estimate includes the required labor and associated administrative costs. This estimate also includes the preparation, training, travel, and logistical costs for the site visit teams to visit 20 sites. The site visit team will include at least two staff members and they will be in the field for at least four days per trip. Master observers will participate in six of the visits to monitor inter-rater reliability and scoring accuracy.

A15. Change in Annual Reporting Burden

This is a new study/data collection.

A16. Plans for Tabulating and Publication of Results

Study Reports

We plan to produce a Program Implementation and Evaluation Guide using data from the site visits. The data will be presented as case studies with practical guidance for local educators. Following PPSS approval, the Guide will be posted on the PPSS public website. The data that will be used to construct the Guide will be collected during site visits that span from September 2011 through November 2011. During the 20 site visits, the research team will conduct interviews and observations. The resulting Guide is expected to be finalized in September 2012.

A17. Seeking Approval to Not Display the OMB Expiration Date

No request is being made for exemption from displaying the expiration date.

A18. Explanation of Exceptions

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.