Adult ESL Explicit Literacy Impact Study (KA)

July 21, 2006

Dear XXXXXXXXX

Thank you for the information you have shared with us in regards to the Adult ESL Explicit

Literacy Impact Study, a groundbreaking study designed to test the effect of a high quality,

innovative curriculum on the English literacy and language development of low-literate adult ESL

learners. The study is sponsored by the U.S. Department of Educations Institute of Education

Sciences, and is scheduled to begin in fall, 2007. This letter is to apprise you of the status of our

program selection process, and to invite you to participate in the next phase of this process.

The selection of programs that can implement the study is key to the studys success and involves a

multi-phase process to ensure that programs that are being considered meet study criteria. As you may

recall, a project staff member interviewed you requesting information on whether your program has:

a.

Site enrollments of low-literate adult ESL literacy learners large enough to support the

evaluation (i.e., enrollment of at least 90 low-literate adult ESL students per semester or

instructional period).

b.

At least 4 classes of about 2025 low-literate adult ESL learners that provide 120 hours of

instruction per semester or other term, and meet at least two hours at a time.

c.

At least 5 trained adult ESL literacy instructors who teach the target classes.

d.

Two pairs (or all 4) of the classes meet at the same time and location.

e.

Managed enrollment or an enrollment process where a majority of learners enter during

the beginning of a course.

Based on our prior discussions, we believe that your program meets these criteria. We now would

like you to participate in a conference call, which will be conducted by project staff, including one

of our research partners. During this call, we will explain the studys objectives and the benefits of

your participation to your program. We will also review each criterion for inclusion in the study with

you to confirm the information we collected earlier and to fill information gaps related to enrollment

policies, class schedules, teacher qualifications, and the nature of your current curriculum. In

addition, we will explain the studys methodology and request from you names of any other people

that we need to contact to secure your participation.

As we proceed to phase two, please remember that the study is specifically about low literacy adult

learners who span a range of low-level literacy abilities. In phase two of the selection process, we

want to focus our discussion more on the specific sites and classes which will participate in the study.

We will want to learn more about the students, including their enrollment patterns, attendance and

retention in these specific classes and sites. We also want to know more about the teachers of these

classes who will potentially participate in the study, including their willingness to participate. To

prepare for the conference call, we would like you to gather more information about these topics,

including any data you may have. Please also consider which staff members are appropriate to join

in the conference with us (i.e. ESL specialists, other site or program directors, teachers who might

want to be involved to help us determine the sites eligibility for the study).

I am including more information about the study and the curriculum, including the benefits of

participation, the site selection criteria, and the random assignment process. One of the project staff

members will call you in a few days to confirm your interest in the study and to schedule the date

and time of the conference call. If you have any questions, please dont hesitate to contact me or

email Anestine Hector-Mason at [email protected]; 202-403-6913.

Thanks for your kind consideration and we look forward to speaking with you.

Sincerely,

Larry Condelli, PhD

Project Director

American Institutes for Research

1000 Thomas Jefferson Street

Washington, DC 20007

Tel: 202-403-5331

Email: [email protected]

Site Selection Criteria for

Explicit Literacy Impact Study

Selecting sites that meet the studys criteria is very important to the studys proper

implementation. Sites in different geographic regions in the US that meet the following

criteria will be considered for the study:

Operate large ESL programs, serving lowliterate

adult

ESL

students.

To meet the

sample size requirements, we need programs in the

study that offer at least four beginning-level ESL

classes per semester with a minimum of 2025 low-

literate students in each class. Over two semesters,

this will result in about 180 low-literacy students

receiving instruction in either the explicit literacy

or the existing ESL group. By low-literacy we

mean students who have only limited literacy skills

in their native language. We verify literacy level

through a brief native language literacy screener.

Spanish

speakers.

While we believe that

speakers of all languages will benefit from the

curriculum, some instructional activities build

on students Spanish literacy. Therefore, we want

a large majority of students in the classes to be

native Spanish speakers to allow us to conduct a

full evaluation of all activities in the curriculum.

Managed

enrollment

policy. To implement

the explicit literacy curriculum effectively, it is

necessary that most students receive the curriculum

from start to finish. This means that the students

are only allowed to enroll at a limited time at the

beginning of the class term. Programs with open-

enrollment policies can only be considered if the

large majority of learners enroll during a limited

time period at the beginning of the session.

Sufficient

class

time.

The curriculum is

designed in 60 one-hour modules of instruction

that are to be followed by one hour of instruction

normally provided (e.g., conversational, life

skills-type instruction). Therefore, in a 2 hour

class, 1 hour would be devoted to the explicit

literacy instruction. This suggests that, ideally,

a class would need to meet at least four times

per week for 2 hours each day for 15 weeks, for

a total of 120+ instructional hours. Other class

arrangements are possible as long as they can

implement the full curriculum within 15 weeks.

Student

retention. Student retention is key

to evaluating the curriculum because students

must receive a sufficient amount of instruction.

Therefore, it is important that the number of

students who fail to complete the entire scheduled

instruction is minimal. We need classes in the study

that have a record of high student retention.

Do

not

already

offer

explicit

literacy

instruction.

Explicit instruction is the systematic,

sustained, and deliberate teaching of language and

literacy skills and of strategies for comprehension.

In ESL literacy, three essential features of this

type of instruction are (a) to help students see

language and literacy connections, (b) to make the

learning objectives transparent, and (c) to provide

a cumulative and sequential approach to literacy

learning. This is a new approach in adult ESL and

we believe that no programs currently provide this

type of instruction.

Teachers. Selected sites must have at least

5teacherswhoarewillingtoparticipateinthestudy.

Two of the teachers will be assigned randomly

to teach the explicit literacy curriculum and one

teacherwillbeaback-upwhowillonlyteachif either

of the two assigned teachers cannot participate.

All teachers should have some experience teaching

low-literate adult ESL students.

Curriculum for

Explicit Literacy Impact Study

The goal of the curriculum is to increase the English literacy skills of students who speak no

or little English and have limited literacy skills in their native language.

Explicit Literacy Principles

The curriculum uses an approach to teaching

known as explicit teaching and will focus on

the development of literacy skills, including skills

associated with learning to read in another language,

vocabulary development, listening/reading and

comprehension. The following are some key

principles of explicit literacy instruction:

Curriculum, instruction, learning experiences

and assessment must be aligned.

Teaching goals and curriculum must be clearly

defined.

Lessons must involve instructional modeling

or demonstration, guided practice, corrective

feedback, and practice.

Instruction must promote the development of

skills and strategies for making meaning.

Teaching of oral language and literacy must

be systematic and purposeful.

Curriculum and instruction must entail

transparent connection between language and

literacy.

Materials

The curriculum reflects research in second

language acquisition, reading, and adult literacy

development. A fully developed set of teaching

materials includes:

illustrated and detailed lesson plans,

demonstration materials on PowerPoint slides

or overheads,

student handouts and manipulatives,

a student workbook,

a student reader with picture dictionaries, and

a cd-rom of recorded reading in two speeds.

Lessons

Lessons are organized by themes of interest to

adult learners and learning activities are integrated

and contextualized. The lessons are structured in a

way that allows teachers to move through a series

of activities that include providing comprehensible

input and guided practice for pairs or small groups;

demonstrating the skill or strategy to be learned,

and conducting a short teacher-led review. Each

lesson will include teacher observations and

strategies for dealing with a multi-level classroom.

Lessons reflect an explicit approach to teaching that

emphasizes the deliberate teaching of skills and

strategies associated with literacy development, a

systematic effort to help students understand how

English works, ample opportunities for students

to engage in the content and alignment of the key

skills through demonstration, guided practice, and

ongoing informal assessment.

The curriculum is rich in illustrations and

photographs and uses multiple modes of learning

(visual, text-based, auditory). It seeks to promote

transfer of knowledge and builds on both the

linguistic knowledge and the real world knowledge

that adults bring to the learning process.

Teacher Training and Ongoing Support

The curriculum developers will conduct a week-long

training session for teachers that will familiarize

them with the instructional strategies that are part of

explicit teaching and to offer them opportunities to

engage in, and reflect on, teaching demonstrations.

Teachers will be supported during implementation

through on-line and on-call coaching, listserv

conversations with fellow teachers, web resources,

and a classroom visit.

Benefits of Participating in the

Explicit Literacy Impact Study

Selected programs will play an important part in a large-scale study that has the potential

to make very important contributions to the Adult English as a Second Language (ESL)

field specifically, and Adult Education in general. In addition, the programs will receive the

following direct benefits:

The

Explicit Literacy curriculum. Each

teacher assigned to deliver the explicit literacy

curriculum will receive a free copy of the

curriculum, including a copy of all corresponding

supplemental materials such as detailed lesson

plans, PowerPoint slides and other demonstration

materials, handouts and manipulatives, student

workbooksandreaderwithpicturedictionary,and

a CD-ROM of recorded readings. This curriculum

is based upon the best research available. Programs

will also receive one additional copy of the

curriculum at the conclusion of the study, at no

charge.

Training.

Each teacher assigned to deliver

the explicit literacy curriculum will receive free

training in a one-week course offered prior to the

start of class. These teachers will receive training

in using the new explicit literacy curriculum and

materials designed to improve the literacy skills of

low-literate adult ESL learners. AIR teamed with

a curriculum developer, Literacy Work, to create

a training program built around principles of

adult learning, effective ESL teaching, and explicit

literacyinstruction.Inadditiontothefreetraining,

all corresponding training materials, travel, and

accommodations during the training are free of

charge to teachers assigned to deliver the explicit

literacy curriculum.

Technical assistance. The curriculum developer

will be available to provide ongoing technical

assistance to the explicit literacy teachers through

a project web site and regular contact by telephone

and e-mail. The web site will provide useful

instructional modeling video clips and a listserv

forum, as well as other useful implementation tips

and materials. The developer will also be available

for questions by phone, and will hold weekly

phone calls with teachers. In addition, Literacy

Work and AIR staff will conduct at least one

class observation during the instructional period

to assess whether teachers are implementing the

curriculum faithfully. Literacy Work staff will

also provide feedback and additional training to

teachers as needed.

Cost-free program evaluation. The Explicit

Literacy Impact study will use a random

assignment design to measure whether the

explicit literacy curriculum makes a difference in

improving the English literacy skills of low-literate

adults compared to the instruction usually offered.

We will provide your program with the results of

this evaluation for the students in your program.

In addition to the results of the impact study, we

will provide you with descriptive analyses on the

characteristics of the students in the program,

their test performance, patterns of attendance and

other data. There will be no burden on program

staff to obtain this information because research

staff will collect all post-test assessment data,

conduct interviews and track students.

Random Assignment in the

Explicit Literacy Impact Study

The random assignment of students and teachers is a key component of the study because

it enables us to assure that changes over time in student performance are due to the new

curriculum rather than to differences in teacher and student characteristics and/or naturally

occurring changes in the students and their social and educational environments.

What is random assignment?

Researchers use random assignment in impact

studies to form two statistically equivalent groups

of participants in the most objective way possible.

For this study, the term participants refers to

teachers and students who are eligible for and agree

to participate in the study. A list of participating

students will be created and those students on the

list will be randomly assigned to classrooms. Some

of the students will be assigned a classroom in which

the explicit literacy curriculum will be delivered

and some will be assigned to classrooms in which

the programs existing curriculum is delivered.

Participating teachers will also be randomly assigned

to teach either the explicit literacy curriculum or

the existing curriculum.

How does random assignment work?

The random assignment process works much the

same as a lottery, so each participantwhether

a student or a teacherhas the same chance of

ending up in the class using the explicit literacy

instruction or to the one using the programs

existing ESL curriculum. In this study, a computer

will be used to randomly assign both the teachers

and the students.

Teachers.

We will work with the programs to

identify teachers who agree to be in the study.

Among this group, we will randomly assign

the teachers to the explicit literacy group or

the existing ESL curriculum group. Only those

assigned to the explicit literacy group, along

with one back-up instructor, will be trained

in the curriculum. The back-up instructor

will only be used for this study if he/she is

needed as a substitute for a teacher that has

been randomly assigned to the explicit literacy

class but is unable to participate for unforeseen

reasons..

Students.

Students who register for ESL

instruction will be screened at intake to

determine their literacy level. Those who agree

to participate in the study and who are at a

low level of literacy (as determined through

a brief native language literacy screener) will

be randomly assigned, using a web-based

computer program, to either the explicit

literacy group or the existing ESL group. The

attached chart illustrates our planned process.

Why randomly assign teachers?

Teachers will be randomly assigned to either group

to ensure we are specifically testing the approach

to teaching ESL and not other factors that are tied

to their willingness to teach the new curriculum

or other teacher characteristics. Teachers who

volunteer to teach this new curriculum may differ,

in key ways that affect the outcome of the study,

from teachers who are not interested in teaching

the new curriculum. To control for this selection

bias we will select from a pool of teachers who are

willing to teach the explicit literacy curriculum and

randomly assign them to one group or the other.

Why randomly assign students?

This study will be measuring the effect of the

explicit literacy instruction relative to the existing

ESL instruction. To do so, we will need to compare

the outcomes of the students assigned to each

group after program completion. Similar to our

reason for randomly assigning teachers, if we were

to allow students to self-select into each group, we

may find that students who seek out explicit literacy

instruction are more motivated or differ in other

important ways to those who choose the existing

program. By randomly assigning students, the two

groups are virtually identical in all ways except that

one group receives the explicit literacy instruction,

while the other receives instruction as it is currently

being offered. This way, any differences in outcomes

among students in both groups can be attributed to

the explicit literacy curriculum.

Is random assignment a fair way to select

students who participate in the explicit

literacy curriculum?

Random assignment is fair because it gives all

participating students an equal chance of being

selected for the explicit literacy curriculum.

Personal factors play no role in whether a student

is selected.

No students will be denied services as a result of this

study. If students are not assigned to the explicit

literacy instruction, they are assigned to the class

that they would have participated in had this study

not been implemented.

What is required of the sites to do random

assignment?

The evaluation team will work with each site

to develop a random assignment process that

is not unduly burdensome. We are developing

a simple web-based system that will record

information regarding the students at intake and

will automatically assign students to classes using

a random process. We will work with the sites to

determine the most appropriate time to collect

the intake information, conduct the screening,

and determine the random assignment group. It is

likely that the screening process will occur during

the intake process and the random assignment will

occur when students show up for the first few days

of class. We also will work closely with sites to

ensure that teachers assigned to the explicit literacy

classes avoid disclosing information about features

and techniques in the explicit literacy curriculum.

Will participants assigned to the existing

ESL

curriculum group ever have an opportunity to

receive instruction from the explicit literacy

curriculum?

Assuming the new curriculum proves effective,

which we do not know prior to an evaluation of

the curriculums impact, a copy of such curriculum

will be provided to all programs after the studys

completion, and programs are free to use it with

students.

What happens if a participant drops out of

the program after being randomly assigned to

either group?

Students who drop out of the class assigned to them

will be viewed as if they were still part of the study.

Researchers will continue to collect follow-up data.

Students who return to the school during the study

period will be assigned to the group to which they

were originally assigned.

Random Assignment Flowchart

Student Application

Program Intake

Initial Screening of Students (program

administers usual screening and

assessment to identify beginning ESL

students who would normally attend

study classes)

Student Literacy Screening

(Native language assessment)

Explicit Literacy Instruction

Existing ESL Instruction

Assessment

Assessment

Analysis of Impact

Students with Low or Advanced

Native Language Literacy Levels Students with Limited Native

Language Literacy

(Informed consent obtained)

Included in study classes

First Two Days of Class

..Students check in at site

..Random assignment occurs

..Initial assessment and interview conducted

..Students informed of class assignment

..Students begin class

Random Assignment (Students)

Excluded from study, but students will

receive normal instruction provided by the

program in classes not in the research study