Evaluation of the Early Warning and Intervention Monitoring System (EWIMS)

On-Time Graduation Project
Purpose of the Project
One in four students in the United States fails to
graduate from high school, and graduation rates are
significantly lower for students who are racial or ethnic
minorities, economically disadvantaged, migrant and/or
limited-English proficient, or receiving special education
services. However, strong foundational research
highlights critical indicators in ninth and tenth grade that
powerfully predict whether students are “on track” for
high school graduation. These indicators, which typically
focus on student engagement measures (attendance)
and course performance (grades, credits earned), can be
used as part of an early warning system to flag at-risk
students early, assign appropriate interventions, and get
students back on track for graduation.
As part of Ohio’s Comprehensive Continuous
Improvement Plan, the state has committed, through
adoption of the Ohio Improvement Process, to
promoting student success through data-driven decision
making, targeted programming, ongoing student
monitoring, and evaluation of improvement process
effectiveness. Use of an early warning system is one
strategy that schools can use within the Ohio
Improvement Process.
One such early warning system is the Early Warning and
Intervention Monitoring System (EWIMS) developed by
the National High School Center. The EWIMS model
encourages educators to use data to consider and
provide appropriate interventions to students, and
provides a means for monitoring student progress over
time. At the core of EWIMS is the use of simple tools that
encourage educators to routinely examine indicators that
identify students as “off track” and take action.
Despite increasingly widespread implementation of early
warning systems by states, districts, and schools, there
have been no rigorous studies testing the impact of using
an early warning system to improve student outcomes
such as staying in school, progressing in school, and
probability of graduating. There is also little research
examining how using an early warning system can shape
a school’s culture for data use (including how schools
allocate their limited dropout prevention resources).
To address this gap, the On-Time Graduation Project will
provide the first rigorous test of the impact of an early
warning system. This project, funded by the U.S.

Department of Education’s Institutes of Educational
Sciences, is being conducted by the Midwest Regional
Educational Laboratory at the American Institutes for
Research.
The project will examine the impact of EWIMS on (1)
student outcomes including student risk status for
dropout, scores on graduation tests, persistence and
progress in school and likelihood of on-time graduation;
and (2) school outcomes including how schools allocate
dropout prevention interventions for students and their
data-use culture. Participating in this high-profile, largescale project will give your school and district an
opportunity to access these innovative resources at no
cost and help inform educational policy in Ohio and at
the national level.

Project Approach
The project will take place in approximately 70 high
schools in Ohio during the 2013–14 and 2014–15 school
years. All schools that participate in the project will
receive the early warning system at no cost. Participating
schools will be assigned by lottery to receive access to
the EWIMS model, including the tool and high-quality
professional development for implementation, in the
2013–2014 school year or the fall of 2015 (following the
completion of the project). Schools assigned to receive
the EWIMS model in the fall of 2015 will continue
“business as usual” practices to identify at-risk students
and allocate dropout prevention resources through
2014–15.
The project will examine student outcomes for all
students in Grades 9 and 10 during the 2013–14 school
year and all students in Grades 9 through 11 during the
2014–15 school year for schools that implement EWIMS
in 2013–2014 and those that do not implement EWIMS
until the fall of 2015. All student outcome data will be
collected from school or district administrative data, the
early warning system tool, or the Ohio Department of
Education. There will be no primary student data
collection for this project. To understand how EWIMS
may impact schools, all participating schools will be
asked to complete an annual Web-based survey about
data use practices, and schools implementing EWIMS
during the two-year project may be asked to participate
in interviews about their experiences using the tool.

1

EWIMS Seven-Step Implementation Cycle

EWIMS Intervention

STEP 1—
Establish Roles
& Process

The EWIMS process is designed to i dentify students who
are at risk of dropping out of school and to support and
monitor at-risk students through school-wide strategies
and targeted interventions. EWIMS is currently in use in
67 districts in six states, and the tool has been
downloaded more than 20,000 times from the National
High School Center’s website.
EWIMS Tool. At the heart of the EWIMS model is an early
warning data tool used to flag students as “at risk” based
on attendance, course performance (grades, credits,
GPA), and behavior indicators. The tool enables users to
identify students who are at risk of dropping out of
school, record assignments to available interventions,
and monitor students’ response to those interventions.
The tool provides a number of reports accessible to
users, including:
 School-Level Reports: Graphical reports that show
trends in student risk status across the school
 Student-Level Reports: Lists of students, grouped by
indicators of risk and/or assigned intervention
programs, that include summary information
 Detailed Student Reports: Simple reports that can be
generated for each individual student and show
student information, indicators of risk status, and any
assigned interventions
 Student-Level Intervention Summary Reports: Lists of
students, their indicators of risk (flagged or not
flagged), and the number and types of interventions to
which each student was assigned
EWIMS Implementation Process. In addition to the tool,
the National High School Center has devised a seven-step
EWIMS implementation process to support
implementation. The process guides users to make
informed decisions about how to use data to support atrisk students and how to continue to monitor their
progress over time. In addition to focusing on individual
students, the process guides users to examine the
success of specific supports or interventions and to
examine possible systemic issues (school climate) that
may relate to dropout trends.

STEP 7—
Evaluate &
Refine EWIMS

STEP 2—Use the
EWIMS Tool

STEP 3—Analyze
EWIMS Data

STEP 6—
Monitor
Students

STEP 5—Assign
& Provide
Interventions

STEP 4—
Interpret
EWIMS Data

Initial Criteria for Participation
The project will include approximately 70 high schools in
Ohio. To qualify, schools must (1) have at least 150 ninthgrade students; (2) a graduation rate between 25 and 95
percent and (3) not already be implementing an early
warning system tool for using data to flag at-risk
students.

Project Timeline
Through January 2014, the project team will discuss
participation with districts and schools that meet the
initial criteria and conduct on-site or virtual meetings
with school principals, guidance counselors and dropout
prevention coordinators. Participating schools will sign
memoranda of understanding in January 2014 and be
assigned by lottery to receive access to the EWIMS
model, including the tool and high-quality professional
development on the implementation process, in January
2014 (treatment group) or the 2015–2016 school year
(“business as usual” control group). Training for EWIMS
implementation in treatment schools will begin in early
2014. Data collection activities will continue throughout
but not beyond the 2014–15 school year.

For Additional Information
For more information or to begin a conversation about
partnering on this project, contact Dr. Nicholas Sorensen
([email protected] or 312-283-2318) or Dr. Mindee
O’Cummings ([email protected] or 202-403-5254).

2