Context for the 12th Grade Motivation Study

Context for the 12^th Grade Motivation Study

As early as 1998, NCES and the National Assessment Governing Board were concerned about school and student participation in NAEP. An AllStates 2000 Task Force was convened to address the issues and make recommendations for improving both school and student participation. The Task Force recognized that there were two related factors for 12^th grade students that had to be addressed: participation and motivation.

In 2001, conversations began on the No Child Left Behind legislation (NCLB) and the requirement that all states receiving Title I funds participate in NAEP reading and mathematics assessments in 4^th and 8^th grades. In the rush to implement the NCLB provisions, the focus on 12^th grade participation took a back seat. As early as 2003, all states and over 99 percent of the schools and more than 95 percent of the students (excluding some special needs students) participated in the NCLB NAEP assessments. However, the participation rate for 12^th grade schools and students continued to decline as illustrated below.

Early in 2002, the Governing Board convened a National Commission on NAEP 12^th Grade Assessment and Reporting followed in 2005 by an Ad Hoc Committee on 12^th Grade Participation and Motivation. NCES began a series of panel discussions with principals, teachers, and district superintendents from the program/operational perspective. The conclusions from the groups were very consistent:

the validity and credibility of the NAEP 12^th grade results are severely compromised by the poor participation of both schools and students and by the perceived low motivation of students to try their best on a test that has no consequences for them or their schools.

The educators went further in their analysis of the situation. They said, in effect, that

while participation rates for both schools and students could be improved with some basic changes to operations, that low motivation is generally assumed to be a problem, is poorly defined and understood, and would be much more difficult to change.

Their bottom line was that if we are unable to improve the motivation of students, then the results will always be, at the least, suspect, and, at worst, misleading.

The principals cautioned that if motivation of students to try on the NAEP assessments is not improved, it will, in the end, create a circular environment where schools will cease to try to help us resulting in participation rates going back down.

This prediction is now being borne out by the states. New York has, for the 2003 and 2005 assessment, refused to participate in 12^th grade assessments because they perceived not only that the test is burdensome to their schools but that because of low motivation the results are misleading. New York, critical to providing a national score, was persuaded by US Department officials to let the NAEP contractor contact the schools and try to get them to participate. Other states such as Maryland, Rhode Island, and Nevada have just refused to permit NAEP 12^th grade assessments in their schools on the principle that the results are not creditable.

The participation and motivation issues have been made more critical with the continued discussions in Congress to support a 10-state pilot study for a 12^th Grade State NAEP.

Much has been done on improving the participation of both schools and students. In 2005, the new NAEP State Coordinators made a special effort to recruit schools bringing the rate to 85 percent, which is comparable to the participation rate of 4^th and 8^th grade schools when it was still voluntary. In 2006 and 2007, the focus was on improving student participation. The student participation rate grew from a low of 66 percent in 2005 to 72 percent in 2006, not good enough but moving in the right direction.

There has been much internal discussion about motivation over the last five years, but no concerted effort to influence student motivation, for NAEP as a program or for other NCES student surveys. NAEP could be criticized for not acting sooner to implement a concerted effort to address a serious threat to the program.

This study, part of an emerging NAEP research agenda around student motivation, seeks to directly ask high school seniors about their motivation to perform well on assessments like NAEP that bear no consequence for them. The study will utilize a series of focus groups with students from small and large schools in urban, suburban, and rural communities to explore 1.) their attitudes towards testing, 2.) the likelihood that they will put forth their best effort on tests that have no personal consequences, and 3.) factors (i.e., incentives) that might influence them to try harder.

A companion study being conducted now is a critical first step in understanding and addressing the issue. The results of the study are not envisioned to provide a definitive answer, but will help to define the magnitude of the problem and can provide insights into whether students can be incentivized to try harder on a low stakes assessment such as NAEP.

The complete research agenda is evolving with the input of the NAEP contractors, the NAEP Validity Studies Panel, and the ETS Design and Analysis Committee, as well as our state partners. The next step is to go back to the principals that were on our working panel to help interpret the findings and make suggestions improving motivation on NAEP.

2