Supporting Statement

NATIONAL ASSESSMENT OF

EDUCATIONAL PROGRESS

Volume I

Supporting Statement

NAEP Exploratory Survey of State Directors on State High School Assessment Policies and Plans

OMB# 1850-0803 v.44

March 3, 2011(rev.3/17/11)

Submittal Related Information

This material is being submitted under the generic National Center for Education Statistics (NCES) clearance agreement (OMB #1850-0803), which provides for NCES to conduct various procedures (such as field tests, cognitive interviews, and exploratory interviews) that will allow it to continue to develop, test, and improve its survey and assessment instruments and methodologies.

Study Design and Content

The National Assessment of Educational Progress (NAEP) is a federally authorized survey of student achievement at grades 4, 8, and 12 in various subject areas, such as mathematics, reading, writing, science, U.S. history, civics, geography, economics, and the arts. NAEP is administered by NCES, in the Institute for Education Sciences, of the U.S. Department of Education. To explore the ways in which NAEP at the high school level can be useful in meeting state needs, NCES proposes to conduct a brief, exploratory online survey of 51 state assessment directors about current and impending state policies governing assessments of all public school students at the high school level. The state assessment directors are the staff directing student assessment development and operations in each state department of education.

The results of this survey will be used internally by NCES in two ways: (a) to explore the feasibility of expanding state participation in the NAEP grade 12 pilot; and (b) to inform NAEP’s research on 12th grade preparedness.

In particular, the information collection will focus on (1) the policies requiring students to pass an assessment (or assessments) as a requirement for high school graduation and (2) state policies mandating assessments in specific high school subjects to meet accountability reporting requirements. State policies on statewide student assessment, and potential change in state policies, are being affected across the states by several factors, including: a) the push for implementation of the Common Core State Standards in language arts and mathematics, especially the assessments being developed through the Race to the Top Multi-State Assessment Consortia¹, b) federal requirement for reporting assessment results for all students in at least one grade in mathematics, English, and science, and c) efforts to ensure that the high school diploma represents learning and knowledge that is consistent with standards for preparedness for college entrance or entrance to the workforce.

In 2009, NAEP conducted the State Grade 12 Pilot project which assessed students in mathematics and reading with state representative samples in 11 states. NCES is also examining the validity of the NAEP 12^th grade assessment results as indicators of students’ college and career preparedness. The question of further development and use of NAEP at the high school level for state education policymakers is likely to be strongly influenced by the three factors outlined above. That is, how will NAEP contribute to, support, or be aligned with the student assessment system needs of states at the high school level?

The spring 2011 exploratory survey will be conducted via a web-based instrument implemented by the study contractor, the Council of Chief State School Officers² (CCSSO), the contractor for the NAEP State Technical Operations Cooperative. The results, along with existing information on assessment policies across the states collected by CCSSO, will be compiled and reported to NCES by CCSSO, to assist NCES in effectively addressing key planning and implementation issues for NAEP. Information from the 51-state survey will allow NCES staff to make informed decisions about implementing NAEP in each state and to improve planning and dialogue with assessment directors about key issues such as the role of Common Core State Standards in state assessments, change in state graduation policies, and appropriate age and grade for high school assessments. In the past, CCSSO has assisted NCES and the Governing Board with information about state policies regarding review procedures for secure tests, policies and procedures for informing parents of student participation in assessments, testing windows for state assessment programs, and use of computer technology in state assessments.

The exploratory survey will focus on the following topics:

• Current state policies on high school student assessments required for all students for graduation in core subject areas, and state support for other assessments during high school,

• State interest and plans for NAEP grade 12 assessments, and

• Changes in state policies on assessment at the high school level related to assessing college or career preparedness.

Data Collection Process

This web-based, exploratory survey will be administered to directors of student assessment in all 50 states and the District of Columbia. CCSSO has an established network of all state assessment directors, through which it regularly collects, shares, and reports information. CCSSO maintains a current list of state directors (updated monthly) and their contact information. The survey will utilize the CCSSO electronic network of all state assessment directors and a web-based instrument, called “State Assessment Profiles”, for collecting responses to the exploratory survey from the state directors. The NAEP exploratory survey will be distributed online in March, 2011, and the target survey period will be designated as two to three weeks. Follow-up requests will be sent by email or made by phone by CCSSO staff. The surveys will be completed in April, 2011.

Each of the state assessment directors for the 50 states plus District of Columbia will be contacted in advance by CCSSO via email and informed of the purpose of the NAEP survey (as outlined above), the relationship of the survey to the NAEP State project with CCSSO, the number of questions, and the expected time to complete it. CCSSO will confirm that the current contact person is the state director intended to complete the survey, and if not, the appropriate person will be identified and notified. The survey and questions will be accessed through the password-protected web-based tool. The contacting email texts and the specific questions to be asked of each state director are listed in Appendices A, B, C, and D respectively.

Under the terms of the CCSSO project contract with NCES, the information from the exploratory NAEP survey will be reported to NCES prior to the May 2011 meeting of the Education Information Management Advisory Consortium (EIMAC) Permanent Standing Task Force on Assessment. Basic analyses and reporting of the data from the survey will be provided by CCSSO for the meeting. Further analysis of the survey data following review and discussion at the May 2011 meeting or in response to inquiries of NCES will be conducted by CCSSO. The complete data set will be made available to NCES.

Information Collection

No individual respondent identifying information will be included in the survey database. The name of the state will be associated with the information provided but individual respondent names and contact information will not be associated with the data.

The following text will precede the actual survey questions as part of the web-based survey:

Your name and contact information will only be used for follow-up if there are any individual responses that require further clarification. Your name and contact information will not be used for any other purpose and will not be maintained in the data file.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this voluntary survey is 1850–0803. The time required to complete this survey is estimated to average 30 minutes per response, including the time to review instructions, gather the information needed, and complete and review the survey. If you have any comments concerning the accuracy of the time estimate or suggestions for improving this survey, please write to: U.S. Department of Education, Washington, DC 20202–4537. If you have any comments or concerns regarding the status of your individual submission of this survey, write directly to: NAEP/2011 State Assessment Directors Survey, National Center for Education Statistics, 1990 K Street, NW, Washington, DC 20006.

Respondent Burden and Costs to the Federal Government

Based on a preliminary review of the survey questions with four state directors, the estimated time for each state director’s response is 30 minutes, for a total of 51x0.5 hrs = 25.5 respondent burden hours.

The total estimated cost to the federal government for conducting this survey is $120,000:

• $50,000 for web survey design and database development and maintenance

• $70,000 for survey development, staff management, data analysis and reporting.

Schedule

The schedule for the exploratory NAEP survey of state directors is:

Date	Event
March 2011	Send notice of survey to 51 directors
April 2011	Complete data collection and report draft data, findings and analysis to NCES
May 2011	Further analysis of survey data following May EIMAC meeting
June 2011	Submit final data, findings, and analysis to NCES

Appendix A – Communication with the State Directors – Initial Request

Each State Director will receive an email from CCSSO asking for responses to questions regarding the state’s high school assessments. The email text will read:

For the spring 2011 Survey of State Directors, CCSSO requests that you provide information in response to several questions about state policies and programs. The purpose of the voluntary survey is to provide information to the National Center for Education Statistics (NCES) regarding state high school assessments policies and programs that will allow NCES staff to make informed decisions about implementing NAEP in each state and to improve planning and dialogue with state directors about key issues with assessments at the high school level. The survey is being conducted under the CCSSO contract with the National Center for Education Statistics (NCES) for the NAEP State Technical Operations Cooperative. A total of five questions will be sent through an internet web survey tool. The expected response time is 30 minutes. CCSSO asks that you complete the voluntary web survey by [initial survey window will be two weeks time from sending date]. The name of your state will be associated with the information provided but individual respondent names and contact information will not be associated with the data. Thank you very much for your cooperation.

If you have any questions about the survey, please contact CCSSO by phone or email [contact person—

Lauren Stillman, 2023367049, [email protected]].

Alyssa Alston, CCSSO project director

Rolf Blank, CCSSO research director

Appendix B – First Follow-up Contact

After two weeks from the date the voluntary survey web instrument was provided to state directors, CCSSO will send a follow-up email to non-respondents and contact non-respondents by telephone. The text of the email and phone contact will be the following:

CCSSO recently requested that you provide responses to five questions for the spring 2011 Survey of State Directors regarding state high school assessments policies and programs. The voluntary survey is being conducted by CCSSO for the National Center for Education Statistics (NCES) to assist the staff with decisions related to NAEP and the state-level program. We are providing the web link to the voluntary survey again. Please complete your responses to the questions by [one week from the date of the follow-up contact], and please let us know if you will be able to complete the voluntary spring 2011 survey of State Directors. We very much appreciate your cooperation and support for this important activity of CCSSO with NCES.

If you have any questions about the survey, please contact CCSSO by phone or email [contact person—

Lauren Stillman, 2023367049, [email protected]].

Alyssa Alston, CCSSO project director

Rolf Blank, CCSSO research director

Appendix C – Second Follow-up Contact

After the one week follow-up period, CCSSO will send another follow-up email and contact any remaining non-respondents by telephone. The text of the email and phone contact will be:

CCSSO has requested that you provide responses to five questions for the spring 2011 Survey of State Directors on state high school assessments policies and programs. The voluntary survey is being conducted by CCSSO for the National Center for Education Statistics (NCES) to assist the staff with decisions related to NAEP and the state-level program. We are providing the web link to the voluntary survey again. The closing date for the survey is [provide date one week from second follow-up]. Please complete your responses to the questions by that date or let us know if you will not be able to complete this voluntary survey. We very much appreciate your cooperation and support for this important activity of CCSSO with NCES.

If you have any questions about the survey, please contact CCSSO by phone or email [contact person—

Lauren Stillman, 2023367049, [email protected]].

Alyssa Alston, CCSSO project director

Rolf Blank, CCSSO research director

Appendix D – Exploratory NAEP Survey of State Directors (spring 2011)

Name: __________________ Contact Information: ____________________________

Your name and contact information will only be used for follow-up if there are any individual responses that require further clarification. Your name and contact information will not be used for any other purpose and will not be maintained in the data file.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this voluntary survey is 1850–0803. The time required to complete this survey is estimated to average 30 minutes per response, including the time to review instructions, gather the information needed, and complete and review the survey. If you have any comments concerning the accuracy of the time estimate or suggestions for improving this survey, please write to: U.S. Department of Education, Washington, DC 20202–4537. If you have any comments or concerns regarding the status of your individual submission of this survey, write directly to: NAEP/2011 State Assessment Directors Survey, National Center for Education Statistics, 1990 K Street, NW, Washington, DC 20006.

1A. For the English Language Arts state high school assessment(s) required for graduation (not EOC), what is the grade level, or grade range, of the content tested?

1B. For the Mathematics state high school assessment(s) required for graduation (not EOC), what is the grade level, or grade range, of the content tested?

1C. For the Science state high school assessment(s) required for graduation (not EOC), what is the grade level, or grade range, of the content tested?

1D. For the Social Studies state high school assessment(s) required for graduation (not EOC), what is the grade level, or grade range, of the content tested?

2. If your state administers a high school assessment that students are required to pass for graduation, are students included in and accommodated to the test based on NCLB rules or are students included based on separate state policy?

___ State does not administer assessment that students must pass for graduation

___ State uses NCLB rules to include and accommodate students.

___ State uses separate state policy to include and accommodate students—

If yes, what is the separate state policy? _____________________________________.

3. Does your state provide funding support for students to participate in other high school assessments that are not statewide assessments? (Please choose No or Yes)

___ No

___ Yes, state provides support for students to take: (check all that apply)

___ Advanced Placement tests

___ International Baccalaureate tests

___ PSAT

___ SAT

___ ACT

___ Armed Services Vocational Aptitude Battery (ASVAB)

___ Other College Board tests

___ Other ACT tests

___ Other assessments (please specify)

4A. In 2009, 11 states participated in the Grade 12 NAEP State Pilot in reading and mathematics. If there is another voluntary Grade 12 NAEP State Assessment, is your state likely to participate? (Please choose one)

___ Yes ___ No ___ Possibly ____ Not sure/Cannot answer

4B. If Yes or Possibly, what do you see as the main reason(s) that your state will or may participate in a Grade 12 NAEP State Assessment?

___ A second indicator of student achievement for state

___ Assessment results that can be compared to the nation or other states

___ A benchmark for international comparisons for the state

___ Access to state-level data from NAEP background questionnaires completed

by students and school personnel

___ Decision by state policymaker for state to participate

___ Other: _____________________________

5. Has your state made any policy change affecting high school assessments related to the goal of assessing student college or career preparedness/readiness within the past 3 years? (Please choose one)

___ No state policy changes regarding high school assessments related to college or career readiness/preparedness

____ Yes, state high school assessments were recently changed to address college or career preparedness/readiness. The following types of changes were made (check all that apply):

___ End of course tests were implemented as graduation requirement

___ New performance levels or passing score required on graduation test

___ Test content was changed

___ High school assessment score can be used for college course credit

___ Other change: ___________________________

1 A large majority of the state departments of education are participating in one of two Assessment Consortia projects supported by grants through the U.S. Department of Education Race to the Top program. Many states will be working to incorporate the new student assessments developed with the Consortia into their state assessment programs.

2 CCSSO is a nonpartisan, nationwide, nonprofit organization of public officials who head departments of elementary and secondary education in the states, the District of Columbia, the Department of Defense Education Activity, and five U.S. extra-state jurisdictions. CCSSO provides leadership, advocacy, and technical assistance on major educational issues. For this exploratory survey, the CCSSO contract director is Alyssa Alston and the research director is Rolf K. Blank.

2