Justification

Volume I:

Request for Clearance for Pretest of the Proposed Fast Response Survey System (FRSS) 99: District Survey of Dropout Prevention Services and Programs

OMB# 1850-0803 v. 24

March 15, 2010

Justification

The National Center for Education Statistics (NCES), U.S. Department of Education (ED), requests OMB approval under the NCES system clearance for Cognitive, Pilot, and Field Test studies (OMB #1850-0803) to pretest a district survey about dropout prevention services and programs, which is part of the Fast Response Survey System (FRSS). This survey, referred to as FRSS 99, will provide the first nationally representative data on this topic by capturing a current snapshot of dropout prevention services and programs available within the nation’s public school districts, and it will cover factors and methods used to identify students at risk of dropping out, the entities districts partner with in their dropout prevention efforts, approaches to reenrolling students who have dropped out, and information used by the district in determining whether to implement additional dropout prevention efforts. The purpose of the pretest is to identify and correct any potential issues with the content and format of the survey before its full scale implementation later this year, so as to assure that it captures the intended meaning of each question and minimizes the burden imposed on respondents. The request to conduct the full survey will be submitted at a later date under OMB generic clearance for quick response surveys (OMB #1850-0733), which are authorized under Section 153 (a) of the Education Science Reform Act of 2002 (Public Law 107-279), stating that the purpose of NCES is “to collect, report, analyze, and disseminate statistical data related to education in the United States and in other nations.”

Design

Overview of Survey Development

NCES has contracted Westat to conduct the FRSS 99 survey and its pretest, including a literature review on dropout prevention programs and services and four rounds of feasibility calls that informed the survey design. The first two rounds of feasibility calls were conducted to identify topics that could be addressed in a short questionnaire and they informed the initial draft of the survey. The third and fourth rounds were conducted to assess the clarity and relevance of the developed survey items, and to gauge whether respondents thought they could answer the questions without too much burden. In the third round of feasibility calls, respondents reviewed the draft survey and provided feedback over the phone for all survey items. Based on their feedback, the survey was revised and, in the fourth round, respondents provided feedback only about the items and definitions that had changed since the third round of calls. The four rounds of feasibility calls, each with nine or fewer respondents, were conducted between March 2009 and February 2010. The resulting draft of the survey was then reviewed by the NCES Quality Review Board (QRB) and revised accordingly to prepare it for the pretest described here.

Consultations Outside of Agency

In addition to the four rounds of feasibility calls conducted with district respondents and some school-level staff (in the first round only), general topics were identified through literature reviews and in consultation between NCES and Mark Dynarski, an expert on dropout prevention and Director of the IES What Works Clearinghouse. Additional comments were requested on various drafts of the survey from several reviewers outside of NCES including, Ed Pacchettii (Special Assistant to the Senior Advisor on the Secretary’s Initiative on College Access), Theda Zawisza (Office of Elementary and Secondary Education (OESE)), and Braden Goetz (OESE).

Sample, Burden, and Cost

In this submission, we are requesting approval to pretest the revised questionnaire with 15 respondents from school districts around the nation, who will be asked to review, complete, and fax back the 3-page paper and pencil survey, and will be invited to provide their feedback by telephone. Administrators in districts of different sizes, urbanicity, and regions of the country, who are knowledgeable about their district’s dropout prevention services and programs will be identified and recruited by phone. In order to recruit 15 districts to participate in the pretest, we anticipate contacting approximately 70 districts. On average, recruitment calls with respondents who agree to participate in the pretest are expected to take about 10 minutes to explain the purpose of the pretest and set up an appointment to discuss the survey; all other recruitment calls are expected to take about 3 minutes. The questionnaire is expected to take approximately 30 minutes to complete and verbal feedback is expected to take another 30 minutes or less. The total expected response burden is about 1 hour and 10 minutes per respondent, plus 2.75 hours for contacts to districts that do not participate in the pretest, for a total estimated burden time of 20.25 hours. The feedback obtained will be used to revise the survey. We anticipate that the estimated cost of the pretest to the federal government will be approximately $10,000. Following the pretest, NCES will submit the revised questionnaire along with an official request for OMB clearance to conduct the national study of FRSS 99.

Data Collection Instrument

A cover letter (Attachment 1) and questionnaire (Attachment 2) will be emailed or faxed to each participating district. The cover letter thanks the respondent for agreeing to participate in the pretest, introduces the purpose and content of the survey, indicates that their participation is voluntary, includes instructions on how to complete and return the survey, includes questions for respondents to consider while completing the survey that will help in providing feedback about the survey, and provides contact information should any questions arise before the scheduled discussion with the survey manager. On the cover of the survey, respondents are assured that their participation is voluntary and their answers may not be disclosed or used in identifiable form for any other purpose unless compelled by law. The public law is cited on the front page of the survey (attachment 2).

The survey is designed to collect basic information on dropout prevention services and programs offered for students enrolled in a public school district. The first two questions ask about the services and programs that are available in the district to address the needs of students who are at risk of dropping out. Some of the services and programs presented are those that may be offered at elementary school, middle/junior high school, and high school instructional levels (e.g., tutoring, summer school for grade retention, and remediation classes), whereas others are typically only offered to students at the secondary level (e.g., General Education Development (GED) preparation courses, and early graduation options). Because the levels at which services and programs may be offered vary, only the first question on the survey asks about services and programs offered at multiple instructional levels. Response options for question 2 are not tied to an instructional level because it is expected that these options are largely available only at the high school level.

The literature and district respondents in feasibility calls indicated that an important component of dropout prevention is offering educational options that are relevant to students’ life or career goals. To capture this in the survey, question 3 asks about various educational options available in the district (e.g., career/technical high school, dual enrollment in postsecondary courses with a career/technical focus) and whether students who are at risk of dropping out participate in those educational options.

Another service that was discussed by respondents in feasibility calls as one that helps to prevent students from dropping out is childcare services while a teen parent is attending classes. Question 4 asks whether the district provides or subsidizes childcare for teen parents while they are attending classes.

The literature and district respondents in feasibility calls also indicated that transitions from a school of one instructional level to a school at a higher instructional level can be particularly difficult for students who are at risk of dropping out. Questions 5 and 6 ask about the processes and supports used by the district to help students in such transitions. Transitional supports may include assigning each student a student or adult mentor or offering an advisement class during the first year at the new school.

Question 7 asks about different types of mentors used in the district specifically to address the needs of students at risk of dropping out. The list of the types of mentors was developed based on the literature review and feedback from district respondents. Examples of mentors include student mentors, teachers, school counselors, or school administrators who formally mentor students; adult mentors employed by the district whose only job is to mentor students; and community volunteers who mentor.

One type of program that has been discussed in the literature as effective in reducing the dropout rate in schools and districts is a school-wide or classroom-wide program that targets to reduce behavioral problems. Question 8 asks whether any of the schools in the district use formal programs designed to reduce behavioral problems in schools or classrooms. Because districts may employ these programs at one or multiple instructional levels, respondents are asked to report by instructional level.

Question 9 asks about the factors used in the district to identify students who are at risk of dropping out. Again, these were identified based on a review of the literature and through discussions with district respondents during feasibility calls about the factors commonly used to identify at-risk students. Among some of the factors that may indicate that a student is at risk are truancy or excessive absences, academic failure, behaviors that warrant suspension or expulsion, and substance abuse. Question 10 asks if the district has a standardized method of identifying students who are at risk of dropping out (e.g., a standardized checklist of at-risk behaviors or an electronic warning system).

Because districts will often work with other entities to address the needs of students who are at risk of dropping out, Question 11 asks about those entities. The list of entities included in the survey was developed based on a review of the literature and discussions with district respondents in feasibility calls. Some of the entities listed include child protective services, local businesses, community mental health agencies, and churches or community organizations (e.g., Boys and Girls Clubs, United Way, and Lion’s Clubs). Question 12 and 13 ask about information provided to students who are considered to be at a higher risk of dropping out, including information about the employment or financial consequences of dropping out and the education and training options available to them (e.g., alternative schools and programs, job training/GED combination programs, GED or adult education programs, and job training programs).

Question 14 and 15 ask about dropout recovery efforts in the district, including whether the district tries to determine the status of students who were expected to return in the fall but who do not return as expected, and whether the district follows up with students who have dropped out before the next school year to encourage them to return.

The final, Question 16, asks about information the district uses to determine whether to implement additional dropout prevention efforts in the district. Some examples of types of information that may be used include dropout rates, graduation rates, and attendance rates. As with the other items in this survey, the list of the types of information districts use was developed based on a review of the literature and from feasibility calls discussions with district respondents.

Timeline

Pretest activities are expected to begin as soon as approval for them is received from OMB. It is anticipated that participant recruitment, completion of the pretest calls, write up of the memorandum summarizing the results, and survey revision will take approximately 6 weeks.