Volume 1- Justification with Att 1 (recruitment call script) and Att 2 (cover letter)

Volume I:

Fast Response Survey System (FRSS) 107: High School English Language Learner (ELL) Programs --Pretest

OMB# 1850-0803 v. 132

March 23, 2015

National Center for Education Statistics

U.S. Department of Education

Justification

The National Center for Education Statistics (NCES), U.S. Department of Education (ED), requests OMB approval under the NCES system clearance for Cognitive, Pilot and Field Test studies (OMB #1850-0803) to conduct two rounds of pretest calls for a Fast Response Survey System (FRSS) survey #107 on high school English language learner (ELL) programs in public school districts. The Office of English Language Acquisition (OELA) requested that NCES conduct this FRSS survey.

The purpose of the pretest calls is to identify and correct any potential issues with the content and format of the survey before conducting full scale implementation, to ensure that the survey captures the intended meaning of each question and minimizes the burden imposed on respondents. The request to conduct the full survey will be submitted at a later date under OMB generic clearance for quick response surveys (OMB#1850-0733), which are authorized under the Education Science Reform Act of 2002. NCES has contracted Westat for all stages of this survey.

Design

Overview of Survey Development

NCES has contracted Westat to develop the survey instrument and collect data for FRSS 107. FRSS has established procedures for developing short surveys on a wide variety of topics. The techniques used to shape the survey design on FRSS 107 include literature reviews on ELL programs, input from the NCES Quality Review Board (QRB), several rounds of feasibility calls, and up to two pretests. The current survey reflects lessons learned from topics and issues identified through literature review, with modifications based on three rounds of feasibility calls with public school district personnel most knowledgeable about high school ELL programs. The first round of feasibility calls was conducted with 12 respondents in August and September 2014 (OMB# 1850-0803 v. 109). Because this is a new survey topic, the first round of calls used an open-ended interview guide to learn more about the ELL populations ages 14 to 21 that school districts serve, the terminology districts use regarding these populations, and the characteristics of the ELL programs and services they provide at the high school level. The second round of feasibility calls was conducted with 13 respondents in October and November 2014 and the third round of feasibility calls was conducted with 15 respondents in January and February 2015. During the second and third rounds of feasibility calls, respondents were asked to review draft survey questions, instructions, and definitions based on the initial round of feasibility calls. During the pretest, we will test all questions on the questionnaire and also obtain estimates of the respondent time required to complete the survey.

NCES Review and Consultations Outside of Agency

The NCES QRB members reviewed a draft list of questionnaire topics prior to the submission of the OMB package for the feasibility calls. Revisions were made to the list of topics based on input from the reviewers, and the list was used to develop an interview guide for the feasibility calls. During the second round of feasibility calls, a draft questionnaire was developed with input from OELA and OCTAE. During the later rounds of the feasibility calls, revisions were made to the draft questionnaire with input from OELA. Following the last round of feasibility calls, the QRB reviewed the draft questionnaire, and revisions were made based on their input. The revised version is will be used for the pretest. In addition to staff from each of the three Divisions at NCES, the QRB also included staff from OELA; the Office of Career, Technical, and Adult Education (OCTAE); and the Office of Postsecondary Education (OPE). The QRB members for this survey are listed below:

Melissa Escalante, OELA

Carlos Martinez, OELA

Debra Suarez, OCTAE

Ricardo Hernandez, OCTAE

Lenore Garcia, OPE

Chris Chapman, NCES

Eugene Owen, NCES (PIAAC)

Arnold Goldstein, NCES (NAEP)

Grady Wilburn, NCES (NAEP)

Richard Reeves, NCES (IPEDS)

Sharon Boivin, NCES (ATES)

Kashka Kubzdela, NCES

Sample, Burden, and Cost

In this submission, we are requesting approval to conduct up to two rounds of pretest calls with the revised questionnaire, with 15 or fewer respondents in each round from school districts around the nation. School districts will be recruited to participate in pretest calls based on various district characteristics including level (secondary or unified), size, urbanicity (locale), and geographic region. Respondents will be recruited by telephone and will be identified as the person in the district who is most familiar with high school ELL programs.

Telephone interviewers will recruit participants for the pretest calls using the recruitment script in attachment 1, and schedule an appointment to complete the pretest calls with cooperating district-level personnel. Following telephone recruitment, interviewers will email a cover letter and draft questionnaire to the participating districts (as discussed below in the Data Collection Instrument section). Respondents will be asked to review, complete, and fax back the paper and pencil questionnaire, and will be scheduled to participate in a telephone debriefing.

In order to recruit 15 respondents per round, we anticipate contacting 45 public school districts (Table 1). On average, recruitment calls with respondents who agree to participate in the pretest calls are expected to take about 10 minutes to explain the purpose of the pretest and set up an appointment to discuss the questionnaire. All other recruitment calls are expected to take about 3 minutes. The questionnaire is expected to take respondents approximately 45 minutes to complete. The pretest debriefing interview is expected to take approximately 30 minutes. The total estimated burden is approximately 48 hours for two rounds of pretest calls. We anticipate that the estimated cost to the federal government will be approximately $5,000 for each round of pretest calls.

Table 1. Maximum burden time for each of up to two rounds of pretest calls for FRSS 107

Respondents	Number of Respondents in Each Round	Number of Responses1 in Each Round	Burden Hours per Respondent	Total Burden Hours
Recruitment – Districts not participating in the pretest	30	30	0.05	2
Recruitment – Districts participating in the pretest	15	15	0.17	3
Pretest questionnaire debrief	15	15	1.25	19
Total per round	45	60	-	24
Total for two rounds	90	120	-	48

¹ Counts each contact (e.g., recruitment and debriefing interview with the same respondents are counted separately).

Data Collection Instrument

For each round of pretest, a cover letter and draft questionnaire will be emailed to each participating school district. The cover letter and questionnaire appear in attachments 2 and 3. The cover letter thanks the respondent for agreeing to participate in the pretest, introduces the purpose and content of the survey, indicates that participation is voluntary, indicates that respondents should complete the questionnaire and fax it back to Westat, includes questions for respondents to consider while completing the questionnaire, and provides contact information should any questions arise before the scheduled discussion with the survey manager. On the cover letter and on the cover of the survey, respondents are assured that their participation is voluntary and their answers may not be disclosed or used in identifiable form for any other purpose unless compelled by law. The public law is cited on the cover letter and the front page of the survey. The materials for the second pretest round (if it takes place), will be similar, except the survey instrument will include the modifications that will result from the first round. The current instrument is discussed below.

Questionnaire

The version of the questionnaire being submitted for the pretest is half a page longer than the final FRSS 107 questionnaire will be. The data requester (OELA) would like to retain all of the current items through pretesting. Based on feedback from the pretest, the questionnaire will be reduced to the allowed three pages of items. This approach has been used successfully in other FRSS surveys.

The purpose of this FRSS survey is to collect national data from school districts on programs and services designed to serve high school ELLs ages 14 to 21, with a particular interest in ELLs who may be at risk for dropping out or not graduating on time. Little is known about the characteristics of older adolescent and young adult ELLs, and there are no national data available on the characteristics of high school education programs and services designed to serve them. Respondents will be asked about instructional programs/approaches currently provided to high school ELLs, including whether they have a newcomer program for high school ELLs and the characteristics of such a program, approximately how many high school ELLs receive content instruction or support in their native language, the availability of materials and services in native languages, and where ELLs are primarily receiving their high school education (e.g., regular high school, alternative school).

Respondents will also be asked about their district’s use of online or computer-based programs for high school ELLs, other programs and services available to high school ELLS (e.g., summer school, remediation classes), high school ELLs’ participation in these programs and services, and programs or services designed specifically for high school ELLs. Additional items ask about relationships districts have with other organizations to provide high school ELLs with academically-oriented programs or services, how often ELLs ages 18 to 21 have newly enrolled in high school in the district, what information districts provide to ELLs ages 18 to 21 seeking to newly enroll in the district, factors taken into account when ELLs ages 18 to 21 seek to enroll (e.g., English proficiency level), and the number of high school ELLs in the district.

Question 1 is a yes/no question that asks which of the listed ELL instructional programs/approaches are currently provided for English learners in high school. Response options include English as a Second Language (ESL) class periods, Pull-out English as a Second Language (ESL) instruction, sheltered content classes, bilingual instruction in content classes, dual language/two-way immersion programs, and support in mainstream content classes. Respondents can also specify “other.”

Question 2 is a yes/no question that asks whether districts have a newcomer program for English learners in high school. If respondents answer yes, they will be directed to answer questions 3 through 5 about that program. If respondents answer no, they will be directed to question 6.

Question 3 asks about the structure of the high school newcomer program. Response items include whether the program consists of a full-day program, half-day program, class periods that total less than half a day, or an after school-school program. Respondents can also specify “other.” Respondents are able to check all that apply.

Question 4 asks for the typical length of time a high school English learner spends in the district’s newcomer program and the maximum length of time a high school student is allowed to stay in the program. Response items include 1 semester or less, 2 semesters, 3-4 semesters, and more than 4 semesters. Respondents can also check “don’t know” for typical length and “no set maximum” for maximum length.

Question 5 is a yes/no question that asks for the types of services included in the district’s high school newcomer program. Response items include English as a Second Language (ESL), sheltered content instruction, content instruction in the students’ native language, native language literacy/language arts instruction, cross-cultural/orientation to the United States, cross-cultural/orientation to school, organization/study skills, and career/vocational education/apprenticeships. Respondents can also specify “other.”

Question 6 is a yes/no question that asks whether the district uses online or computer-based programs for high school English learners for the following purposes: English language acquisition, English language and literacy instruction, content area instruction, native language support in content area instruction, organizational and study skills, or some other purpose.

Question 7 asks for an estimate of how many high school English learners in the district receive 1) content instruction and 2) instructional support in their native language. Respondents are asked to report separately on the use of native language for English learners whose language is 1) the most common native language in the district, 2) the second most common native language in the district, and 3) any other non-English language. Response options are “no students,” “few students,” “some students,” and “most or all students.”

Question 8 is a two-part question. Part 1 is a yes/no question that asks what materials and services districts have available in the most common native language of English learners in the district. Part 2 asks about materials and services available in other native languages of English learners in the district. Response options in part 2 are: “yes, for all languages,” “yes, for some languages,” and “no.”

Question 9 is a two-part question, Part 1 asks for the approximate percentage of traditionally-aged high school English learners (e.g., ages 14 to 18) who receive their education in one of the following settings: regular high school, magnet high school, charter high school, career/technical high school, alternative school or program for at-risk students, district-administered adult education program; or some other educational setting. Respondents are asked to report on where a student is primarily enrolled. Part 2 asks for this same information for older high school English learners (e.g., ages 19 to 21). Response options are: “none,” 1-25%,” “26-50%,” “51-75%,” “76-100%,” and “don’t know.”

Question 10 is a two-part question. Part 1 is a yes/no question that asks whether the following programs and services are available to high school students in the district: summer school, remediation classes, credit recovery course/program, flexible scheduling, alternative schools or programs for at-risk students, career and technical training, distance education course/program, district-administered GED courses,, tutoring, mentoring, or some other program or service. Part 2 asks approximately how many high school English learners participate in the programs and services listed. Response options are: “none or few,” “some,” “most,” and “don’t know.”

Question 11 is a yes/no question that asks whether the district has any programs or services designed specifically for English learners in high school. Response items include: tutoring, summer school, credit recovery course/program, mentoring program, or distance education course/program, or other. Respondents can also specify “other.”

Question 12 is a yes/no question that asks whether the district has a before- or after- school period of instruction or program designed specifically for English learners in high school to provide additional academic instruction or support.

Question 13 asks what type of relationship, if any, districts have with other organizations to provide high school English learners with academically-oriented programs or services. Response items are: community colleges, other types of colleges or universities, public libraries, community organizations providing GED or adult education programs, organizations providing job training programs, faith-based organizations, community organizations, and employers. Respondents can also specify “other.” Response options include: “no relationship,” “informal relationship,” or “formal relationship.”

Question 14 asks about how often in the last 12 months English learners ages 18 to 21 newly enrolled in high school in the district. Response options include “never,” “rarely,” “sometimes,” “often,” and “don’t know.”

Question 15 is a yes/no question that asks whether districts provide English learners ages 18 to 21 seeking to newly enroll in the district with information about educational programs and services that include: academic programs at the regular high school, district-administered newcomer program, alternative school or program for at-risk students, career and technical training, GED or adult education programs, and free or low-cost English classes. Respondents can also specify “other.” This question asks respondents about programs offered by their school district and those offered by other entities.

Question 16 asks to what extent districts consider various factors when providing information about educational programs or services for English learners ages 18 to 21 who are seeking to newly enroll in the district. Response items include English proficiency level, literacy in their native language, limited or interrupted formal education, length of time needed to accrue sufficient credits to graduate, whether the student will be able to meet high school graduation requirements in content area classes, whether the student will be able to pass state tests for graduation, and age of student. Respondents can also specify “other.” Response options are “not at all,” minor extent,” moderate extent,” and “major extent.”

Question 17 asks for the current total number of high school English learners in the district.

Timeline

Pretest activities are expected to begin as soon as approval is received from OMB. It is anticipated that recruitment, debriefing, write up of the memorandum summarizing the results, and survey revisions will take approximately 4-6 weeks for each pretest round.

Attachment 1: FRSS 107 Pretest Call Recruitment Script

FRSS 107

FRSS 107 High School English Language Learner Programs

Pretest Call Recruitment Script

Hello, my name is __________________.

I am calling from Westat on behalf of the National Center for Education Statistics, within the U.S. Department of Education regarding a survey on high school English language learner programs. We would like your help in identifying the district-level person who is most familiar with high school English language learner programs in your district.

Who at your district is most knowledgeable about your district’s English language learner programs at the high school level?

(RECORD CONTACT INFORMATION)

May I please speak to that person?

CONNECTED TO DISTRICT–LEVEL PERSON MOST FAMILIAR WITH HIGH SCHOOL ENGLISH LANGUAGE LEARNER PROGRAMS

Hello, my name is __________________.

I’m calling from Westat on behalf of the National Center for Education Statistics, within the U.S. Department of Education regarding a survey on high school English language learner programs. We would like your help in reviewing our draft questionnaire to ensure that it is clear and easy to complete. Specifically, we would like you to complete a questionnaire and then obtain your comments about the survey by telephone. This is a short questionnaire which should take about 45 minutes to complete. [The call will take about 30 minutes.]

Your input, while voluntary, will be essential in developing a questionnaire that is relevant. All information you will provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable from for any other purpose unless otherwise compelled by law (Education Sciences Reform Act of 2002, 20 U.S.C. 9573.)

1. May I have your email address to send you the survey materials?

2. We ask that you complete the questionnaire and fax it back to us before you talk to the survey manager. When would be a good time for the survey manager, Laurie Lewis, to call you to discuss the survey and obtain your comments? How about [SUGGEST A TIME]. [Just to be sure, you are in the [Eastern, Central, Mountain, Pacific] time zone?]

3. What is the best telephone number for the survey manager to reach you on?

Thank you. Your insights will be very helpful.

Attachment 2: FRSS 107 Pretest Cover Letter

April 2015

Dear Pretest Participant:

Thank you for agreeing to participate in the preset of the survey on high school English language learner programs. Westat, a research company located in Rockville, Maryland, is conducting this survey for the National Center for Education Statistics (NCES), within the U. S. Department of Education. The survey is part of the Fast Response Survey System (FRSS), a system charged with collecting information on important and emerging issues related to education. The survey is designed to be completed by district personnel most knowledgeable about high school English language learner programs in their district. The purpose of the survey is to provide nationally representative data about high school English language learner programs in U.S. public school districts.

We are currently conducting the pretest of the survey. Your input, while voluntary, will be essential in developing a survey that is relevant, clear, and not overly burdensome to respondents. Your participation is very important because your comments will improve the survey before the actual data collection begins. Your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose unless otherwise compelled by law (Education Sciences Reform Act of 2002, 20 U.S.C. 9573.)

We ask that you (1) complete the questionnaire and fax it to Westat; (2) keep track of the time you spend filling out the questionnaire; (3) write down any comments about the questionnaire; and (4) discuss your comments with me by telephone at the time scheduled. Please keep the following questions in mind as you complete the questionnaire:

1. Are the instructions and definitions clear and helpful to you?

2. Are the survey questions clear and easy to interpret?

3. Would you have access to the information necessary for answering these questions?

4. How long did it take you to answer questions 9 and 10?

Please fax the completed questionnaire to me. My toll-free fax number is 1-800-254-0984. My colleague and I will call you at the scheduled time to get your feedback on the questionnaire and discuss any comments or suggestion you may have about the study. In the meantime, feel free to call me at Westat’s toll-free number, 800-937-8281, ext. 8284, if you have any questions. You may also reach me by email at [email protected]. Thank you for your much needed assistance!

Sincerely,

Laurie Lewis

Westat Survey Manager