4 10 OMB SS PartB RPR Review 1 response cb gs

SUPPORTING STATEMENT B:

REQUEST FOR CLEARANCE OF INFORMATION COLLECTION FORMS FOR

“An Examination of Trends in Algebra II Enrollment and Completion in Texas Public High Schools”

December 2014

Submitted to: Submitted by:

U.S. Department of Education SEDL

Institute of Education Sciences 4700 Mueller Blvd.
555 New Jersey Ave. NW, Rm. 308 Austin, TX 78723

Washington, DC 20208 Phone: (800) 476-6861

4700 Mueller Blvd. Austin, TX 78723

800-476-6861

www.relsouthwest.org

This publication was prepared for the Institute of Education Sciences (IES) under contract ED-IES-12-C-00012 by Regional Educational Laboratory Southwest, administered by SEDL. The content of the publication does not necessarily reflect the views or policies of IES or the U.S. Department of Education, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. government. The publication is in the public domain. Authorization to reproduce in whole or in part for educational purposes is granted.

Contents

Introduction 3

Research Questions 4

Data to be Collected 6

1. Respondent Universe and Sampling Methods 7

2. Description of Procedures for the Collection of Information 7

Data Sources 8

3. Description of Procedures for Maximizing Response Rates 9

4. Description of Tests, Procedures, and Methods 10

5. Individuals Consulted on Statistical Aspects of the Design and Key Staff 11

Attachment B-1. District Staff Survey 13

Introduction

The U.S. Department of Education (ED) requests clearance for data collection under the Office of Management and Budget (OMB) clearance agreement (OMB number [IES to complete]) for activities related to the Regional Educational Laboratory (REL) Program. ED, in consultation with SEDL, intends to study changes in student mathematics course-taking and course failures, as well as changes in district mathematics course offerings and diploma plan placement, that occur after implementation of Texas House Bill 5 (HB 5)—the Foundation High School Program. This is a descriptive study.

Under the Foundation High School Program, students may select from among three different graduation plans—the Foundation Plan, the Foundation plus Endorsement Plan, and the Distinguished Plan. Prior to implementation of the Foundation High School Program, which commenced with the incoming cohort of grade 9 students in the 2014-15 school year, students were required to complete algebra II in order to graduate from a Texas public high school¹. Under the Foundation High School Program, students may elect to complete algebra II², but they are no longer required to do so in order to graduate. Currently, the public universities in Texas require students to have completed algebra I, geometry, and algebra II in order to gain admission. Despite the disparity between some of the new graduation plans and admission requirements to state four-year colleges, the Texas Higher Education Coordinating Board has stated that the universities will not be changing their admissions requirements. As such, removing the algebra II graduation requirement for all high school students will affect some students’ eligibility for admission to state four-year colleges and universities.

In addition, the State Board of Education has opted to change the Texas 10 percent rule, which offers admission to state-funded universities to all Texas high school students who graduate in the top 10 percent of their class, to only include students who graduate in the top ten percent of their class and complete the foundation plus distinguished plan beginning with the 2014–15 incoming cohort of grade 9 students. Thus, opting not to complete algebra II could have important consequences for Texas high school graduates.

OMB approval is being requested for a survey data collection in Texas public school districts to discern what districts in Texas are doing in response to HB 5 with regard to diploma plan placement, advanced mathematics course offerings, and information dissemination about the new graduation requirements.

The Texas Education Agency (TEA) wishes to understand any changes in algebra II enrollment, algebra II completion, and third mathematics course³ failure rates that occur after enactment of HB 5. In particular, this study will inform TEA about the mathematics courses students are completing by the end of their junior year⁴, as well as whether or not fewer students, particularly low-income and minority students, are enrolling in algebra II, completing algebra II, and/or failing their third mathematics course by the end of junior year. TEA will be able to use this information to answer questions about changes in course taking patterns and course failure rates that may arise from parents, education practitioners and administrators, policymakers, and researchers. It will also allow them to respond to inquiries regarding any variation in how districts responded to the HB 5 graduation requirements. For example, the Associate Commissioner for Standards and Programs, who is in charge of curriculum and instruction for TEA is interested in learning which mathematics courses student are completing. In particular the Commissioner is interested knowing the number of new math courses districts are creating and/or adding in order for students to be able to complete the endorsements. Similarly, district administrators are curious to know what other districts are doing with regard changes in the courses offered in the districts in order to allow students to complete the new endorsements, particularly in mathematics.

Research Questions

This study will examine relationships between the implementation of the Foundation High School Program graduation plans and algebra II enrollment and completion rates in Texas. The study will also investigate changes in third mathematics course failure rates and differences in the mathematics courses students complete for students who entered grade 9 before and after the implementation of HB 5. Specifically, it will address the following research questions:

1. What is the trend in student enrollment in algebra II by the end of grade 11 for students who entered grade 9 during the 2004-05 through 2014-15 academic years?

   1. Does the trend in student enrollment in algebra II by the end of grade 11 across these cohorts differ by:

      1. Gender?

      2. Race/ethnicity?

      3. Free or reduced price lunch eligibility?

      4. Student achievement?

   2. Does the trend in student enrollment in algebra II by the end of grade 11 over across these cohorts differ between:

      1. High-minority and low-minority districts⁵?

      2. High-income and low-income districts⁶?

      3. Geographic locations (i.e., urban, rural, suburban)?

      4. Regions of Texas (i.e., north, south, east, west, and central)?

2. What is the trend in student completion of algebra II by the end of grade 11 for students who entered grade 9 during the 2004-05 through 2014- academic years?

   1. Does the trend in student completion of algebra II by the end of grade 11 across these cohorts differ by:

      1. Gender?

      2. Race/ethnicity?

      3. Free or reduced price lunch eligibility?

      4. Student achievement?

   2. Does the trend in student completion of algebra II by the end of grade 11 across these cohorts differ between:

      1. High-minority and low-minority districts?

      2. High-income and low-income districts?

      3. Geographic locations (i.e., urban, rural, suburban)?

      4. Regions of Texas (i.e., north, south, east, west, and central)?

3. What is the trend in third mathematics course failure by the end of grade 11 for students who entered grade 9 during the 2004-05 through 2014-15 academic years?

   1. Does the trend in third mathematics course failure by the end of grade 11 across these cohorts differ by:

      1. Gender?

      2. Race/ethnicity?

      3. Free or reduced price lunch eligibility?

      4. Student achievement?

   2. Does the trend in third mathematics course failure by the end of grade 11 across these cohorts differ between:

      1. High-minority and low-minority districts?

      2. High-income and low-income districts?

      3. Geographic locations (i.e., urban, rural, suburban)?

      4. Regions of Texas (i.e., north, south, east, west, and central)?

4. How are districts enacting the graduation requirement provisions of HB 5?

5. Which advanced mathematics courses are districts offering students who enter grade 9 after the enactment of House Bill 5?

6. Which mathematics courses do students who enter grade 9 after the enactment of House Bill 5 complete by the end of their junior year?

   1. Do the mathematics courses that students complete differ by:

      1. Gender?

      2. Race/ethnicity?

      3. Free or reduced price lunch status?

      4. Student achievement?

      5. The diploma plan into which districts automatically place their students⁷?

7. Are there differences in the mathematics courses that students complete by the end of their junior year before (2006-07 through 2015-16) and after enactment of House Bill 5 (2016-17)?

   1. Do the mathematics courses that students complete differ by:

      1. The diploma plan into which districts automatically place their students?

Data to be Collected

Data collection for this project consists of online surveys to be administered to district administration staff who work in Texas public school districts, as well as collection from two sources of extant data. Specifically, in this OMB clearance package, ED is requesting clearance for the following data collection approach:

• A web-based survey of district administrative staff in Texas public high school districts, including follow-up telephone calls

• Extant data collection consisting of student records data to be obtained from the Texas Education Agency

• A review of information on school district websites for each of Texas’ school districts

ED believes that the data collections for which clearance is being requested represent the bare minimum necessary to describe trends in algebra II enrollment, algebra II completion, and third mathematics course failures; district responses to changes in the Texas high school graduation plans; and student mathematics course-taking patterns.

The remainder of Part B addresses the following: respondent universe and sampling methods; description of procedures for maximizing response rates; description of tests, procedures and methods; and contact information for statistical consultants and key staff.

1. Respondent Universe and Sampling Methods

This study uses data from all grade 9 students who entered a Texas public high school during the 2004-05 through 2014-15 academic years for whom data is available—students who would have had the opportunity to complete algebra II by the end of their junior years in academic years 2006-07 through 2016-17. Additionally, district administrative staff from each of the 1,026 public school districts in Texas that include grades 9 through 12 will be asked to complete a brief online survey. TEA is interested in learning how all districts in the state are responding to the changes in graduation requirements being implemented under House Bill 5. As such, the entire universe of public school districts will be asked to participate in the online survey. We expect to achieve an 85 percent response rate to the online survey.

2. Description of Procedures for the Collection of Information

ED’s contractor, REL Southwest, will manage data collection and ensure quality and timeliness. ED’s contractor proposes to collect both primary and secondary data; however, OMB clearance is only being sought for primary data collection. This study will utilize extant student data collected and archived by TEA, as well as a survey administered to district administrators. Table 1 lists the data source used to address each of the research questions.

Table 1. Data sources used to address each research question

Research Question
	TEA	Survey
1. What is the trend in student enrollment in algebra II by the end of grade 11 for students who entered grade 9 during the 2004-05 through 2014-15 academic years?	X
2. What is the trend in student completion of algebra II by the end of grade 11 for students who entered grade 9 during the 2004-05 through 2014-15 academic years?	X
3. What is the trend in third mathematics course failure by the end of grade 11 for students who entered grade 9 during the across the 2004-05 through 2014-15 academic years?	X
4. How are districts enacting the graduation requirement provisions of HB 5?		X
5. Which advanced mathematics courses are districts offering students who enter grade 9 after the enactment of House Bill 5?		X
6 Which mathematics courses do students who enter grade 9 after the enactment of House Bill 5 complete by the end of their junior year?	X
7. Are there differences in the mathematics courses that students complete by the end of their junior year before (2014-15 through 2015-16) and after enactment of House Bill 5 (2016-17)?	X

Data Sources

Primary data will be collected via an online survey administered to district administrative staff. The online survey will be created using an online survey software package and administered by ED’s contractor. ED’s contractor will collect up-to-date contact information for each of the district superintendents in Texas from TEA’s website. This date is updated frequently, so it is expected to be complete and accurate.⁸ ED’s contractor will contact district superintendents in all 1026 of the public school districts in Texas via email and invite them to complete the online survey. District superintendents will be asked to complete the survey themselves or to forward the email message to the person in the district best equipped to respond to the survey. The email message will contain a unique link for accessing the survey. This will reduce burden on respondents, as they will not be required to provide any background information about themselves or their districts. It will also assist with tracking and identifying nonrespondents. The survey should take approximately 5-10 minutes for respondents to complete. All districts that do not complete the online survey will be contacted via telephone and asked to complete the survey orally. A researcher will record the responses in an online survey form.

To answer research questions 4 and 5, we will use an online survey, as well as information posted on district websites collected as part of a separate study to describe how districts are responding to the provisions of HB 5. Specifically, we will use data collected by a separate fast response study to describe which graduation plan, if any, districts are automatically placing students in upon enrollment in grade 9.⁹ We will use the online survey to fill in the gaps from the website search regarding which graduation plan, if any, districts are automatically placing students and to determine the number and type of endorsements districts are offering. REL Southwest will administer the online survey to district staff who oversee curriculum and instruction.

The survey content is displayed in Attachment A.

3. Description of Procedures for Maximizing Response Rates

ED is committed to obtaining complete data for this evaluation. A large share of this descriptive study relies heavily on administrative data. ED’s contractor anticipates a 95-percent response rate from TEA on all student variables in the administrative data. That is, ED’s contractor assumes that 5 percent or less of the data will be missing on key variables. A key to achieving complete administrative data is tracking the data components from TEA with email and telephone contact to the appropriate parties to resolve issues of missing or delayed data files. ED’s contractor has a strong working relationship with representatives from TEA. All administrative data files will be reviewed for consistency and completeness. If a data file has too many missing values or if an instrument in the implementation study has too few items completed to be counted as a response, ED’s contractor will seek to obtain more complete responses by e-mail or phone.

Based on its prior experience with administering surveys to administrative staff in a variety of schools, districts, and states, ED’s contractor expects the response rate for the survey to be at 85 percent. The survey will be sent via electronic mail to the district superintendent at each of the school districts in Texas.¹⁰ The electronic mail message will inform superintendents about the purpose of the survey and the intended use for the data. The district superintendent will be asked to complete the survey or to forward it to the individual in the district office best equipped to respond to the questions, such as the district director of curriculum and instruction. The email message will also contain a link to the online survey. After one week, a second electronic mail message will be sent to all nonrespondents. After an additional week, ED’s contractor will begin to make telephone calls to the district to collect this information. During the telephone call, district administrative staff will be asked to complete the survey orally or to provide contact information for other district staff who might be better able to respond to the survey. If another staff member is identified, this staff member will be contacted via telephone. A researcher will read the survey aloud to respondents and record the responses in an online survey form. The data collection window for the survey will be approximately 1.5 months.

In addition, a number of steps will be taken to maximize response rates. For example, in order to reduce burden on respondents, ED’s contractor has created an online survey that is simple and brief. Because the survey is online, respondents will be able to complete the survey from any location and will not have to worry about losing materials and/or mailing the completed survey. Respondents will also receive assurances of confidentiality, and they will be encouraged to participate as a way to help TEA understand how districts are responding to the new graduation requirements implemented under HB 5. Respondents will be informed that every measure will be taken to protect the confidentiality of the data collected and the data will be used for the purpose of the study. All survey responses will be kept confidential, and will only be used for the purpose of the study. No one at the school, district, or the state will have access to survey responses that include respondents’ names, school names, or other information that could potentially be used to identify individuals or schools. ED’s contractors believe that district staff will be interested in sharing changes made in response to HB 5 with TEA, as well as any satisfaction or dissatisfaction with the changes to the high school graduation plans implemented under HB 5. Additionally, respondents will be given a contact number and email address to reach ED’s contractor with questions.

4. Description of Tests, Procedures, and Methods

To assess trends, ED’s contractor will plot the algebra II enrollment, algebra II completion, and third mathematics course failure rates by the end of grade 11 for 10 pre-legislation cohorts and one post-legislation cohort using extant data collected by TEA. Descriptive statistics (e.g., frequencies, percentages) will be used to describe enrollment, completion and failure rates over this time period for each cohort as a whole, and for selected subgroups. This is an entirely descriptive study. There will be no causal language or attempts to imply causality in included in any documents describing this study or its findings. Additionally, ED’s contractor will administer online district administrator surveys. The survey items have been reviewed by SEDL colleagues who were formerly employed as school and district administrators, as well as by staff at TEA.

ED’s contractor has also pretested the survey with 5 district administrators by asking the participants to complete the survey and comment on the clarity of the questions, whether the questions assess the intended constructs, whether the tone of the questions is appropriate for the audience, and whether the length of the survey is suitable. On average, it took district administrators approximately 7 minutes to complete the survey. The survey was administered online. District administrators reported not being able to fill in the mathematics course codes yet, as the course schedules for next year have not been completed. ED’s contractor did not make any changes to the survey based on the pretest—all respondents found the questions to be clear and easy to answer and a preliminary analysis of the data showed that the responses were properly aligned with the questions. Descriptive statistics will be used to present the findings from the survey.

5. Individuals Consulted on Statistical Aspects of the Design and Key Staff

The following individuals were consulted on the statistical, data collection, and analytic aspects of this study through REL Southwest’s Technical Working Group (TWG):

Dan Goldhaber, Ph.D.

Director, CALDER (National Center for Analysis of Longitudinal Data in Education Research)

Vice President, American Institutes for Research (AIR)

Director, Center for Education Data & Research (CEDR), University of Washington Bothell

Co-Editor, Education Finance and Policy

3876 Bridge Way N,Suite 201

Seattle, WA 98103

Ph: 206-547-1562

Fax: 206-547-1641

E-mail: [email protected]

Geoffrey Borman, Ph.D.

Professor of Education, University of Wisconsin—Madison

Deputy Director of the University of Wisconsin's Predoctoral Interdisciplinary Research Training Program

Senior Researcher, Consortium for Policy Research in Education.

348 Education Building
1000 Bascom Mall
Madison, WI 53706-1326

Ph: 608-263-3688

Fax: 608-265-3135

E-mail: [email protected]

Johannes M. (Hans) Bos, Ph.D.

Vice President and Program Director, International Development, Evaluation, and Research (IDER) Program

American Institutes for Research

2800 Campus Drive, Suite 200
San Mateo, CA 94403

Ph: 650-843-8100

Fax: 650-843-8200

E-mail: [email protected]

W. Steven Barnett, Ph.D.

Board of Governors Professor and Director of the National Institute for Early Education Research

Rutgers University

73 Easton Avenue
New Brunswick, NJ 08901

Ph: 848-932-4350 x23132

Fax: 732-932-4360

E-mail: [email protected]

Attachment B-1. District Staff Survey

1 Prior to HB 5, students were only allowed to complete the minimum high school graduation requirements, which did not include algebra II, if they received special permission.

2 Algebra II is required for students who wish to complete the STEM endorsement or the Distinguished Plan.

3 Students in Texas must complete algebra I and geometry in order to graduate from a public high school. After students have completed algebra I and geometry, they must select a third advanced mathematics course from an approved list of courses, which includes algebra II.

4 Due to time constraints inherent in the REL contract, we will be able to follow students only through grade 11. However, course-taking patterns suggest that if students have not completed Algebra II by the end of their junior year, it is unlikely they will do so in their senior year.

5 High-minority districts are districts with percentages of minority students (i.e., American Indian or Alaska Native, Black, Hispanic, Native Hawaiian or Other Pacific Islander, Two or more races) that are in the top one-third of all districts in the state. Low-minority districts are districts with percentages of minority students that are in the bottom one-third of all districts in state.

6 High-income districts are districts with percentages of students who are eligible for free/reduced price lunch that are in the top one-third of all districts in the state. Low-income districts are districts with percentages of students who are eligible for free/reduced price lunch that in the bottom one-third of all districts in the state.

7 Several district in Texas have indicated that they will begin placing all incoming grade 9 students in the distinguished diploma plan. Students will have the opportunity to opt into a different diploma plan after their sophomore year.

8 If an email message is undeliverable, we will contact the district to obtain an accurate email address for the district superintendent.

9 Several district in Texas have indicated that they will begin placing all incoming grade 9 students in the distinguished diploma plan. Students will have the opportunity to opt into a different diploma plan after their sophomore year.

10 We will obtain contact information, email addresses and telephone numbers, for district superintendents from TEA.

2