HSLS_OMB_Supp Statement_20080214

High School Longitudinal Study of 2009 (HSLS:09)

Supporting Statement
Request for OMB Review (SF83-I)
OMB# 1850-xxxx

Submitted by:

National Center for Education Statistics

U.S. Department of Education

February 14, 2008

Contents

List of Tables v

A. Justification 5

1. Circumstances Making the Collection of Information Necessary 5

a. Purpose of This Submission 5

b. Legislative Authorization 5

c. Prior and Related Studies 5

2. Purpose and Use of Information Collection 5

3. Use of Improved Information Technology and Burden Reduction 5

4. Efforts to Identify Duplication and Use of Similar Information 5

5. Impact on Small Businesses or Other Small Entities 5

6. Consequences of Collecting the Information Less Frequently 5

7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5 5

8. Consultations Outside NCES 5

9. Explanation of Any Payment or Gift to Respondents 5

10. Assurance of Confidentiality Provided to Respondents 5

11. Justification for Sensitive Questions 5

12. Estimates of Annualized Burden Hours and Costs 5

13. Estimates of Other Total Annual Cost Burden 5

14. Annualized Cost to the Federal Government 5

15. Explanation for Program Changes or Adjustments 5

16. Plans for Tabulation and Publication and Project Time Schedule 5

17. Reason(s) Display of OMB Expiration Date Is Inappropriate 5

18. Exceptions to Certification for Paperwork Reduction Act Submissions 5

B. Collection of Information Employing Statistical Methods 5

1. Target Universe and Sampling Frames 5

2. Statistical Procedures for Collecting Information 5

a. School Frames and Samples 5

b. Student Frames and Samples 5

c. Weighting, Variance Estimation, and Imputation 5

3. Methods for Maximizing Response Rates 5

4. Individuals Consulted on Statistical Design 5

List of Tables

Table Page

1. Incentives by respondent type proposed for field test 5

2. Teacher incentives 5

3. Estimated burden on respondents for field test and full-scale studies 5

4. Estimated burden on parents for field test and full-scale studies 5

5. Estimated burden on teachers for field test and full-scale studies 5

6. Estimated burden on school administrators for field test and full-scale studies 5

7. Estimated burden on school counselors for field test and full-scale studies 5

8. Total costs to NCES 5

9. HSLS:09 Schedule 5

10. Illustrative school sample allocation and expected yields (full-scale study HSLS:09) 5

11. Illustrative school sample allocation and expected yields (field test HSLS:09) 5

12. Illustrative student sample allocation and expected yields for 9th- and 12th-graders (field test HSLS:09) 5

12. Illustrative student sample allocation and expected yields for 9th and 12th graders (field test HSLS:09)—Continued 5

13. Illustrative student sample allocation and expected yields for ninth-graders (full-scale study HSLS:09) 5

13. Illustrative student sample allocation and expected yields for ninth-graders (full-scale study HSLS:09)—Continued 5

14. Consultants on statistical aspects of HSLS:09 5

List of Exhibits

Exhibit Page

1. HSLS:09 data security plan outline 5

2. Preliminary outline for HSLS:09 Base-Year Field Test Report 5

High School Longitudinal Study of 2009 (HSLS:09)

This document has been prepared to support the clearance of study data elements and procedures under the Paperwork Reduction Act of 1995 and 5 CFR 1320 for the study titled High School Longitudinal Study of 2009 (HSLS:09). This study is being conducted by RTI International¹—with the American Institutes for Research (AIR), Windwalker Corporation, Horizon Research Inc., Research Support Services (RSS), and MPR Associates (MPR) as subcontractors—under contract to the U.S. Department of Education (Contract number ED-04-CO-0036/0003).

The purpose of this OMB submission is to request emergency clearance for the sampling and recruitment activities for the HSLS:09 field test and main study. We will separately submit a request for clearance that includes the instrument items and data collection procedures. The separate submission is necessary to afford sufficient time to draw the sample and begin to recruit schools for the field test while continuing to work on the assessment and questionnaire items.

In this supporting statement for Standard Form (SF) 83-I, we report the purposes of the study, review the data elements for which clearance is requested, and describe how the collected information addresses the statutory provisions of Section 153 of the Education Sciences Reform Act of 2002 (P.L. 107-279). Subsequent sections of this document respond to the Office of Management and Budget (OMB) instructions for preparing supporting statements to SF 83-I. Section A addresses OMB’s specific instructions for justification and provides an overview of the study’s design and data elements. Section B describes the collection of information employing statistical methods.

A.Justification

1.Circumstances Making the Collection of Information Necessary

a.Purpose of This Submission

The materials in this document support a request for emergency clearance to conduct the sampling and recruiting activities as part of the field test and main study for HSLS:09. The basic components and key design features of HSLS:09 are summarized below:

Base Year

• baseline survey of high school 9^th graders, in fall term, 2009;

• cognitive test in mathematics;

• parents and mathematics and science teachers to be surveyed in the base year (School administrator and school counselor information will also be collected.);

• administrative records collected on coursetaking behavior in grades 8 and 9;

• sample sizes of 800 schools and over 21,000 students (Schools are the first-stage unit of selection, with 9^th graders randomly selected within schools.);

• oversampling of private schools and Asians/Pacific Islanders; and

First Follow-up

Specifications have not yet been provided for follow-ups to the base year study, though the following have been discussed:

• follow-up in 2012 in the spring term, when most sample members are juniors, but some are dropouts or in other grades;

• student questionnaires, mathematics assessment, and school administrator questionnaires to be administered;

• returning to the same schools, but separately following transfer students; and

• high school transcript component in 2013 (records data for grades 9–12).

Second Follow-up

• post–high school follow-ups by web survey and computer-assisted telephone interview.

HSLS:09 will provide a link to its predecessor longitudinal studies, which address many of the same issues of transition from high school to postsecondary education and the labor force. At the same time, HSLS:09 will bring a new and special emphasis to the study of youth transition by exploring the path that leads students to pursue and persist in courses and careers in the fields of science, technology, engineering, and mathematics (STEM). HSLS:09 will measure math achievement gains in the first 3 years of high school, but also will relate tested achievement to students’ choice, access, and persistence—both in mathematics and science courses in high school, and thereafter in the science, technology, engineering, and mathematics pipelines in postsecondary education and in STEM careers. That is to say, the HSLS:09 assessments will serve not just as an outcome measure, but also as a predictor of readiness to proceed into STEM courses and careers. Questionnaires will focus on factors that motivate students for STEM coursetaking and careers.

Additionally, HSLS:09 will focus on students’ decisionmaking processes. Generally, the study will question students on when, why, and how they make decisions about courses and postsecondary options, including what factors, from parental input to considerations of financial aid for postsecondary education, enter into these decisions.

HSLS:09 supports two of the three goals of the American Competitiveness Initiative (ACI), which aims to strengthen math and science education, foreign language studies, and the high school experience in the United States. Information collected from students, parents, teachers, counselors, and school administrators will help to inform and shape efforts to improve the quality of math and science education in the United States, increase our competitiveness in STEM-related fields abroad, and improve the high school experience.

There are several reasons why the transition into adulthood is of special interest to federal policy and programs. Adolescence is a time of physical as well as psychological changes. Attitudes, aspirations, and expectations are sensitive to the stimuli that adolescents are exposed to, and environments influence the process of choosing among opportunities. Parents, educators, and those involved in policy decisions in the educational arena all share the need to understand the effects that the presence or absence of good educational guidance from the school, in combination with that from the home, can have on the educational, occupational, and social success of youth.

These patterns of transition cover individual as well as institutional characteristics. At the individual level the study will look into educational attainment and personal development. In response to policy and scientific issues, data will also be provided on the demographic and background correlates of educational outcomes. At the institutional level, HSLS:09 will focus on school effectiveness issues, including tracking, promotion, retention, and curriculum content, structure, and sequencing, especially as these affect students’ choice of and assignment to different mathematics and science courses and achievement in these two subject areas.

By collecting extensive information from students, parents, teachers, school counselors, school administrators, and school records, it will be possible to investigate the relationship between home and school factors and academic achievement, interests, and social development at this critical juncture. The school environment will be captured primarily through student, teacher, and administrator reports. The extent to which schools are expected to provide special services to selected groups of students to compensate for limitations and poor performance (including special services to assist those lagging in their understanding of mathematics and science) will be examined. Base year teachers will report on sampled students’ specific classroom environment, as well as supply information about their own background and training. Moreover, the study will focus (in particular through the base-year parent survey) on basic policy issues related to parents’ role in the educational success of their children, including parents’ educational attainment expectations for their children, beliefs about and attitudes toward curricular and postsecondary educational choices, and the correlates of active parental involvement in the school; these are among the many questions HSLS:09 will address about the home education support system and its interaction with the student and the school.

Additionally, since the survey will focus on ninth-graders, it will also permit the identification and study of high school dropouts and underwrite trend comparisons with dropouts identified and surveyed in the High School and Beyond Longitudinal Study (HS&B), the National Education Longitudinal Study of 1988 (NELS:88), and the Education Longitudinal Study of 2002 (ELS:2002).

In sum, through its core and supplemental components, HSLS:09 data will allow researchers, educators, and policymakers to examine motivation, achievement, and persistence in STEM coursetaking and careers. More generally, HSLS:09 data will allow researchers from a variety of disciplines to examine changes in young people’s lives and their connections with communities, schools, teachers, families, parents, and friends along a number of dimensions, including the following:

• academic (especially in math and science), social, and interpersonal growth;

• transitions from high school to postsecondary education, and from school to work;

• students’ choices about, access to, and persistence in math and science courses, majors, and careers.

• the characteristics of high schools and postsecondary institutions and their impact on student outcomes;

• family formation, including marriage and family development, and how prior experiences in and out of school correlate with these decisions; and

• the contexts of education, including how minority and at-risk status is associated with education and labor market outcomes.

b.Legislative Authorization

HSLS:09 is sponsored by the National Center for Education Statistics (NCES), within the Institute of Education Sciences (IES), in close consultation with other offices and organizations within and outside the U.S. Department of Education (ED). HSLS:09 is authorized under Section 153 of the Education Sciences Reform Act of 2002 (P.L. 107-279, Title 1 Part C), which requires NCES to

“collect, report, analyze, and disseminate statistical data related to education in the United States and in other nations, including —

(1) collecting, acquiring, compiling (where appropriate, on a State-by-State basis), and disseminating full and complete statistics … on the condition and progress of education, at the preschool, elementary, secondary, postsecondary, and adult levels in the United States, including data on—

(A) State and local education reform activities; …

(C) student achievement in, at a minimum, the core academic areas of reading, mathematics, and science at all levels of education;

(D) secondary school completions, dropouts, and adult literacy and reading skills;

(E) access to, and opportunity for, postsecondary education, including data on financial aid to postsecondary students; …

(J) the social and economic status of children, including their academic achievement…

(2) conducting and publishing reports on the meaning and significance of the statistics described in paragraph (1);

(3) collecting, analyzing, cross-tabulating, and reporting, to the extent feasible, information by gender, race, ethnicity, socioeconomic status, limited English proficiency, mobility, disability, urbanicity, and other population characteristics, when such disaggregated information will facilitate educational and policy decisionmaking; …

(7) conducting longitudinal and special data collections necessary to report on the condition and progress of education…”

Section 183 of the Education Sciences Reform Act of 2002 further states that

“all collection, maintenance, use, and wide dissemination of data by the Institute, including each office, board, committee, and Center of the Institute, shall conform with the requirements of section 552A of title 5, United States Code [which protects the confidentiality rights of individual respondents with regard to the data collected, reported, and published under this title].”

c.Prior and Related Studies

In 1970 NCES initiated a program of longitudinal high school studies. Its purpose was to gather time-series data on nationally representative samples of high school students that would be pertinent to the formulation and evaluation of educational polices.

Starting in 1972 with the National Longitudinal Study of 1972 (NLS:72), NCES began providing educational policymakers and researchers with longitudinal data that linked educational experiences with later outcomes, such as early labor market experiences and postsecondary education enrollment and attainment. The NLS:72 cohort of high school seniors was surveyed five times (in 1972, 1973, 1974, 1979, and 1986). A wide variety of questionnaire data were collected in the follow-up surveys, including data on students’ family background, schools attended, labor force participation, family formation, and job satisfaction. In addition, postsecondary transcripts were collected.

Almost 10 years later, in 1980, the second in a series of NCES longitudinal surveys was launched, this time starting with two high school cohorts. High School and Beyond (HS&B) included one cohort of high school seniors comparable to the seniors in NLS:72. The second cohort within HS&B extended the age span and analytical range of NCES’ longitudinal studies by surveying a sample of high school sophomores. With the sophomore cohort, information became available to study the relationship between early high school experiences and students’ subsequent educational experiences in high school. For the first time, national data were available showing students’ academic growth over time and how family, community, school, and classroom factors promoted or inhibited student learning. In a leap forward for educational research, researchers, using data from the extensive battery of cognitive tests within HS&B, were also able to assess the growth of cognitive abilities over time. Moreover, data were now available to analyze the school experiences of students who later dropped out of high school. These data became a rich resource for policymakers and researchers over the next decade and provided an empirical base to inform the debates of the educational reform movement that began in the early 1980s. Both cohorts of HS&B participants were resurveyed in 1982, 1984, and 1986. The sophomore cohort was also resurveyed in 1992. Postsecondary transcripts also were collected for both cohorts.

The third longitudinal study of students sponsored by NCES was the National Education Longitudinal Study of 1988 (NELS:88). NELS:88 further extended the age and grade span of NCES longitudinal studies by beginning the data collection with a cohort of eighth graders. Along with the student survey, it included surveys of parents, teachers, and school administrators. It was designed not only to follow a single cohort of students over time (as had NCES’s earlier longitudinal studies, NLS:72 and HS&B), but also, by “freshening” the sample at each of the first two follow-ups, to follow three nationally representative grade cohorts over time (8th-grade, 10th-grade, and 12th-grade cohorts). This provided not only comparability of NELS:88 to existing cohorts, but it also enabled researchers to conduct both cross-sectional and longitudinal analyses of the data. In 1993, high school transcripts were collected, further increasing the analytic potential of the survey system. Students were interviewed again in 1994 and 2000, and in 2000–2001 their postsecondary educational transcripts were collected. In sum, NELS:88 represents an integrated system of data that tracked students from middle school through secondary and postsecondary education, labor market experiences, and marriage and family formation.

The Education Longitudinal Study of 2002 (ELS:2002) was the fourth longitudinal high school cohort study conducted by NCES. ELS:2002 started with a sophomore cohort and was designed to provide trend data about the critical transitions experienced by students as they proceed through high school and into postsecondary education or their careers. Student questionnaires and assessments in reading and mathematics were collected along with surveys of parents, teachers, and school administrators. In addition, a facilities component and school library/media studies component were added for this study series. Freshening occurred at the first follow-up in 2004 to allow for a nationally representative cohort of high school seniors, which was followed by the collection of high school transcripts. An additional follow-up was conducted in 2006.

These studies have investigated the educational, personal, and vocational development of students, and the school, familial, community, personal, and cultural factors that affect this development. Each of these studies has provided rich information about the critical transition from high school to postsecondary education and the workforce. HSLS:09 will continue on the path of its predecessors while also focusing on the factors associated with choosing, persisting in, and succeeding in STEM coursetaking and careers.

2.Purpose and Use of Information Collection

HSLS:09 is intended to be a general-purpose dataset, that is, it will be designed to serve multiple policy objectives. Policy issues to be studied through HSLS:09 include the identification of school attributes associated with achievement (especially in mathematics); the influence that parent and community involvement have on students’ achievement and development; the factors associated with dropping out of the educational system; changes in educational practices over time; and the transition of different groups (for example, racial and ethnic, gender, and socioeconomic status groups) from high school to postsecondary institutions and the labor market, and especially into STEM curricula and careers. HSLS:09 will inquire into students’ values and goals, investigate factors affecting risk and resiliency, gather information about the social capital available to sample members, inquire into the nature of student interests and decision-making, delineate students’ curricular and extracurricular experiences, and catalogue their school programs and coursetaking experiences and results. HSLS:09 will obtain teacher evaluations of the effort and ability of each student as well as information about the classroom and teacher background. HSLS:09 will include measures of school climate, each student’s native language and language use, student and parental educational expectations, attendance at school, course and program selection, planning for college, interactions with teachers and peers, perceptions of safety in school, parental income, resources, and home education support system. The HSLS:09 data elements will support research that speaks to the underlying dynamics and educational processes that influence student achievement, growth, and personal development over time.

The objectives of HSLS:09 also encompass the need to support both longitudinal and cross-cohort analyses and to provide a basis for important descriptive cross-sectional analyses as well. HSLS:09 is first and foremost a longitudinal study; hence survey items will be chosen for their usefulness in predicting or explaining future outcomes as measured in later survey waves. Compared to its earlier counterparts, there are considerable changes to the design of HSLS:09 that will have some impact on the ability to produce trend comparisons. NELS:88 began with an eighth-grade cohort in the spring term; while this cohort is not markedly different from the fall-term ninth-grade cohort of HSLS:09 in terms of student knowledge base, it differs at the school level in that the HSLS:09 time point represents the beginning of high school rather than the point of departure from middle school. HSLS:09 includes a spring-term 11th-grade follow-up (even though none of the predecessor studies do) because only modest gains have been seen on assessments in the final year of high school and the 11th-grade follow-up minimizes unit response problems associated with testing in the spring term of the senior year. The design of HSLS:09 calls for information to be collected from parents of 12th-graders and the use of transcripts to provide continuous data for grades 9–12. These data elements will provide the basis for trend analysis between HSLS:09 and its predecessor studies.

We are exploring the possibility of conducting a pilot test prior to the field test to determine the feasibility of using school computers and to test out the computer-based assessment. The survey questions would not be included in this pilot test. The purpose of this pilot test would be to help us understand the issues associated with using school computers for the student assessment and to test out issues associated with programming the assessment items. Pilot testing these two issues before the field test allows the field test to be dedicated to testing the efficacy of the items, which could be compromised if we experience unexpected difficulties with the computers themselves or with how the assessment screens were programmed.

As part of the pilot test, we plan to ask a series of questions to a convenience sample of about 20 schools to identify the issues associated with using the school’s computer laboratories and computer equipment for the student component of the HSLS:09. At each school, we will ask about the availability of a computer lab or a location at the school with computers that might be available for the sessions. For schools that have computers available, we will ask about the capacity of the computer lab (or other location with a set of computers) with regard to number of computers and internet connectivity, the security of the computers at the school, and whether RTI and NCES will be permitted to use the computer lab (or comparable location with a set of computers) to conduct HSLS:09. As a back-up, we are prepared to bring in 5 laptops per school to conduct the student assessment and survey. The questions we plan to ask the school are:

1. Do you have a computer lab in your school or other location with multiple computers?

2. How many computers are there in the computer lab (or comparable location) that can be connected to the Internet?

3. What type of internet connections do you have in the computer lab (or comparable location)?

   1. High Speed Connection

   2. Dial-up connection

   3. None

4. Which operating system (Windows 2000/XP, Mac O/S, Linux, etc.) runs on these computers?

5. What web browser(s) (name and version) are installed on these computers? (i.e., Internet Explorer 6.0, Mozilla Firefox 2.0, Netscape 6, etc.)

6. Is the internet activity of these computers recorded and/or monitored in any way?

7. How many students and/or classes per day use the computers in the computer lab?

8. Can RTI International use the computers at the school for conducting the web-based student assessment and survey for students participating in the High School Longitudinal Study?

9. Are the school computers protected by:

   1. antivirus software

   2. anti-spyware software

   3. internet firewall?

10. Will you allow RTI International to run checks on the school computers to verify that they are not infected with viruses or spyware?

11. Will you allow RTI International to remove viruses and spyware found as the result of the check proposed in Question 9?

In addition to asking questions of the school, we will ask approximately 3-5 schools to allow us to pilot test the computer-based assessment. We will ask students from these schools to complete preliminary assessment screens to identify issues such as the presentation or display of the items which could impact the responses provided by students.

The content of the assessment battery and the questionnaires will be discussed in a later OMB submission, and data elements for the questionnaires will be explicitly presented at that time.

3.Use of Improved Information Technology and Burden Reduction

For the first time in the series of NCES longitudinal studies, all questionnaire data will be collected in electronic media only. In addition, the student assessment will also be a computer-assisted two-stage adaptive test. For the student component, we will use the school’s computer lab when available, and, as a backup, we will bring multiple laptops into the school for use by the sampled students. A member of the research team will be present to assist students with computer issues as needed.

School administrators, teachers, and parents will be given a username and password and will be asked to complete the questionnaire via the Internet. Follow-up for school administrators, teachers, and parents who do not complete the web questionnaire by self-administration will be in the form of computer-assisted telephone interviewing (CATI). Computer control of interviewing offers accurate and efficient management of survey activities, including case management, scheduling of calls, generation of reports on sample disposition, data quality monitoring, interviewer performance, and flow of information between telephone and field operations.

Additional features of the system include (1) online help for each screen to assist interviewers in question administration; (2) full documentation of all instrument components, including variable ranges, formats, record layouts, labels, question wording, and flow logic; (3) capability for creating and processing hierarchical data structures to eliminate data redundancy and conserve computer resources; (4) a scheduler system to manage the flow and assignment of cases to interviewers by time zone, case status, appointment information, and prior cases disposition; (5) an integrated case-level control system to track the status of each sample member across the various data collection activities; (6) automatic audit file creation and timed backup to ensure that, if an interview is terminated prematurely and later restarted, all data entered during the earlier portion of the interview can be retrieved; and (7) a screen library containing the survey instrument as displayed to the interviewer.

4.Efforts to Identify Duplication and Use of Similar Information

Since the inception of its secondary education longitudinal studies program in 1970, NCES has consulted with other federal offices to ensure that the data collected in this important series of longitudinal studies do not duplicate the information from any other national data sources within the U.S. Department of Education or other government agencies. In addition, NCES staff have regularly consulted with nonfederal associations such as the College Board, American Educational Research Association, the American Association of Community Colleges, and other groups to confirm that the data to be collected through this study series are not available from any other sources. These consultations also provided, and continue to provide through the HSLS:09 Technical Review Panel, methodological insights from the results of other studies of secondary and postsecondary students and labor force members, and they ensure that the data collected through HSLS:09 will meet the needs of the federal government and other interested agencies and organizations.

Other longitudinal studies of secondary and postsecondary students (i.e., NLS:72, HS&B, NELS:88, ELS:2002) have been sponsored by NCES in the past. HSLS:09 builds on and extends these studies rather than duplicating them. These earlier studies were conducted during the 1970s, 1980s, 1990s, and the early 2000s and represent educational, employment, and social experiences and environments different from those experienced by the HSLS:09 student sample. In addition to extending prior studies temporally as a time series, HSLS:09 will extend them conceptually. The historical studies do not fully provide the data that are necessary to understand the role of different factors in the development of student commitment to attend higher education and then to take the steps necessary to succeed in college (take the right courses, take courses in specific sequences, etc.). Using items and inventories, the study will enable researchers to move beyond the traditional covariates to ask, “How do students and parents construct their choice set?” Further, HSLS:09 will focus on the factors associated with choosing and persisting in mathematics and science coursetaking and STEM careers. These focal points present a marked difference between HSLS:09 and its predecessor studies.

The only other dataset that offers so large an opportunity to understand the key transitions into postsecondary institutions and/or the world of work, is the Department of Labor (Bureau of Labor Statistics) longitudinal cohorts, the National Longitudinal Survey of Youth 1979 and 1997 cohorts (NLSY79, NLSY97). Clearly, however, the NLSY youth cohorts represent temporally earlier cohorts than HSLS:09. There are also important design differences between the NLSY79/NLSY97 and HSLS:09 that render them more complementary than duplicative. NLSY is a household-based longitudinal survey; HSLS:09 is school based. For both NLSY cohorts, baseline Armed Service Vocational Aptitude Battery (ASVAB) test data are available, but there is no longitudinal high school achievement measure. While the NLSY97 also gathers information from schools (including principal and teacher reports and high school transcripts), it cannot study school processes in the same way as HSLS:09, given its household sampling basis. Any given school contains only one to a handful of NLSY97 sample members, a number that constitutes neither a representative sample of students in the school, nor a sufficient number to provide within-school estimates. Thus, although both studies provide important information for understanding the transition from high school to the labor market, HSLS:09 is uniquely able to provide information about educational processes and within-school dynamics and how these affect both school achievement and ultimate labor market outcomes, including outcomes in science, technology, engineering, and mathematics education and occupations.

5.Impact on Small Businesses or Other Small Entities

This section has limited applicability to the proposed data collection effort. Target respondents for HSLS:09 are individuals (typically nested within an institutional context) of public and private schools; base-year data collection activities will involve no burden to small businesses or entities.

6.Consequences of Collecting the Information Less Frequently

This submission describes the field test and full-scale data collection for the base year of HSLS:09. Base-year data collection will take place in the fall of 2009, preceded by a field test in 2008. First follow-up data collection will take place 2½ years later, in the spring term of 2012, with a field test in 2011. The initial out-of-school follow-up is tentatively scheduled for 3 years thereafter.

The rationale for conducting HSLS:2009 is based on a historical national need for information on academic and social growth, school and work transitions, and family formation. In particular, recent education and social welfare reform initiatives, changes in federal policy concerning postsecondary student support, and other interventions necessitate frequent studies. Repeated surveys are also necessary because of rapid changes in the secondary and postsecondary educational environments and the world of work. Indeed, longitudinal information provides better measures of the effects of program, policy, and environmental changes than would multiple cross-sectional studies.

To address this need, NCES began the National Longitudinal Studies Program more than 35 years ago with the National Longitudinal Study of 1972 (NLS:72). This study collected a wide variety of data on students’ family background, schools attended, labor force participation, family formation, and job satisfaction at five data collection points through 1986. NLS:72 was followed approximately 10 years later by High School and Beyond (HS&B), a longitudinal study of two high school cohorts (10th- and 12th-grade students). The National Education Longitudinal Study of 1988 (NELS:88) followed an eighth-grade cohort, which, upon completion in 2000, reflected a modal respondent age of about 26 years. The Education Longitudinal Study of 2002 (ELS:2002) followed a 10th-grade cohort and allows for the availability of a 32-year trend line.

The scheduled student follow-ups of HSLS:09 are less frequent than the 2-year interval employed with HS&B, NELS:88, and ELS:2002. The first follow-up takes place at 2½ years after the base year, and the second follow-up 3 years after the first follow-up. However, parent data may be collected at grade 12, and a high school transcripts study to be conducted soon after graduation will provide continuous coursetaking data for the cohort’s high school careers for all on-time or early completers. The initial data collection occurs at the start of the students’ high school careers and will allow researchers to understand decisionmaking processes as they pertain to the selection of STEM-related courses. By following up at the end of the students’ junior year, researchers will be able to measure achievement gain as well as postsecondary planning information. Collecting parent and transcript information in the 12th grade will minimize burden on schools and respondents, while also allowing for further intercohort comparability with the main transition themes of the prior studies. The second follow-up is scheduled to occur in the second year after high school, which is on track with the timing of the predecessor studies, thus facilitating comparisons in the domain of postsecondary access and choice. Despite the changes in grade cohorts and data collection time points for the first two rounds, general trends will still be measurable, since the same key transitions, albeit with slightly different data collection points, will be captured with the HSLS:09 data.

Probably the most cost-efficient and least burdensome method for obtaining continuous data on student careers through the high school years comes through the avenue of collecting school records. In most cases, transcript data are more accurate than self-report data as well. High school transcripts were collected for a subsample of the HS&B sophomore cohort, as well as for the entire NELS:88 cohort retained in the study after eighth grade and the entire ELS:2002 sophomore and senior cohorts. The collection of administrative records will take place at the onset of HSLS:09 to identify coursetaking behaviors in grades 8 and 9, and a full transcript study is tentatively scheduled to take place after high school graduation.

7.Special Circumstances Relating to Guidelines of 5 CFR 1320.5

All data collection guidelines in 5 CFR 1320.5 are being followed. No special circumstances of data collection are anticipated.

8.Consultations Outside NCES

Consultations with persons and organizations both internal and external to the National Center for Education Statistics, the U.S. Department of Education (ED), and the federal government have been pursued. In the planning stage for the HSLS:09, there were many efforts to obtain critical review and to acquire comments regarding project plans and interim and final products. We are in the process of convening the Technical Review Panel, which become the major vehicle through which future consultation is achieved in the course of the project. Consultants outside ED and members of the Technical Review Panel include the following individuals:

Technical Review Panel

Dr. Clifford Adelman

The Institute for Higher Education Policy • Suite 400

1320 19th Street, NW

Washington, DC 20036

Phone: (202) 861-8223 ext 228

Fax: (202) 861-9307

E-mail: [email protected]

Dr. Kathy Borman

Department of Anthropology, SOC 107 University of South Florida

4202 Fowler Avenue Tampa, FL 33620

Phone: (813) 974-9058

E-mail: [email protected]

Dr. Daryl E. Chubin

Director, Center for Advancing Science & Engineering Capacity

American Association for the Advancement of Science (AAAS)

1200 New York Avenue NW

Washington, DC 20005

Dr. Jeremy Finn

State University of New York at Buffalo Graduate School of Education

409 Baldy Hall

Buffalo, NY 14260

Phone: (716) 645-2484

E-mail: [email protected]

Dr. Thomas Hoffer

NORC

1155 E. 60th Street

Chicago, IL 60637

Phone: (773) 256-6097

E-mail: [email protected]

Dr. Vinetta Jones

Howard University

525 Bryant Street NW

Academic Support Building

Washingon DC 20059

Phone: (202) 806-7340 or (301) 395-5335

E-mail: [email protected]

Dr. Donald Rock

Before 10/15: K11 Shirley Lane

Trenton NJ 08648

Phone: 609-896-2659

After 10/15: 9357 Blind Pass Rd, #503

St Pete Beach, FL 33706

Phone : (727) 363-3717

E-mail: [email protected]

Dr. James Rosenbaum.

Institute for Policy Research

Education and Social Policy

Annenberg Hall 110 EV2610

Evanston, Illinois 60204

Phone: (847) 491-3795

E-mail: [email protected]

Dr. Russ Rumberger.

Gevirtz Graduate School of Education

University of California

Santa Barbara, CA 93106.

Phone: (805) 893-3385

E-mail: [email protected]

Dr. Philip Sadler

Harvard-Smithsonian Center for Astrophysics
60 Garden St., MS 71

Cambridge, MA 02138.

Office: D-315, Phone: (617) 496-4709,

Fax: (617) 496-5405.

E-mail: [email protected]

Dr. Sharon Senk

Department of Mathematics

Division of Science and Mathematics Education

D320 Wells Hall

Phone: (517) 353-4691 ( office )

E‑mail: [email protected]

Dr. Timothy Urdan

Santa Clara University

Department of Psychology

500 El Camino Real

Santa Clara, CA 95053

Phone: (408) 554-4495

Fax: (408) 554-5241

E-mail: [email protected].

Other Consultants Outside ED

Dr. Eric Bettinger, Associate Professor, Economics

Case Western Reserve University

Weatherhead School of Management

10900 Euclid Avenue

Cleveland, Ohio 44106

Phone: (216) 386-2184

E-mail: [email protected]

Dr. Audrey Champagne, Professor Emerita

University of Albany

Educational Theory and Practice

Education 119

1400 Washington Avenue

Albany NY 12222

Phone: (518) 442-5982

E-mail: none listed

Dr. Stefanie DeLuca, Assistant Professor

Johns Hopkins University

School of Arts and Sciences

Department of Sociology

532 Mergenthaler Hall

3400 North Charles Street

Baltimore, Maryland 21218

Phone: (410) 516-7629

E-mail: [email protected]

Dr. Laura Hamilton

RAND Corporation

4570 Fifth Avenue, Suite 600

Pittsburgh, PA 15213

Phone: (412) 683-2300 x4403

E‑mail: [email protected]

Dr. Jacqueline King

Director for Policy Analysis

Division of Programs and Analysis

American Council for Education

Center for Policy Analysis

One Dupont Circle NW

Washington DC, 20036

Phone: (202) 939-9551 | Fax: 202-785-2990

E-mail: j[email protected]

Dr. Joanna Kulikowich, Professor of Education

The Pennsylvania State University

232 CEDAR Building

University Park, PA 16802-3108

Phone: (814) 863-2261

E‑mail: [email protected]

Dr. Daniel McCaffrey

RAND Corporation

4570 Fifth Avenue, Suite 600

Pittsburgh, PA 15213

Phone: (412) 683-2300 x4919

E-mail: [email protected]

Dr. Jeylan Mortimer

University of Minnesota - Dept of Sociology

909 Social Sciences Building,

267 19th Avenue South,

Minneapolis, MN 55455

Room 1014a Social Sciences

Phone: (612) 624-4064

E-mail: [email protected]

Dr. Aaron Pallas

Teachers College

Columbia University

New York, NY 10027

Phone: (646) 228-7414

E-mail: [email protected]

Ms. Senta Raizen, Director

WestEd

Nat’l Ctr. For Improving Science Education

1840 Wilson Blvd, Suite 201A

Arlington, Virginia 22201-3000

Phone: (703) 875-0496

Fax: (703) 875-0479

E-mail: [email protected]

9.Explanation of Any Payment or Gift to Respondents

Table 1 shows the incentive structure, by respondent type, requested for HSLS:09. In some cases, incentive experiments have been proposed to determine the effectiveness of the incentive on response rates. A description and rationale for each incentive or incentive experiment is provided below.

Table 1. Incentives by respondent type proposed for field test

Respondent	Experiment?	Incentive
School	Yes	$500 technology allowance vs. $0
Student (in-school administration)	Yes	$20 cash vs. $10
Student (web/CATI)	No	$20 check mailed to respondent
School coordinator	No	$100 cash base honorarium; up to $150 cash for high student response
Math or science teacher	No	$10 to $40 check mailed to respondent; sliding scale based on the number of students to report
School counselor	No	No incentive
School administrator	No	No incentive
Parents	No	No incentive

NOTE: CATI = computer-assisted telephone interviewing.

Incentives for schools. Securing the cooperation of schools to participate in voluntary research has become increasingly difficult. Our experience is that many schools already feel burdened by mandated “high stakes” testing and, at the same time, are hampered by fiscal and staffing constraints. Moreover, we will face roadblocks not only at the school, but also at the district level, where research studies must sometimes comply with stringent requirements to submit formal and detailed applications similar to those one would submit to an IRB before individual schools can even be contacted. A successful incentive program can greatly reduce labor costs associated with school recruitment and refusal conversion efforts.

Upon suggestions from the government to consider offering an incentive to schools and that laptop computers are an appropriate level of incentive, we have designed an experiment for the field test to determine if this level of incentive would encourage schools to participate in HSLS:09. We considered offering laptop computers as an incentive, though this idea was ruled out after considering drawbacks such as compatibility, usefulness of equipment, and security issues involved with transferring laptops containing confidential information to school staff. An incentive experiment was proposed at the school-level for the field test to help offset some of the challenges associated with obtaining school cooperation. For the field test, we planned for an experiment comparing the effect of a $500 technology allowance against no incentive. All schools within a given district would receive the same incentive. The technology allowance would be in the form of a check written to the school that can be used at the school’s discretion, though we recommend it be used toward technology for the school to align with the focus of the study.

Incentives for students. We have planned an incentive experiment to test the effectiveness of a $20 student incentive versus a $10 incentive on student response rates in the schools. The $20 incentive is the same incentive as was offered to seniors in the ELS:2002 First Follow-Up Study. As part of ELS:2002 First Follow-Up Field Test, RTI conducted an experiment that suggested the efficacy of using a $20 student incentive for high school students. The efficacy of this incentive was confirmed in the main study, which achieved a 93.5 percent in-school student response rate.

Our experience from several recent large-scale nationally-representative data collections with an assessment component demonstrates that this level of incentive is necessary to achieve target response rates, as shown in Table 2. The HSLS:09 RFP called for a minimum response rate of 92 percent.  The ELS:2002 Base Year student data collection fell five percentage points short of the 92% student participation rate with a token incentive.  We also fell short of 92 percent with the $15 incentive for the 15-year-old sample in PISA, but we achieved 93% in the ELS First Follow-up with the $20 incentive. It was for that reason that we requested to test the $20 incentive for the HSLS Field Test.  

For the ELS:2002 First Follow-Up Study, seniors were offered this incentive as a motivation to attend the spring data collection session at a time when seniors are typically apathetic toward participation in additional testing activities. This level of incentives is requested for the 9^th grade cohort to offset some of the stress associated with test taking. Students are reporting more and more frequently that they would prefer to remain in their assigned class than participate in a research study to minimize missing important lessons that would prepare them for high-stakes testing. To encourage these students to leave their assigned class to participate in the study, we request to incentivize 9^th grade students at the same level as was successful with 12^th grade students in 2004. All participating students at a school will receive the same level of incentive. Student-level incentives also aid in motivating school officials to participate by giving something back to the students. A student incentive has worked successfully on recent studies. Table 2 shows the incentives that were offered on recent NCES studies and the response rate achieved.

Table 2. Student Incentives on Prior Studies

Study	Grade or Age	Incentive	Student Response Rate, %
PIRLS 2006	4th Grade	Token*	95.6
PISA 2006	Age 15	$15**	90.7
ELS:2002 First Follow-Up	12th Grade	$20	93.6
ELS:2002 Base Year	10th Grade	Token	87.3

* High student response could be attributed, in part, to the classroom sampling model employed on the study and the effectiveness of the token incentive for the younger population.

** About half of the schools took part outside of school hours (either after school or on a Saturday) for a higher student incentive. The response rate provided here is only for the students who participated during school hours.

Incentives for students will be provided only with the permission of the school principal. In cases where the principal is reluctant to have Session Administrators give cash to students, we will offer gift certificates, donations to student groups, or other equivalent contributions approved by the schools.

It is possible that some students will be unable to participate during the in-school session. In these situations, we will work with the school to obtain contacting information for the students and attempt to have the student complete the student questionnaire via a web survey or CATI. We anticipate that the number of students completing the questionnaire via web survey or CATI will be small. We propose to offer these students $20 for participating.

School coordinator honorarium. The role of the school coordinator bears a heavy burden to ensure that data collection is successful in the school. The coordinator is expected to coordinate logistics with the data collection contractor, supply a list of eligible students for sampling to the data collection contractor, supply parent contacting information for sampled students, communicate with teachers about the study, distribute parental consent forms and reminder notices, coordinate the assignment of students to each session, assist the test administrator in ensuring the sampled students attend the testing session, assist the test administrator in arranging for follow-up sessions as needed and distribute materials for the staff components of the study. All of these activities will occur under a tight timeline for HSLS:09 due to the fall data collection and the time at which the student list is ready at the school.

The school coordinator honorarium is based on the percentage of sampled students who participate in the study. The role of the school coordinator is critical for the success of the study. We planned for the same level of honorarium as was offered in PISA 2006 based on the similar timing and burden on the school coordinator. The school coordinator honorarium is planned at a base of $100 with up to an additional $50 for achieving high student response at the school. This is an increase from the honorarium offered to school coordinators in the 2004 round of ELS, which ranged from $50 as the base incentive to $100 for coordinators in schools that had high student response rates. We propose increasing the level of these incentives to $100 and $150 respectively, to compensate for the additional work that is required to ensure that we receive a complete list of ninth graders as quickly as possible at the start of the school year as well as to compensate for the logistical burden of coordinating multiple sessions when required due to computer lab or laptop capacity in the school. This incentive is planned for the field test and main study.

Incentives for teachers. Math and science teachers will provide information on classroom attributes, teaching practices, and teaching experience. Past experience has demonstrated the need for a teacher-level incentive to achieve high response rates and many schools have required that teacher compensation be commensurate with their hourly wage. Thus, we have proposed a $25 teacher incentive for both the field test and main study.

Incentive for counselors. We are not proposing to offer an incentive for the counselors to complete their questionnaires. Counselors would typically provide the information requested in the questionnaire as well as the administrative records as part of their normal duties. Because of the nature of the study, we suspect that many school principals will designate a counselor to perform the school coordinator duties, in which case the counselor will receive the coordinator honorarium described above.

Incentive for school administrators. We have achieved high response rates for the school administrator questionnaire on ELS:2002, the ELS:2002 follow-up conducted in 2004, and in PISA:2006. Based on past experience, we are not offering an incentive for the school administrator questionnaire on HSLS:09.

Incentives for parents. There is no precedent for offering an incentive to complete the parent questionnaire. Thus, we have not included a parent incentive in our budget for the HSLS:09.

Reimbursement of reasonable school expenses. In some cases there may be requests from schools for reimbursement of expenses associated with the testing session (for example, keeping the school open for a special make-up testing session that occurs outside of normal school hours). Such cases will be reviewed by project staff on an individual basis and will be approved if the request is deemed reasonable.

10.Assurance of Confidentiality Provided to Respondents

RTI has developed a data security plan (DSP) for HSLS:09 that was acceptable to Neil Russell and the computer security review board. The HSLS:09 plan will strengthen confidentiality protection and data security procedures developed for ELS:2002 and represents best-practice survey systems and procedures for protecting respondent confidentiality and securing survey data. An outline of this plan is provided in exhibit 1. The HSLS:09 DSP will:

Exhibit 1. HSLS:09 data security plan outline

HSLS:09 Data Security Plan Summary Maintaining the Data Security Plan Information Collection Request Our Promise to Secure Data and Protect Confidentiality Personally Identifying Information That We Collect and/or Manage Institutional Review Board Human Subject Protection Requirements Process for Addressing Survey Participant Concerns Computing System Summary General Description of the RTI Networks General Description of the Data Management, Data Collection, and Data Processing Systems Integrated Monitoring System Receipt Control System Instrument Development and Documentation System Data Collection System Document Archive and Data Library Employee-Level Controls Security Clearance Procedures Nondisclosure Affidavit Collection and Storage Security Awareness Training Staff Termination/Transfer Procedures Subcontractor Procedures

Physical Environment Protections System Access Controls Survey Data Collection/Management Procedures Protecting Electronic Media Encryption Data Transmission Storage/Archival/Destruction Protecting Hard-Copy Media Internal Hard-Copy Communications External Communications to Respondents Handling of Mail Returns, Hard-Copy Student Lists, and Parental Consent Forms Handling and Transfer of Data Collection Materials Tracing Operations Software Security Controls Data File Development: Disclosure Avoidance Plan Data Security Monitoring Survey Protocol Monitoring System/Data Access Monitoring Protocol for Reporting Potential Breaches of Confidentiality Specific Procedures for Field Staff

• establish clear responsibility and accountability for data security and the protection of respondent confidentiality with corporate oversight to ensure adequate investment of resources;

• detail a structured approach for considering and addressing risk at each step in the survey process and establish mechanisms for monitoring performance and adapting to new security concerns;

• include technological and procedural solutions that mitigate risk and emphasize the necessary training to capitalize on these approaches; and

• be supported by the implementation of data security controls recommended by the National Institute of Standards and Technology (NIST) for protecting federal information systems.

Under this plan, HSLS:09 will conform totally to federal privacy legislation, including:

• the Privacy Act of 1974 (5 U.S.C. 552a);

• Section C of Education Sciences Reform Act of 2002 (P.L. 107-279);

• the USA Patriot Act of 2001 (P.L. 107-56);

• the Family Educational Rights and Privacy Act (FERPA) (20 U.S.C. 1232g; 34 CFR Part 99);

• the Protection of Pupil Rights Amendment (PPRA) (20 U.S.C. § 1232h; 34 CFR Part 98);

• the Freedom of Information Act (5 U.S.C. 552);

• the Hawkins-Stafford Elementary and Secondary School Improvement Amendments of 1988 (P.L. 100-297);

• Title IV of the Improving America’s Schools Act of 1994 (P.L. 103-382); and

• the Office of Management and Budget (OMB) Federal Statistical Confidentiality Order of 1997.

HSLS:09 also will conform to NCES Restricted Use Data Procedures Manual and NCES Standards and Policies. The plan for maintaining confidentiality includes obtaining signed confidentiality agreements and notarized nondisclosure affidavits from all personnel who will have access to individual identifiers. Each individual working in HSLS:09 will also complete the e-QIP clearance process. The plan also includes annual personnel training regarding the meaning of confidentiality and the procedures associated with maintaining confidentiality, particularly as it relates to handling requests for information and providing assurance to respondents about the protection of their responses. The training will also cover controlled and protected access to computer files under the control of a single database manager; built-in safeguards concerning status monitoring and receipt control systems; and a secured and operator-manned in-house computing facility.

Invitation letters will be sent to states, districts, and schools describing the voluntary nature of this survey. The material sent will include a brochure to describe the study and to convey the extent to which respondents and their responses will be kept confidential. (Materials are provided in appendix A.)

All recruiting materials and procedures will be reviewed and approved by RTI’s Committee for the Protection of Human Subjects prior to sample selection. This committee serves as RTI’s Institutional Review Board (IRB) as required by 45 CFR 46. It is RTI policy that the all RTI research involving human subjects, regardless of funding source, undergoes IRB review in a manner consistent with the regulations in 45 CFR 46 to ensure that all such RTI studies comply with applicable regulations concerning informed consent, confidentiality, and protection of privacy.

11.Justification for Sensitive Questions

The data elements are still in development and will be discussed in a separate submission to OMB.

12.Estimates of Annualized Burden Hours and Costs

Estimates of response burden for the HSLS:09 base-year field test and full-scale data collection activities are shown in tables 3 through 7. Because the proposed field test will be the first application of the proposed instrumentation, the estimates of response burden is based on initial estimates developed from experience with ELS:2002 and other educational longitudinal studies (e.g., NELS:88, HS&B). Please note that the time students will spend completing the cognitive assessment has not been included in the estimated burden. High school seniors will complete the assessment only and not the questionnaire; therefore, they are not represented in the burden estimate.

Table 3. Estimated burden on respondents for field test and full-scale studies

Respondents	Sample	Expected response rate	Number of respondents	Average burden/ response1	Range of response times	Total burden (hours)
Freshmen
Field test (2008)	1,250	92	1,150	30 minutes	—	575
Full-scale (2009)	20,000	92	18,400	30 minutes	—	9200
Total	21,250		19,550			9,775
Seniors
Field test (2008)	1,250	92	1,150	0 minutes (assessment only)	—	0
Total	1,250		1,150		—	0

¹ Please note that the time students will spend completing the cognitive assessments has not been included in the estimated burden.

Table 4. Estimated burden on parents for field test and full-scale studies

Parents	Sample	Expected response rate	Number of respondents	Average burden/response	Range of response times	Total burden (hours)
Total	21,250		19,550			6,842
Field test (2008)	1,250	92	1,150	30 minutes	30	575
Full-scale (2009)	20,000	92	18,400	30 minutes	30	9,200

Table 5. Estimated burden on teachers for field test and full-scale studies

Teachers (math, science) (linked to students)	Sample	Expected response rate	Number of respondents	Average burden/response	Range of response times	Total burden (hours)
Total	10,295		9,471			6,314
Field test (2008)	645	92	593	40 minutes	4–100 minutes	395
Full-scale (2009)	9,650	92	8,878	40 minutes	4–100 minutes	5,919

Table 5a. Estimated burden on teachers for field test and full-scale studies – 9th grade math & science teachers (possible option)

Teachers (math, science) (linked to students)	Sample	Expected response rate	Number of respondents	Average burden/response	Total burden (hours)
Total	13,253		12,193		6,096
Field test (2008)	853	92	785	30 minutes	392
Full-scale (2009)	12,400	92	11,408	30 minutes	5,704

Table 6. Estimated burden on school administrators for field test and full-scale studies

School administrators	Sample	Expected response rate	Number of respondents	Average burden/response	Range of response times	Total burden (hours)
Total	850		832			420
Field test (2008)	55	98	49	30 minutes	—	24.5
Full-scale (2009)	800	98	784	30 minutes	—	392

Table 7. Estimated burden on school counselors for field test and full-scale studies

Counselors	Sample	Expected response rate	Number of respondents	Average burden/response	Range of response times	Total burden (hours)
Total	850		781			390.5
Field test (2008)	55	92	46	30 minutes	—	23
Full-scale (2009)	800	92	736	30 minutes	—	368

For high school students, we have used $6.55 per hour for the field test and $7.25 per hour for the main study to estimate the cost to participants. For freshmen, the cost is estimated at $7,533 for the field test and $133,400 for the main study. For seniors, who will participate only in the field test and complete only the cognitive assessment battery, the cost is estimated as $5,024.

For parents, assuming a $20 hourly wage, the cost to parent respondents is estimated to be $11,500 for the 2008 field test and $184,000 for the 2009 base year main study.

For teachers in the linked design (math and science teachers providing contextual data for student analysis), teacher burden is highly variable because teachers may have different numbers of classes to provide information for, or (even more important) different numbers of students to rate. In ELS:2002, for example, based on the same linked design, burden in the student ratings portion of the teacher questionnaire ranged from as few as 1 student (4 minutes student-rating burden) to as many as 25 students (100 minutes student-rating burden) in small schools where there was only a single teacher for a particular subject in the relevant (ninth) grade.

There is a possibility that we may survey all math and science teachers in a sampled school who instruct 9^th graders. If that is the case, the estimated burden on teachers actually decreases, because the teachers would not be rating individual students on learning approaches and behavior (see Table 5a). Under this scenario, the incentive would be $25 for teachers to complete a 30-minute questionnaire.

Also, sample sizes for the teacher sample are harder to predict with full accuracy than other sample sizes in HSLS, since the number is not preset for this component and some of the information needed to model probable sample sizes is not available from other national datasets. (Ideally, one would be able to tap comprehensive national statistics for how many science and mathematics teachers, in each school in a simulated stratified probability-proportionate-to-size (PPS) sample, were engaged in teaching ninth-graders.)

Costs to respondents may be estimated as follows. Assuming an hourly wage of $20 for school personnel, field test respondent costs amount to $7,900 and main study respondent costs for this component to $118,373. Under the 9^th grade math and science teacher option, the cost for this component increases slightly though exact cost estimates are not available yet. This option has not been accepted or approved yet, but is mentioned only to give complete and thorough context.

For school administrators (the greater part of the questionnaire is typically completed by clerical staff in the school office with the last section completed by the school principal), again assuming a $20 hourly cost, the cost to respondents is $490 in the field test and $7,840 in the main study.

For the counselor questionnaire, the respondent dollar cost, assuming an average hourly rate of $20 for school employees, is estimated to be $460 in the field test and $7,360 in the main study.

Included in the parent, teacher, school administrator, and counselor notification letters will be the following burden statement:

“According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number of this information collection is [1850-New], and it is completely voluntary. The time required to complete this information collection is estimated to average 30 minutes for the parent, teacher, and school administrator questionnaires, including the time to review instructions and complete and review the information collection. The student questionnaire will be no more than 35 minutes in length, and the math test will take about 40minutes. If you have any comments concerning the accuracy of the time estimate or suggestions for improving the interview, please write to: U.S. Department of Education, Washington, DC 20202-4651. If you have comments or concerns regarding the status of your individual interview, write directly to: Dr. Laura LoGerfo, National Center for Education Statistics, 1990 K Street NW, Washington, DC 20006.”

13.Estimates of Other Total Annual Cost Burden

There are no capital, startup, or operating costs to respondents for participation in the project. No equipment, printing, or postage charges will be incurred.

14.Annualized Cost to the Federal Government

Estimated costs to the federal government for HSLS:09 are shown in 8. The estimated costs to the government for data collection for the field test and full-scale studies are presented separately. Included in the contract estimates are all staff time, reproduction, postage, and telephone costs associated with the management, data collection, analysis, and reporting for which clearance is requested.

Table 8. Total costs to NCES

Costs to NCES	Amount
Total HSLS:09 base-year costs	$14,485,784
Salaries and expenses	$719,900
Contract costs	$15,205,684
Field test (2008)	$2,820,025
Salaries and expenses	$215,648
Contract costs	$3,035,673
Full-scale survey (2009)	$11,665,759
Salaries and expenses	$504,252
Contract costs	$12,170,011

NOTE: All costs quoted are exclusive of incentive fee. Field test costs represent Tasks 2 and 5 of the HSLS:09 contract; base-year main study costs include tasks 1, 3, 4, and 6.

15.Explanation for Program Changes or Adjustments

This is a new collection. This submission requests data collection approval for the field test and base year of HSLS:09. Thus, there is no precedent for the study in terms of a previously approved collection for which approval has expired.

16.Plans for Tabulation and Publication and Project Time Schedule

The HSLS:09 field test will be used to test and improve the instrumentation and associated procedures. Publications and other significant provisions of information relevant to the data collection effort will be a part of the reports resulting from the field test and main study, and both public use (Data Analysis System) and restricted use (electronic codebook microdata) files will be important products resulting from the full-scale survey. The HSLS:09 data will be used by public and private organizations to produce analyses and reports covering a wide range of topics.

Data files will be made available to a variety of organizations and researchers, including offices and programs within the U.S. Department of Education, the Congressional Budget Office, the Department of Health and Human Services, the Department of Labor, the Department of Defense, the National Science Foundation, the American Council on Education, and a number of other education policy and research agencies and organizations. The HSLS:09 contract requires the following reports, publications, or other public information releases:

• detailed methodological reports (one each for the field test and full-scale survey) describing all aspects of the data collection effort;

• complete full-scale study data files and documentation for research data users;

• a Data Analysis System (DAS) for public access to HSLS:09 results;

• an ECB for restricted access to HSLS:09 microdata; and

• a “first look” summary of significant descriptive findings for dissemination to a broad audience (the analysis deliverable will include technical appendices).

Final deliverables are scheduled for completion by mid-2010.

The operational schedule for the HSLS:09 field test and full-scale study is presented in 9.

17.Reason(s) Display of OMB Expiration Date Is Inappropriate

The expiration date for OMB approval of the information collection will be displayed on data collection instruments and materials. No special exception to this requirement is requested.

18.Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification statement identified in the Certification for Paperwork Reduction Act Submissions of OMB Form 83-I.

Table 9. HSLS:09 Schedule

Activity	Start	End
Field test
School sampling	2/2008	2/2008
Sample recruitment	2/2008	11/2008
List receipt, student sampling	8/2008	11/2008
Student/staff data collection	9/2008	12/2008
Parent data collection	10/2008	12/2008
Nonresponse follow-up	10/2008	12/2008
Base year
School sampling	2/2008	2/2008
Sample recruitment	2/2008	11/2009
List receipt, student sampling	8/2009	11/2009
Student/staff data collection	9/2009	11/2009
Parent data collection	10/2009	2/2010
Nonresponse follow-up	10/2009	3/2010

B.Collection of Information Employing Statistical Methods

This submission requests clearance for sampling and school recruitment activities for the High School Longitudinal Study of 2009 (HSLS:09) field test and full-scale study to be completed in 2008 and 2009, respectively. This section provides a description of the target universe for this study, followed by an overview of the sampling and statistical methodologies proposed for the field test and the full-scale study. We will also address suggested methods for maximizing response rates and for tests of procedures and methods, and we will introduce the statisticians and other technical staff responsible for design and administration of the study.

1.Target Universe and Sampling Frames

The target population for the HSLS:09 full-scale study consists of 9th grade students in public and private schools that include 9th and 11th grades; their parents; and corresponding math and science teachers, school administrators, and high school counselors. The needed respondent samples will be selected from all public and private schools with 9th and 12th grades in the 50 states and the District of Columbia.² Excluded from the target universe will be specialty schools such as Bureau of Indian Affairs schools, special education schools for the handicapped, area vocational schools that do not enroll students directly, and schools for the dependents of U.S. personnel overseas.

The primary sampling units (PSU) of schools for this study will be selected from the two databases of the U.S. Department of Education. The Common Core of Data (CCD) will be used for selection of public schools, while private schools will be selected from the Private School Survey (PSS) universe files. To eliminate overlap between the field test and full-scale study samples, the full-scale study sample of schools will be selected prior to the field test sample. However, the early selected full-scale study sample will be “refreshed” by a small supplemental sample of schools that will become eligible in the time between the administration of the field test and of the full-scale study. The secondary sampling units (SSU) of students will be selected from student rosters that will be secured from the sample schools. The PSU and SSU sampling procedures for this study are detailed in the next section.

2.Statistical Procedures for Collecting Information

The following section describes sampling procedures for the field test and full-scale study for which clearance is requested. First discussed is the selection plan for the full-scale study sample of schools, followed by the selection plan for the field test sample, to reflect the sequence that will be observed for PSU selections. Next, selection procedures for the student samples will be presented for the field test and full-scale study that will be conducted in 2008 and 2009, respectively. This section also includes descriptions of the procedures that will be followed after data collection, including survey weight adjustments, to measure and reduce bias due to nonresponse.

a.School Frames and Samples

RTI plans to use NCES’ latest Common Core of Data (CCD:2005–2006) as the public school sampling frame and Private School Survey (PSS:2005–2006) as the private school sampling frame. Given that these two sample sources provide comprehensive listings of schools, and that CCD and PSS data files have been used as school frames for a number of other school-based surveys, it is particularly advantageous to use these files in HSLS:09 for comparability and standardization across NCES surveys.

As mentioned earlier, the survey population for the full-scale study of HSLS:09 consists of all 9th graders in the 50 states and District of Columbia enrolled in

• regular public schools, including state department of education schools, that include 9th and 11th grades; and

• Catholic and other private schools that have 9th and 12th grades.

Excluded for this study will be the following:

• schools with no 9th or 11th grade;

• ungraded schools;

• Bureau of Indian Affairs schools;

• special education schools;

• area vocational schools that do not enroll students directly;

• Department of Defense schools; and

• closed public schools.

The school samples will be selected using a stratified probability-proportionate-to-size (PPS) methodology for which a composite size measure methodology developed by RTI statisticians (Folsom, Potter, and Williams, 1987) will be used. This methodology will support the desired oversampling of students in key analytical domains (e.g., Asians and Pacific Islanders), maintains near equal sampling weights for students within each domain, and results in approximately equal total student sample sizes within sampled schools. Details of school sample selection for the full-scale study and field test are provided next.

Full-Scale Study School Samples

The public and private school samples for the full-scale study will be large enough to secure 800 participating schools, combined. The needed samples will be selected from the CCD (2005–2006) and PSS (2005–2006) within sampling strata defined by

• school type: Public, Catholic, or Other private schools;

• Census region: Northeast, Midwest, South, or West; and

• locality: City, Suburban, Town, or Rural.

As illustrated in 10, the starting sample of selected schools will be proportional to the number of ninth-grade students within each stratum, based on information from the CCD and PSS. Should enrollment information be unavailable for certain schools, RTI will impute the needed enrollment counts to the median value of the enrollment for ninth graders within race/ethnicity categories in each school stratum. We expect to select the full-scale and field test samples of schools in January 2008, with the full-scale sample selected first from the entire sampling frames unconditionally.

Table 10. Illustrative school sample allocation and expected yields (full-scale study HSLS:09)

School Stratum	Total		Northeast		Midwest		South		West
	Sampled	Participating	Sampled	Participating	Sampled	Participating	Sampled	Participating	Sampled	Participating
Total	1,349	800	242	144	338	201	504	298	265	157
Public, total	1,012	600	167	100	241	142	395	234	209	124
Public, city	280	167	42	25	59	35	106	63	73	44
Public, suburban	387	229	74	44	91	54	135	80	87	51
Public, town	118	70	23	14	28	16	41	24	26	16
Public, rural	227	134	28	17	63	37	113	67	23	13
Catholic, total	168	100	46	28	58	35	41	24	23	13
Catholic, city	96	58	21	13	33	20	30	18	12	7
Catholic, suburban	54	31	19	10	19	11	8	5	8	5
Catholic, town	16	10	4	4	6	4	3	1	3	1
Catholic, rural	2	1	2	1	0	0	0	0	0	0
Other private, total	169	100	29	16	39	24	68	40	33	20
Other private, city	74	44	11	6	15	9	28	17	20	12
Other private, suburban	56	32	8	5	16	8	25	15	7	4
Other private, town	17	10	3	1	4	4	8	4	2	1
Other private, rural	22	14	7	4	4	3	7	4	4	3

As mentioned earlier, however, a refresher sample of schools will be added to the full-scale sample to account for new schools or those that become eligible after the sampling frames are constructed. For this purpose, frame comparison will be conducted between the 2005–2006 CCD and the 2006–2007 CCD to determine the frequency of new public high schools. Moreover, districts associated with the refresher subsample of schools will be contacted to identify eligible schools recently opened in their jurisdiction. The districts will be provided with a list of all public schools on the sampling frame in their district to help them identify the appropriate schools. Analogous activities will be carried out for private schools using available information from relevant sources such as Quality Education Data (QED), since the 2006–2007 version of the PSS will not be available in time for refreshing the sample of private schools. However, there is a possibility that NCES will be able to secure an early release copy of the next PSS for RTI to include in this investigation. Should such a copy be made available to RTI, it will be used for sample refreshing and related quality control activities.

Obviously, a sample size larger than 800 schools will be necessary to compensate for the anticipated nonresponse and ineligibility. As per NCES standards, we will target a weighted response rate of at least 70 percent at the school level. In unweighted terms, this means that a sample of size 1,143 schools will be required to secure 800 (or, 1,143  0.7) participating schools. Based on our experience with the Education Longitudinal Study of 2002 (ELS:2002), about 4 percent of sampled schools will emerge as ineligible for this study. Consequently, the projected size for the starting sample will be 1,190 (or, 1,143 × 1.04) schools. Moreover, based on ELS:2002 response rates, we expect that an additional sample of 159 schools will be needed to secure 800 participating schools, for a grand total of 1,349 (or, 1,190 + 159) schools.

We will closely monitor the school recruitment activities and release additional schools as needed to ensure that we reach our goal of 800 participating schools. To this end, in addition to the above sample of 1,349 schools, a reserve pool of 251 schools will be selected should observed yield rates fall below expectations. Operationally, the entire sample of 1,600 (or, 1,349 + 251) schools will be randomly partitioned within each stratum into two release pools and a reserve pool. The two release pools will compose the basic sample of 1,349 schools, and schools in the second pool will be released in waves as needed to achieve the sample size goal. The reserve pool will be released selectively in waves by simple random sampling within stratum for strata with low yield rates, when necessary.

Once the school sample has been selected, RTI will use data from QED to obtain principal and district superintendent names along with related information that will be needed for contacting schools. Contacted schools will be asked to provide student rosters for those expected to participate in the field test and the full-scale study, accordingly. For refusing schools, an abbreviated questionnaire will be used to obtain important school-characteristic data to complement frame information. The resulting information will enable us to conduct a more effective analysis of nonresponse bias.

Field Test School Sample

Using probability-based selection of the full-scale study sample of 1,600 schools from the complete CCD and PSS sampling frames, sample schools will be removed from the frames so that a purposive sample can be selected from among the remaining schools to yield 55 participating schools for the field test study. This sample will be divided into 44 public and 11 private schools and will be selected from schools that have both 9th and 12th grades in the states of New York, California, Florida, Illinois, and Texas.

To the extent possible, the stratification plan to be used for selection of this sample will be similar to the one used for the full-scale study sample. Given the small sample size for the field test, however, a somewhat coarser stratification might become necessary to avoid empty strata. As illustrated in 11, we will select a slightly larger sample of 84 schools to ensure that at least 50 schools will provide student lists for the field test. Moreover, an additional sample of 20 schools will be selected and kept in a reserve pool should yield rates fall below expectations.

Table 11. Illustrative school sample allocation and expected yields (field test HSLS:09)

School Stratum	Total		New York		California		Florida		Illinois		Texas
	Sampled	Participating	Sampled	Participating	Sampled	Participating	Sampled	Participating	Sampled	Participating	Sampled	Participating
Total	84	50	17	10	17	10	16	10	17	10	17	10
Public, total	67	40	14	8	13	8	13	8	14	8	13	8
Public, city	19	11	4	3	3	2	4	2	4	2	4	2
Public, suburban	25	15	5	3	5	3	5	3	5	3	5	3
Public, town	8	5	2	1	2	1	1	1	2	1	1	1
Public, rural	15	9	3	1	3	2	3	2	3	2	3	2
Catholic, total	8	5	1	1	2	1	2	1	1	1	2	1
Catholic, city	4	2	1	1	1	1	1	0	0	0	1	0
Catholic, suburban	2	2	0	0	0	0	1	1	0	0	1	1
Catholic, town	1	0	0	0	1	0	0	0	0	0	0	0
Catholic, rural	1	1	0	0	0	0	0	0	1	1	0	0
Other private, total	9	5	2	1	2	1	1	1	2	1	2	1
Other private, city	4	2	0	0	1	0	1	1	1	0	1	1
Other private, suburban	3	2	0	0	1	1	0	0	1	1	1	0
Other private, town	1	0	1	0	0	0	0	0	0	0	0	0
Other private, rural	1	1	1	1	0	0	0	0	0	0	0	0

b.Student Frames and Samples

All sampled schools will be contacted and asked to upload their student lists to a secure website to serve as sampling frames for student samples. Moreover, a backup option will allow schools to provide their student lists via e‑mail of zipped/password-protected files. If the school cannot provide electronic lists, we will ask for paper lists to be faxed to a fax machine in a locked room at RTI. For data security reasons, we will request that paper lists not be mailed. RTI will ask each sample school to provide the following information for each eligible student:

• student ID number;

• full name;

• sex;

• race (White; Black; Asian; Native Hawaiian or Other Pacific Islander; American Indian or Alaska Native);

• ethnicity (Hispanic indicator, regardless of race); and

• whether or not an Individualized Education Program (IEP) has been filed for the student (yes, no).

Race/ethnicity will be needed to guide oversampling of minority students. Moreover, race/ethnicity along with gender and IEP indicators often serve as effective variables for nonresponse adjustments in the full-scale study.

As requested by NCES, no students will be excluded from the sampling frame because of disabilities or language problems. Specifically, the HSLS:09 field test and full-scale study will include students with severe mental disabilities, those with limited command of the English language for understanding the survey materials, and students with physical or emotional problems. Schools will identify such students, and we will work with the schools to determine if any accommodations can be made for these students to complete the survey and assessment. Students who cannot complete the survey or cognitive tests will be excused from doing so; however, contextual information about such students will be collected from teachers, principals, high school counselors, and parents.

The student lists will be reviewed for quality, and schools whose lists fail the quality checks will be recontacted by the school recruiter to resolve observed discrepancies.³ We will proceed with selecting sample students when we have either confirmed that the list received is correct or received a corrected list. Students will be sampled on a flow basis as student lists are received. We will stratify the lists by race/ethnicity and select a systematic sample of students from the resulting lists. For schools that provide paper lists, RTI will use a two-stage process that we have used effectively to select systematic samples from paper lists. This simple, yet scientific, method eliminates the need for data entry of the entire list of students when such lists are provided on paper. Instead, only information for sampled students will be data-entered.

Field Test Student Sample

A random sample of 27 students from the 9th grade and 27 students from the 12th grade will be selected in each of the 55 sample schools, for a total of 1,485 (or, 55 ´ 27) students in each grade. Based on the ELS:2002 eligibility and response rates of 95 and 92 percent, respectively, this will result in a sample of 1,298 (1,485 ´ 0.95 ´ 0.92) responding students in each grade. This sample has grown from the original design of 50 schools and 25 students per grade to ensure that the sample size is adequate for needs of the field test math assessment. Table 12 shows an allocation of the sample and responding students for each grade, by school and student characteristics, overall and for each of the five participating states based on the original proportion of 50 schools and 25 students per grade. The five additional schools will be apportioned across state and school-type accordingly, with 4 more public schools and one more private school. During the recruitment process, we will ask schools when their student lists will be ready; however, we anticipate requesting lists and drawing student samples on a flow basis for the field test between August and November of 2008.

Table 12. Illustrative student sample allocation and expected yields for 9th- and 12th-graders
(field test HSLS:09)

School Stratum	Total		Hispanic		Asian		Black		Other
	Sample	Respondent	Sample	Respondent	Sample	Respondent	Sample	Respondent	Sample	Respondent
Total	1,250	1,093	231	198	50	50	219	198	750	647
Public, city	275	242	50	44	11	11	49	44	165	143
Public, suburban	375	323	70	59	15	15	70	60	220	189
Public, town	125	110	20	19	5	5	20	18	80	68
Public, rural	225	198	41	36	9	9	40	36	135	117
Catholic, city	50	44	10	8	2	2	8	8	30	26
Catholic, suburban	50	44	10	8	2	2	8	8	30	26
Catholic, rural	25	22	5	4	1	1	4	4	15	13
Other private, city	50	44	10	8	2	2	8	8	30	26
Other private, suburban	50	44	10	8	2	2	8	8	30	26
Other private, rural	25	22	5	4	1	1	4	4	15	13
New York	250	218	47	39	10	10	43	39	150	130
Public, city	75	66	14	12	3	3	13	12	45	39
Public, suburban	75	64	14	11	3	3	14	12	44	38
Public, town	25	22	4	4	1	1	4	3	16	14
Public, rural	25	22	5	4	1	1	4	4	15	13
Catholic, city	25	22	5	4	1	1	4	4	15	13
Other private, rural	25	22	5	4	1	1	4	4	15	13
California	250	219	46	40	10	10	44	40	150	129
Public, city	50	44	9	8	2	2	9	8	30	26
Public, suburban	75	65	14	12	3	3	14	12	44	38
Public, town	25	22	4	4	1	1	4	4	16	13
Public, rural	50	44	9	8	2	2	9	8	30	26
Catholic, city	25	22	5	4	1	1	4	4	15	13
Other private, suburban	25	22	5	4	1	1	4	4	15	13
Florida	250	219	46	40	10	10	44	40	150	129
Public, city	50	44	9	8	2	2	9	8	30	26
Public, suburban	75	65	14	13	3	3	14	12	44	37
Public, town	25	22	4	3	1	1	4	4	16	14
Public, rural	50	44	9	8	2	2	9	8	30	26
Catholic, suburban	25	22	5	4	1	1	4	4	15	13
Other private, city	25	22	5	4	1	1	4	4	15	13
Illinois	250	218	46	39	10	10	44	39	150	130
Public, city	50	44	9	8	2	2	9	8	30	26
Public, suburban	75	64	14	11	3	3	14	12	44	38
Public, town	25	22	4	4	1	1	4	3	16	14
Public, rural	50	44	9	8	2	2	9	8	30	26
Catholic, rural	25	22	5	4	1	1	4	4	15	13
Other private, suburban	25	22	5	4	1	1	4	4	15	13

Table 12. Illustrative student sample allocation and expected yields for 9th and 12th graders
(field test HSLS:09)—Continued

School Stratum	Total		Hispanic		Asian		Black		Other
	Sample	Respondent	Sample	Respondent	Sample	Respondent	Sample	Respondent	Sample	Respondent
Texas	250	219	46	40	10	10	44	40	150	129
Public, city	50	44	9	8	2	2	9	8	30	26
Public, suburban	75	65	14	12	3	3	14	12	44	38
Public, town	25	22	4	4	1	1	4	4	16	13
Public, rural	50	44	9	8	2	2	9	8	30	26
Catholic, suburban	25	22	5	4	1	1	4	4	15	13
Other private, city	25	22	5	4	1	1	4	4	15	13

Field Test Teacher, High School Counselor, and Parent Samples

One math and one science teacher will be selected for each ninth-grade student. Where sample students have more than one math or science teacher in fall 2008, we will randomly sample one of the teachers. On the other hand, a number of sample students may not have any math and/or science teachers—a possible reflection of block scheduling—so such students will have no sample teacher. Moreover, for each sample school there will be one sample high school counselor. Where there is more than one counselor at the school, the lead/head/senior counselor will be selected to be in the sample. Our experience with this procedure in previous NCES studies, such as the HS&B Administrator and Teacher Survey, suggests that the senior counselors are the most familiar with the school’s counseling infrastructure. If this counselor declines to respond, a different counselor, if available, will be substituted. Lastly, for each sample student there will be one sample parent. In two-parent households, we will follow the NELS:88/ELS:2002 procedures to ask the parents to identify the parent most knowledgeable about the student’s school situation and experience.

Full-Scale Study Student Sample

A sample of 25 students from 9th grade will be randomly selected from the selected 800 schools (600 public and 200 Catholic and other private schools) for a base sample of 20,000 (or, 800 × 25) students. Moreover, this base sample will be augmented by selecting 1,800 additional Asian/Pacific Islander students for a total sample of 21,800 students.⁴ This augmentation is required to ensure that this subpopulation meets the minimum sample size needed to achieve the following general precision requirements:

• detect a 15% change in proportions across waves of the study;

• detect a 5% change in means;

• produce relative standard errors of 10% or less for proportion estimates based on data from a single wave of data collection; and

• produce relative standard errors of 2.5% or less for estimated means based on data from a single wave of data collection.

Using student enrollment counts from the CCD/PSS and relying on our experience from the field test, the student sampling rates will be set in advance based on race/ethnicity. Students will be sampled from the student lists RTI will receive from sample schools, using a stratified, systematic sampling procedure. Sample sizes will be monitored by race/ethnicity and the sampling rates will be adjusted, if necessary, to achieve all sample size goals. While we expect to achieve the stated response and eligibility rates, an early identification of low sample yields will be vital in making sure we can adjust appropriately to reach our target yields. 13 shows a possible student sample allocation and yield for the HSLS:09 full-scale study. We anticipate requesting student lists and drawing student samples on a flow basis between August and November of 2009.

Table 13. Illustrative student sample allocation and expected yields for ninth-graders (full-scale study HSLS:09)

School Stratum	Total		Hispanic		Asian		Black		Other
	Sample	Respondents	Sample	Respondents	Sample	Respondents	Sample	Respondents	Sample	Respondents
Total	21,800	19,053	2,645	2,026	2,419	1,899	2,684	2,039	14,052	13,089
Northeast	3,924	3,430	477	364	434	339	483	367	2,530	2,360
Public, city	681	595	83	63	76	60	84	64	438	408
Public, suburban	1,211	1,058	148	113	135	105	149	113	779	727
Public, town	370	324	45	34	40	33	46	35	239	222
Public, rural	463	405	56	43	51	41	57	43	299	278
Catholic, city	353	310	43	33	39	31	44	33	227	213
Catholic, suburban	292	255	35	27	32	24	36	27	189	177
Catholic, town	90	78	11	8	10	7	11	9	58	54
Catholic, rural	27	24	3	3	3	2	3	3	18	16
Other private, city	164	143	20	15	18	14	20	15	106	99
Other private, suburban	126	109	15	11	14	9	15	11	82	78
Other private, town	38	34	5	4	4	3	5	4	24	23
Other private, rural	109	95	13	10	12	10	13	10	71	65
Midwest	5,477	4,787	665	510	608	478	673	512	3,531	3,287
Public, city	954	835	116	89	105	85	118	88	615	573
Public, suburban	1,460	1,276	177	136	162	126	180	136	941	878
Public, town	447	391	54	41	50	39	55	42	288	269
Public, rural	1,008	881	122	94	112	88	124	94	650	605
Catholic, city	545	476	66	51	61	48	67	51	351	326
Catholic, suburban	313	273	38	29	35	27	38	29	202	188
Catholic, town	96	84	12	9	11	8	11	10	62	57
Other private, city	245	214	30	23	27	21	30	23	158	147
Other private, suburban	250	219	31	23	28	22	31	24	160	150
Other private, town	77	67	9	7	8	7	9	7	51	46
Other private, rural	82	71	10	8	9	7	10	8	53	48
South	8,121	7,096	985	754	902	709	1,000	759	5,234	4,874
Public, city	1,716	1,500	208	159	190	150	211	161	1,107	1,030
Public, suburban	2,170	1,896	264	201	241	190	267	203	1,398	1,302
Public, town	664	580	80	62	74	58	82	62	428	398
Public, rural	1,826	1,595	221	170	203	160	225	171	1,177	1,094
Catholic, city	491	429	60	46	55	41	60	46	316	296
Catholic, suburban	126	109	15	11	15	11	16	11	80	76
Catholic, town	38	34	5	4	4	3	5	4	24	23
Other private, city	463	405	56	43	51	41	57	43	299	278
Other private, suburban	397	347	48	37	44	34	49	37	256	239
Other private, town	121	106	15	11	13	11	15	11	78	73
Other private, rural	109	95	13	10	12	10	13	10	71	65

Table 13. Illustrative student sample allocation and expected yields for ninth-graders (full-scale study HSLS:09)—Continued

School Stratum	Total		Hispanic		Asian		Black		Other
	Sample	Respondents	Sample	Respondents	Sample	Respondents	Sample	Respondents	Sample	Respondents
West	4,278	3,740	518	398	475	373	528	401	2,757	2,568
Public, city	1,199	1,048	145	111	133	105	148	112	773	720
Public, suburban	1,398	1,221	170	129	155	123	172	131	901	838
Public, town	428	374	51	41	48	37	53	40	276	256
Public, rural	354	310	43	33	39	31	44	33	228	213
Catholic, city	191	167	23	18	21	17	24	18	123	114
Catholic, suburban	125	109	15	11	15	11	15	11	80	76
Catholic, town	38	34	5	4	3	3	5	4	25	23
Other private, city	327	286	40	30	36	27	40	31	211	198
Other private, suburban	104	92	12	10	11	9	13	10	68	63
Other private, town	32	28	4	3	4	3	4	3	20	19
Other private, rural	82	71	10	8	10	7	10	8	52	48

Full-Scale Study Teacher, High School Counselor, and Parent Samples

Analogous to the field test sample, one math and one science teacher will be selected for each 9th-grade student. Where sample students have more than one math or science teacher in fall 2009, we will randomly sample one of the teachers. In addition, for each sample school there will be one sample high school counselor and one sample parent. In two-parent households, the parent most knowledgeable with the student’s school situation and experience will be asked to participate.

We expect that a number of sample students will have the same math and science teachers; however, in most schools the above design can include virtually all eligible teachers. As such, an alternative approach under consideration involves conducting a census of ninth-grade teachers, instead of using a linked student-teacher design. Our survey protocols will be developed in such a way that either approach could be implemented without any ramifications on other aspects of this study.

c.Weighting, Variance Estimation, and Imputation

After data collection, survey data must go through several steps before analysis and reporting tasks can begin. Once data have been compiled and edited, survey weights will be computed, followed by variance estimation and imputation of missing data. In this section we provide a brief overview of each of these steps for the HSLS:09 full-scale study.

Weighting

Virtually all survey data are weighted before they can be used to produce reliable estimates of population parameters. While reflecting the selection probabilities of sampled units, weighting also attempts to compensate for practical limitations of a sample survey, such as differential nonresponse and undercoverage. Furthermore, by taking advantage of auxiliary information about the target population, weighting can reduce the variability of estimates. The weighting process essentially entails four major steps. The first step consists of the computation of design or base weights. In the second step, base weights will be adjusted for nonresponse, while in the third step nonresponse-adjusted weights will be further adjusted so that aggregate counts can match reported estimates for the target population. Finally, adjusted weights will go through a series of quality control checks to detect extreme outliers and to prevent any computational as well as procedural errors.

The HSLS:09 multilevel and multicomponent design introduces significant complexity to the task of weighting. Cognizant of this complexity, RTI will make every effort to keep the resulting weights as simple and intuitive as possible. A minimum of two sets of weights will be required for the analysis of the HSLS:09 data: school weights and student weights. While we expect to secure the stated rates of response, when response rates fall below the accepted limit (both at unit and item levels) we will carry out detailed nonresponse bias analysis to measure the extent of the incurred bias and to identify effective methods for nonresponse adjustment.

Several methods have been suggested for measuring nonresponse bias. In the simplest form, this bias can be approximated temporally by comparing responses obtained from those who respond earlier in the data collection period against late respondents. The incurred bias due to nonresponse can be measured more systematically, however, as the difference between survey estimates and their respective target parameters—the values that would result if a complete census were conducted and all units responded. For instance, when estimating a population mean () based on respondents only nonresponse bias can be expressed as

.

However, for variables that are available from the sampling frame,  can be estimated by without sampling error, in which case the bias in can then be estimated by

.

Moreover, an estimate of the population mean based on respondents and nonrespondents can be obtained by

.

where is the weighted unit nonresponse rate, based on design weights prior to nonresponse adjustment. Consequently, the bias in can then be estimated by

.

That is, the estimate of the nonresponse bias is the difference between the mean for respondents and the mean for nonrespondents, multiplied by the weighted nonresponse rate, using the design weights prior to nonresponse adjustment. This basic approach will be used to measure bias in key survey estimates by relying on data that will be available for both respondents and nonrespondents.

As an attempt to reduce some of the bias due to nonresponse, when appreciable bias is detected at any level, design weights will be adjusted within cells indexed by variables that are deemed strong predictors of response status. In order to identify such variables, which typically include sampling stratification variables and indicators that can efficiently partition units into homogenous segments, we will rely on classification procedures such as CHAID (Chi-square automatic interaction detection method). CHAID is a hierarchical clustering algorithm that successively partitions units according to a categorical characteristic. The algorithm begins with all sample units as a whole and cycles over each predictor to find the optimal partition of the units. The most significant predictor is identified, resulting in partitioning of units into smaller subsets. Next, the algorithm is applied to each partitioned subset of units to find further partitions using the remaining predictors. The process stops after a specified number of partitioning steps or if none of the partitions at a given step is found to be significant.

For HSLS:09 all weight adjustments—including those for nonresponse and poststratification—will be calculated using RTI’s generalized exponential model (GEM) software.⁵ GEM is a raking procedure that is a generalization of the logic-type model, which has been proven to produce weights with less variability than what is achievable via traditional methods. GEM is superior to standard raking methods in two regards. First, it allows a much larger set of variables and their interactions to be used during the model development for nonresponse and raking adjustments, hence enabling the weighted data to mimic the distribution of the target universe with respect to a more comprehensive set of indices. Second, this desirable property is achieved while preventing the adjusted weights from becoming too extreme. That is, GEM produces study estimates that better represent the target universe without increasing variance of estimates significantly, which would otherwise reduce the power of statistical tests.

Variance Estimation

For variance estimation, we will create sets of 200 balanced repeated replication (BRR) weights for school and student samples. The BRR weights are appropriate for use in NCES’s Data Analysis System (DAS) and do not affect the analysis weights used for point estimation. The BRR weighting process will replicate the full weighting process and will use procedures developed for a number of other studies, including ELS:2002 and the National Study of Postsecondary Faculty (NSOPF). In addition, analysis strata and primary sampling units (PSUs) created from the sampling PSUs will be included on the electronic code book (ECB) for analysts wanting to use Taylor series variance estimation rather than BRR weights.

Imputation of Missing Data

Missing values due to item nonresponse will be imputed after the data are edited. Imputation will be performed for items commonly used to define analysis domains, items that are frequently used in crosstabulations, and items needed for weighting. Items from HSLS:09 that are subject to imputation will be imputed using RTI’s weighted sequential hot deck procedure.⁶ By incorporating the sampling weights, this method of imputation takes into account the unequal probabilities of selection in the original sample while controlling the expected number of times a particular respondent’s answer will be used as a donor.

3.Methods for Maximizing Response Rates

Our procedures for maximizing response rates at the institution and respondent levels are based on our successful experience on predecessor and other similar studies. In this section we discuss methods for maximizing response rates for school recruitment as well as for students, parents, and school staff.

School Recruitment. Achieving high school participation rates on voluntary research studies has proven increasingly difficult in recent years. Our experience is that many schools already feel burdened by mandated “high stakes” testing and, at the same time, are hampered by fiscal and staffing constraints. Moreover, we will face roadblocks not only at the school, but also at the district level, where research studies must sometimes comply with stringent requirements to submit formal and detailed applications similar to those one would submit to an IRB before individual schools can even be contacted. The keystone of our plan to work with school districts and schools is to demonstrate the importance of the study while maintaining flexibility in our negotiations with school districts and schools.

Immediately after drawing the sample, recruitment for the field test will commence. Sample materials to be sent to states, districts, and schools are provided in appendix A. We will send succinct yet compelling advance materials to the school districts and schools to introduce a study. Within a few days of receiving the materials, a trained recruiter will contact the school district or school to discuss their participation in the study. Our recruiters are hired for their knowledge, skill, and articulation with the proven ability to develop relationships with district and school contacts that will foster participation and persist throughout the in-school follow-ups for the longitudinal study.

As much as possible, we will shift the burden from the school to RTI staff. Possible ways of shifting the burden include scheduling contacts or survey administrations to best fit the school calendar, mailing consent forms to parents from RTI, providing compensation for time/help completing forms, offering a session administrator to come to the school to compile sampling information, and having a session administrator coordinate all aspects of survey day (e.g., posting reminders, processing consents, and gathering students). These options have proven helpful on similar studies to gain cooperation in schools that expressed scheduling, burden, or staffing concerns.

One of the key factors to a successful recruitment period is time. A task force convened in 2004 to help NCES brainstorm ways to improve school response rates in their international studies recommended that all recruitment activities begin at least 1 year prior to the start of data collection. Though we will not have a full year to recruit schools for the field test, our request for approval to begin recruitment for both the field test and the main study will afford us the benefit of having sufficient time to recruit for the main study.

It is worth noting that our proposed sample design will not cluster schools at the district level. This will mitigate the undesirable situation of losing clusters of schools from sample districts that opt not to participate in this study.

An incentive experiment was proposed at the school-level for the field test to help offset some of the challenges associated with obtaining school cooperation. A successful incentive program can greatly reduce labor costs associated with school recruitment and refusal conversion efforts. For the field test, we planned for an experiment comparing the effect of a $500 technology allowance against no incentive. All schools within a given district would receive the same incentive. The technology allowance would be in the form of a check written to the school that can be used at the school’s discretion, though we recommend it be used toward technology for the school to align with the focus of the study.

The small number of responding schools that will be involved in the field test, 50 to 55 schools, calls for an uncomplicated design protocol. As such, we propose a simple design whereby sample schools in each of the five states are randomly partitioned into control and experimental groups. All schools within the experimental group will be offered an incentive of a $500 technology allowance for participation in HSLS:09. Schools in the control group, however, will be offered no incentive for their participation in the field test.

Student. Ensuring a high student response at each school begins several weeks prior to the student session. Session administrators will work closely with the school coordinators to coordinate the logistics of the sessions and notify students about the sessions. Because the sampled students are not selected by classroom and are dispersed across multiple classes, there is a heavy burden on the school coordinator to inform students about the session, distribute parental consent materials, and ensure that the students arrive at the prescribed location at the scheduled date and time. Session administrators will assume as much of this burden as is possible and permissible by the school.

In our experience, ensuring that students are made aware of the session is the most critical aspect of making sure they arrive at the session at the scheduled time. Despite receiving the consent form to take home, students don’t necessarily distinguish the form from other materials they take home, and they often forget about the session without frequent reminders. To help remind students about the sessions, we will implement options such as distributing postcard reminders a day or two prior to the session, notifying the teachers of selected students, asking the school coordinator to make an announcement on the PA system, and having the session administrator visit a few days prior to the session and convene a brief meeting of the student sample members to encourage participation. We will be collecting parent contacting information from each school from which the parent survey will be conducted. If phone numbers are provided, the session administrator will contact parents a day or two prior to the session to remind the students when they should arrive.

Each week, project staff will conduct group strategy calls with the session administrators to discuss the status of the schools with test dates scheduled for the coming two weeks. The purpose of these conference calls is to learn about the preparedness of each school for the student session, identify any concerns about anticipated response rate or computer capabilities at the school, provide a forum for brainstorming solutions to anticipated problems, and share success stories and lessons learned from other schools. Project staff will follow up frequently with SAs who report problems or concerns with the preparations for student sessions at particular schools.

Our plans for student incentives were described in Section A9. We have planned an incentive experiment to test the effectiveness of a $20 student incentive versus a $10 incentive on student response rates in the schools. We have demonstrated that the $20 incentive threshold is the most effective means of achieving the target 92% student response rate for high school seniors in the ELS:2002 First Follow-Up Study. We hypothesize that the $20 incentive would have an equal effect on 9^th grade students in HSLS:09. Seniors were offered this incentive as a motivation to attend the spring data collection session at a time when seniors are typically apathetic toward participation in additional testing activities. This level of incentives is requested for the 9^th grade cohort to offset some of the stress associated with test taking. Students are reporting more and more frequently that they would prefer to remain in their assigned class than participate in a research study to minimize missing important lessons that would prepare them for high-stakes testing. To encourage these students to leave their assigned class to participate in the study, we request to incentivize 9^th grade students at the same level as was successful with 12^th grade students in 2004. All participating students at a school will receive the same level of incentive. Student-level incentives also aid in motivating school officials to participate by giving something back to the students.

Parent. We will have several opportunities to interact with parents to encourage their participation in the study. The parental consent form will be sent home with the students several weeks before the student session, and the letter will mention that the parent interview is forthcoming. We will collect parent contacting information from the school after the student sample is identified. We will send a letter to the parent via email and Federal Express to initiate the parent interview, providing a URL and credentials for the web instrument and a telephone number that can be used for a telephone interview. If we have a telephone number, the SA will contact the parent to remind him/her of the student session, and will take the opportunity to build a relationship with the parent and encourage participation from both the student and parent. Parents who do not complete the web instrument will be followed up via CATI. Paper-and-pencil versions of the questionnaire will be available for parents who do not have a telephone or internet access. The parent interview will be translated into Spanish and 3 Asian languages to accommodate limited English proficient parents.

There is no precedent for offering an incentive to complete the parent questionnaire. Thus, we have not included a parent incentive in our budget for the HSLS:09.

School Staff (School Administrators, Counselors, Teachers). School staff will receive a letter to initiate their questionnaire about three weeks prior to the student session. The session administrator will work with the school coordinator to prompt school staff to complete their interview. While at the school, the SA will prompt for any outstanding staff questionnaires. If the questionnaires still have not been completed by one week after the session(s) are complete in the school, we will commence CATI follow-up.

Teachers are the only staff that will have an option to complete a paper-and-pencil (PAPI) version of the questionnaire. We will strongly encourage electronic participation and will use the PAPI option as a last resort. The SA will have PAPI questionnaires in his/her possession during the session to distribute to teachers if needed. Past experience has demonstrated the need for a teacher-level incentive to achieve high response rates and many schools have required that teacher compensation be commensurate with their hourly wage. Thus, we have proposed a $25 teacher incentive for both the field test and main study. If we decide to ask teachers to rate the sampled students on their learning approaches or behavior, we will tie the incentive structure to the number of students on whom teachers must report.

4.Individuals Consulted on Statistical Design

A number of individuals have consulted with NCES and RTI on the sampling design and recruitment plans for the HSLS:09. Members of the Technical Review Panel are listed in section A8 of this document. In addition, Dr. Laura LoGerfo, Research Scientist, and Dr. Jeffrey Owings, Associate Commissioner for the Elementary/ Secondary and Library Studies Division, at NCES have reviewed and approved the statistical aspects of the study. Other statistical reviewers at NCES include Marilyn Seastrom, Chief Statistician; and the following statistical program staff:  John Wirt, Tate Gould, and Michael Ross. Section A15 provides the names of additional consultants on statistical aspects of HSLS:09.

Exhibit 2. Preliminary outline for HSLS:09 Base-Year Field Test Report

Executive Summary Introduction Chapter 1 Field Test Preparation: Sampling and Instrumentation 1.1 Sample Design and Selection 1.1.1 Selection of the Field Test States 1.1.2 School Sampling 1.1.3 Student Sampling 1.1.4 Sampling Teachers, Administrators, and Counselors 1.2 Instrumentation 1.2.1 Mathematics Assessment 1.2.2 Student Questionnaire 1.2.3 Parent Questionnaire 1.2.4 Teacher Questionnaire 1.2.5 Administrator Questionnaire 1.2.6 Counselor Questionnaire Chapter 2 Securing Cooperation 2.1 Securing Endorsements 2.2 Securing State Cooperation 2.3 Securing District, Diocese, and School Cooperation 2.4 School-Level Response Results 2.4.1 Analysis of School Response Rates 2.4.2 Responses to Incentives and Burden 2.5 Obtaining Parental Consent 2.6 Recommendations for Main Study Chapter 3 Data Collection 3.1 Recruitment and Training of Data Collection Staff 3.1.1 Assessors 3.1.2 School Recruiters 3.1.3 Help Desk Staff and Interviewers 3.2 In-School Student Survey Procedures and Results 3.3 Procedures and Results for Surveys of Other School Populations 3.3.1 Teachers 3.3.2 School Administrators 3.3.3 School Counselors 3.4 Parent Survey Procedures and Results 3.5 Recommendations for Main Study Chapter 4 Analysis of Student Survey Results 4.1 Mathematics Assessments 4.1.1 Choice of Item Pool 4.1.2 Timing and Completion Rates 4.1.3 Item Performance 4.1.3.1 Classical Item Analysis 4.1.3.2 Item Response Theory 4.1.4 Reliability and Factor Structure 4.1.5 Differential Item Functioning 4.1.6 Selecting Items 4.1.6.1 Measuring Change Over Time 4.1.7.1 Comparing to Prior and Ongoing Studies

4.2 Student Questionnaire 4.2.1 Editing and Retrieval of Critical Items 4.2.2 Item Nonresponse 4.2.3 Inter-item Consistency 4.2.4 Logical Consistency of Responses to Filter and Dependent Questions 4.2.5 Response Variation by Item Position in Questionnaire 4.3 Recommendations for Main Study Chapter 5 Analysis of Teacher, School Administrator, and School Counselor Survey Results 5.1 Teacher Survey Responses 5.1.1 Item Nonresponse 5.1.2 Inter-Item Consistency 5.1.3 Logical Consistency of Responses to Filter and Dependent Questions 5.2 School Administrator Survey Responses 5.2.1 Item Nonresponse 5.2.2 Inter-Item Consistency 5.2.3 Logical Consistency of Responses to Filter and Dependent Questions 5.3 School counselor Responses 5.3.1 Item Nonresponse 5.3.2 Inter-Item Consistency 5.3.3 Logical Consistency of Responses to Filter and Dependent Questions 5.4 Recommendations for Main Study Chapter 6 Analysis of Parent Survey Results 6.1 Item Nonresponse 6.2 Inter-Item Consistency 6.3 Logical Consistency of Responses to Filter and Dependent Questions 6.4 Comparisons of Parent and Student Responses 6.5 Reliability of Parent Responses 6.6 Recommendations for Main Study Chapter 7 Survey Control System and Data Processing 7.1 System Design, Development, and Testing 7.2 Data Capture 7.3 Data Processing and File Preparation 7.4 Recommendations for Main Study Chapter 8 Conclusions References Appendices (Instruments, Sampling Specifications, Mailout Materials and Forms, TRP Membership) Listing of NCES Working Papers to Date

Table 14. Consultants on statistical aspects of HSLS:09

Name	Affiliation	Telephone
James Chromy	RTI	(919) 541-7019
Steven J. Ingels	RTI	(202) 728-1962
Mansour Fahimi	RTI	(301) 230-4675
Peter H. Siegel	RTI	(919) 541-5902
Daniel J. Pratt	RTI	(919) 541-6615
John Riccobono	RTI	(919) 541-7006
Deborah Herget	RTI	(919) 485-7793
Gary Phillips	AIR	(202) 403-6916
Steve Ferrara	AIR	(202) 403-5431

Appendix A. Recruitment Materials

STATE RECRUITING LETTER

NCES/ED Letterhead

[Date]

[Title First Name Last Name]

[State Department of Education Official’s Title]

[State Department of Education]

[Address]

[City, State Zip]

Dear [Name]:

I am writing to inform your state education agency about a vitally important new national study: the High School Longitudinal Study of 2009 (HSLS:09). The study will follow a cohort of 9^th grade students as they progress through high school and enter post-secondary institutions or the work force. The goals of the study are to assess achievement gains throughout high school and to understand students' choice, access, and persistence in science, technology, engineering, and mathematics (STEM) courses, postsecondary education, and careers.  The study will be conducted by RTI International on behalf of the U.S. Department of Education’s National Center for Education Statistics (NCES).

HSLS:09 will build upon and extend a series of longitudinal high school studies that have been conducted each decade since 1972. The study will measure achievement and also focus on how home, school, and community factors influence the plans and thought processes of 9th graders and how they may be linked to their high school coursetaking, college and career decisions. Information collected by students, parents, teachers, counselors, and school administrators will help to inform and shape efforts to improve the quality of math and science education in America, increase our global competitiveness in STEM-related fields, and improve the high school experience.

The first phase of HSLS:09 will be conducted in the fall of 2008. Fifty-five public and private schools enrolling 9^th- and 12^th-graders in the states of California, Florida, Illinois, New York, and Texas have been selected to participate. The main study will take place in the fall of 2009 with 9^th-graders from 800 schools across the country.

In February, we will begin contacting the school districts and schools that have been selected. Each school’s participation is important in order to provide reliable, statistically significant data from an inclusive and diverse group of American secondary schools and students. A list of the school districts and schools selected in your state is attached to this letter.

In each school, the first phase will include a math assessment of 25 9^th-graders and 25 12^th-graders as well as the administration of a background questionnaire. The student assessment will take approximately 40 minutes to complete and the student questionnaire will require another 35 minutes. Ninth grade math and science teachers, a school administrator, a school counselor, and a parent of each selected student will be asked to complete questionnaires. Each of these will require about 30 minutes per respondent. All data will be collected through a web-based application or telephone interview.

Enclosed you will find an HSLS:09 brochure to further explain the study. Should you have any questions, please call the HSLS:09 information number, 866-253-1063, or send an e-mail to [email protected]. You may also contact Laura LoGerfo at NCES at 202-502-7402 for more information.

We look forward to working with your schools to make HSLS:09 a success. Thank you for your support.

Sincerely,

Mark Schneider

Commissioner

National Center for Education Statistics

Enclosures:

HSLS:09 Brochure

List of Selected Schools in [State]

DISTRICT RECRUITING LETTER

NCES Letterhead

[Date]

[Superintendent’s Name]

Superintendent

[District]

[Address]

[City, State Zip]

Dear [Name]:

I am writing to request your district’s participation in a vitally important new national study: the High School Longitudinal Study of 2009 (HSLS:09). The study will focus on mathematics and science and follow a cohort of 9^th grade students as they progress through high school and enter post-secondary institutions or the work force. The goals of the study are to assess achievement gains throughout high school and to understand students' choice, access, and persistence in science, technology, engineering, and mathematics (STEM) courses, postsecondary education, and careers. The study will be conducted by RTI International on behalf of the U.S. Department of Education’s National Center for Education Statistics (NCES).

HSLS:09 will build upon and extend a series of longitudinal high school studies that have been conducted each decade since 1972. The study will measure achievement and also focus on how home, school, and community factors influence the plans and thought processes of 9th graders and how they may be linked to their high school coursetaking, college and career decisions. Information collected by students, parents, teachers, counselors, and school administrators will help to inform and shape efforts to improve the quality of math and science education in America, increase our global competitiveness in STEM-related fields, and improve the high school experience.

Within your district, [LIST FT SCHOOL NAME(S) – IF MULTIPLE SCHOOLS, ADD “AND” BEFORE LAST SCHOOL] [have/has] been selected to participate in the first phase of HSLS:09 to be conducted in the fall of 2008. [IF DISTRICT IS ALSO IN MAIN STUDY ADD: We have also selected the following [school/schools] for the main study in the fall of 2009: [LIST MS SCHOOLS. IF MULTIPLE SCHOOLS, ADD “AND” BEFORE LAST SCHOOL].] With your permission, RTI will contact these schools to discuss study details and to invite them to join the study.

In each school, participation in the first phase of HSLS:09 will include a math assessment of about 25 9^th-graders and 25 12^th -graders, as well as the administration of a background questionnaire. [IF ALSO IN MAIN STUDY ADD: The student component of the main study will include about 25 9^th-graders.] The student assessment will take approximately 40 minutes to complete and the student questionnaire will require another 35 minutes. Ninth grade math and science teachers, a school administrator, a school counselor, and a parent of each selected student will be asked to complete questionnaires. Each of these will require about 30 minutes per respondent. All data will be collected through a web-based application or telephone interview.

We are asking you to encourage your school(s) to participate in this important phase which will take place in fall 2008. [IF SCHOOL INCENTIVE: Participating school(s) in your district will receive a $500 technology allowance as a token of our appreciation.] Participating students will receive a [$20/$10] incentive for participating. School and student participation is voluntary, but we hope all selected districts and schools will choose to contribute to the study. A representative from RTI will contact you in the next few days to answer any questions you may have about HSLS: 09.

Enclosed you will find an HSLS:09 brochure to further explain the study. Should you have any questions, please call the HSLS:09 information number, 866-253-1063, or send an e-mail to [email protected].

We look forward to working with your schools in this endeavor to advance the quality of education for our country’s secondary students. Thank you for your support.

Sincerely,

Mark Schneider

Commissioner

National Center for Education Statistics

Enclosures:

HSLS: 09 Brochure

SCHOOL RECRUITMENT LETTER

NCES Letterhead

[Date]

[Principal’s Name]

[Title]

[School]

[Address]

[City, State Zip]

Dear [Name]:

I am writing to request your school’s participation in a vitally important new national study: the High School Longitudinal Study of 2009 (HSLS:09). The study will focus on mathematics and science and follow a cohort of 9^th grade students as they progress through high school and enter post-secondary institutions or the work force. The goals of the study are to assess achievement gains throughout high school and to understand students' choice, access, and persistence in science, technology, engineering, and mathematics (STEM) courses, postsecondary education, and careers. The study will be conducted by RTI International on behalf of the U.S. Department of Education’s National Center for Education Statistics (NCES).

HSLS:09 will build upon and extend a series of longitudinal high school studies that have been conducted each decade since 1972. The study will measure achievement and also focus on how home, school, and community factors influence the plans and thought processes of 9th graders and how they may be linked to their high school coursetaking, college and career decisions. Information collected by students, parents, teachers, counselors, and school administrators will help to inform and shape efforts to improve the quality of math and science education in America, increase our global competitiveness in STEM-related fields, and improve the high school experience.

Your school has been selected to participate in the first phase of HSLS:09 to be conducted in the fall of 2008. HSLS will include a math and science assessment of 25 9^th-graders and 25 12^th-graders as well as the administration of student background questionnaires. Student participation will take about ninety minutes. Ninth grade math and science teachers, a school administrator, a school counselor, and a parent of each selected student will be asked to complete questionnaires. Each of these will require about 30 minutes per respondent. All data will be collected through a web-based application or telephone interview.

[IF PUBLIC & CATHOLIC: We have been given permission to contact you by your [IF PUBLIC: district superintendent] [IF CATHOLIC: diocese] and we encourage you to include HSLS on your fall 2008 calendar.] [IF OTHER PRIVATE: We encourage you to include HSLS on your fall 2008 calendar] [IF SCHOOL INCENTIVE: Your school will receive a $500 technology allowance as a token of our appreciation for your participation.] Participating students will receive a [$10/$20] incentive for participating. A representative from RTI will contact you in the next few days to answer any questions you may have about HSLS: 09. We hope that by contacting you now it will be easier for you to fit us into your school’s fall 2008 calendar.

Enclosed you will find an HSLS brochure to offer further explanation of the study. Should you have any questions, please call the HSLS information number, 866-253-1063, or send an e-mail to [email protected].

Your participation in this endeavor is important to advance the quality of education for our country’s secondary students. We look forward to working with your school to make HSLS:09 a success. Thank you for your support.

Sincerely,

Mark Schneider

Commissioner

National Center for Education Statistics

Enclosure:

HSLS: 09 Brochure

HSLS:09 Brochure Text

High School Longitudinal Study of 2009; NCES logo; RTI logo

Conducted for: National Center for Education Statistics of U.S. Department of Education

Conducted by: RTI International

What is HSLS:09?

HSLS:09 is the High School Longitudinal Study of 2009. The study is sponsored by the National Center for Education Statistics and carried out by RTI International.

What is the focus of the study?

The focus of the study is to understand the impact of the high school experience on students’ learning and their educational and career choices, and also to explore the transitions students make from high school to postsecondary education, the labor force, and adult roles. The distinctive features of HSLS:09 are that it is longitudinal – the same students will be followed over time, regardless of the path they take (for example, dropouts, the college-bound, and those who go directly into the military or work force after high school) – and multilevel (in addition to surveying and testing students, information will be gathered from parents, teachers, school administrators and counselors to better understand the many home, school, peer and community influences on students’ development and choices). While all educational and career choices are of interest, additional information will be collected about science, technology, engineering and mathematics (STEM) coursetaking and career preparation.

When is the study being conducted?

The first phase of this study will take place in the fall of 2008. A follow-up with students and school personnel will take place in the spring of 2011. The main phase of data collection will occur in the fall of 2009. The first follow-up for that phase will take place in the spring of 2012.

How will HSLS:09 data be used?

HSLS:09 data will allow researchers, educators, and policy makers to examine motivation, achievement, and persistence in STEM course taking and careers. More generally, HSLS:09 data will allow researchers to examine changes in young people’s lives and the influence of communities, schools, teachers, families, parents, and friends on student transitions, progress, and outcomes.

How will HSLS:09 data be used?

HSLS:09 data will allow researchers, educators, and policymakers to examine motivation, achievement, and persistence in high school coursetaking; and entry into careers (either directly or by way of postsecondary education). More generally, HSLS:09 data will allow researchers to examine changes in young people’s lives and the influence of communities, schools, teachers, families, parents, and friends on student transitions, progress, and outcomes.

Why is participation important?

HSLS:09 will build upon and extend a series of longitudinal high school studies that have been conducted each decade since 1972. The study will measure achievement and various influences on the plans and decision-making of high school students. Information collected from students, parents, teachers, counselors, and school administrators will help to inform and shape efforts to improve the quality of the high school experience, including math and science education in America.

How many schools and students will be involved?

The first phase will involve 55 schools and about 27 9^th graders and 27 12^th graders in each school. The main phase will involve a nationally representative sample of 800 high schools, including both public and private schools.

What is involved for students?

Sampled students will complete a mathematics test and a background questionnaire. The in-school sessions will take about 90 minutes. The assessments and questionnaires will be completed on school computers, if available, or RTI will bring laptop computers to the school for use by the students for the study. Students will participate in another in-school session 2 years later with additional follow-ups planned after high school.

What is involved for school staff?

A school administrator will be asked to complete a questionnaire about the school and its environment. Mathematics and science teachers will be asked to complete questionnaires about their backgrounds and approaches to teaching. A school counselor will be asked to complete a questionnaire about school counseling practices. Each questionnaire will take approximately 30 minutes to complete.

What is involved for parents?

One parent of each sampled student will be asked to complete a questionnaire. Each interview will take about 30 minutes.

Who will be responsible for data collection?

Trained HSLS:09 staff will facilitate administration of assessments and questionnaires and provide all required materials. Schools will be asked to designate a school contact to assist HSLS:09 staff with in-school arrangements.

Do students, staff or parents have to participate?

Participation is voluntary, but participation is important to ensure the completeness and accuracy of the results.

Will the names of participants and their responses be kept confidential?

Student, parent, and staff answers may be used only for statistical purposes and may not be disclosed or used, in identifiable form for any other purpose except as required by law. The data collected will be used in analyses to understand students’ coursetaking behaviors, motivation and achievement, and how students decide what to do after high school.

Who do I contact for further information about HSLS:09?

For additional information, you may send email to [email protected], or contact:

RTI Project Director:

Mr. Dan Pratt

(919) 541-6615

(866) 253-1063

RTI Principal Investigator:

Dr. Steven Ingels

(202) 974-7834

NCES Project Officer:

Dr. Laura LoGerfo

(202) 502-7402

Among the organizations endorsing HSLS:09 are:

American Association of School Administrators

American Counseling Association

American Federation of Teachers

Council of Chief State School Officers

National Association of Secondary School Principals

National Catholic Educational Association, Department of Secondary Schools

National Center for Improving Science Education/WestED

National Council of Teachers of Mathematics

National Education Association

National Parent Teacher Association

National School Boards Association

National Science Teachers Association

Appendix B. IRB Approval for Recruitment Materials

1 RTI International is a trade name of the Research Triangle Institute.

2 While the full-scale HSLS:09 sample will include only 9^th grade students, the field test sample will include both 9^th and 12^th grade students to prognosticate the progression that will be observed when reassessing the sample 9^th grade students in 2012.

3Inevitably, there will be inconsistencies between student counts obtained from the sample schools and CCD/PSS. When the relative magnitude of an observed discrepancy exceeds 25 percent, such cases will call for further examinations. For instance, for public schools this measure will be the absolute value of (List – CCD)/List.

4 Sample augmentation will not be necessary for Hispanic or Black students, since sufficient sample sizes to support analyses by race/ethnicity will be secured for such students as part of the base sample of 20,000 students.

5Folsom, R.E., and A.C. Singh (2000). “The Generalized Exponential Model for Sampling Weight Calibration for Extreme Values, Nonresponse, and Poststratification.” Proceedings of the Section on Survey Research Methods of the American Statistical Association, pp. 598-603.

6Iannacchione, V.G. (1982). “Weighted Sequential Hot Deck Imputation Macros.” In Proceedings of the Seventh Annual SAS User’s Group International Conference (pp.759–763). Cary, NC: SAS Institute, Inc.

Supporting Statement Request for OMB Review (SF83I) 0