Final HSLS Revised no 2 Part A
Final HSLS Revised no 2 Part A.doc
High School Longitudinal Study of 2009 (HSLS:09)
OMB: 1850-0852
High School Longitudinal Study of 2009 (HSLS:09)
Supporting
Statement
Request for OMB Review (SF83-I)
OMB# 1850-xxxx
Submitted by:
National Center for Education Statistics
U.S. Department of Education
July 31, 2008
Contents
A. Justification 1
1. Circumstances Making the Collection of Information Necessary 1
a. Purpose of This Submission 1
b. Legislative Authorization 4
c. Prior and Related Studies 5
2. Purpose and Use of Information Collection 6
a. Content Justifications 8
3. Use of Improved Information Technology and Burden Reduction 92
4. Efforts to Identify Duplication and Use of Similar Information 92
5. Impact on Small Businesses or Other Small Entities 93
6. Consequences of Collecting the Information Less Frequently 93
7. Special Circumstances Relating to Guidelines of 5 CFR 1320.5 94
8. Consultations Outside NCES 95
9. Explanation of Any Payment or Gift to Respondents 97
10. Assurance of Confidentiality Provided to Respondents 99
11. Justification for Sensitive Questions 101
12. Estimates of Annualized Burden Hours and Costs 101
13. Estimates of Other Total Annual Cost Burden 104
14. Annualized Cost to the Federal Government 104
15. Explanation for Program Changes or Adjustments 105
16. Plans for Tabulation and Publication and Project Time Schedule 105
17. Reason(s) Display of OMB Expiration Date Is Inappropriate 106
18. Exceptions to Certification for Paperwork Reduction Act Submissions 106
B. Collection of Information Employing Statistical Methods 107
1. Target Universe and Sampling Frames 107
2. Statistical Procedures for Collecting Information 107
a. School Frames and Samples 107
b. Student Frames and Samples 111
c. Weighting, Variance Estimation, and Imputation 117
3. Methods for Maximizing Response Rates 120
4. Individuals Consulted on Statistical Design 122
Appendix A. Recruitment Materials A-1
Appendix B. IRB Approval for Recruitment Materials B-1
Appendix C. Student Questionnaire C-1
Appendix D. Parent Questionnaire D-1
Appendix E. School Administrator Questionnaire E-1
Appendix F. Mathematics Teacher Questionnaire F-1
Appendix G. Science Teacher Questionnaire G-1
Appendix H. Counselor Questionnaire H-1
Appendix I. TRP Meeting Minutes I-1
List of Tables
Table Page
1. Incentives by respondent type proposed for field test 97
2. Student Incentives on prior studies 98
3. Estimated burden on respondents for field test and full-scale studies 102
4. Estimated burden on parents for field test and full-scale studies 102
5. Estimated burden on math teachers for field test and full-scale studies 102
5a. Estimated burden on science teachers for field test and full-scale studies 102
6. Estimated burden on school administrators for field test and full-scale studies 102
7. Estimated burden on school counselors for field test and full-scale studies 103
8. Total costs to NCES 105
9. HSLS:09 Schedule 106
10. Illustrative school sample allocation and expected yields (full-scale study HSLS:09) 109
11. Illustrative school sample allocation and expected yields (field test HSLS:09) 111
12. Illustrative student sample allocation and expected yields for 9th- and 12th-graders (field test HSLS:09) 113
13. Illustrative student sample allocation and expected yields for ninth-graders (full-scale study HSLS:09) 116
14. Consultants on statistical aspects of HSLS:09 124
List of Exhibits
Exhibit Page
1. HSLS:09 data security plan outline 100
2. Preliminary outline for HSLS:09 Base-Year Field Test Report 123
High School Longitudinal Study of 2009
This document has been prepared to support the clearance of study data elements and procedures under the Paperwork Reduction Act of 1995 and 5 CFR 1320 for the study titled High School Longitudinal Study of 2009 (HSLS:09). This study is being conducted for NCES by RTI International1—with the American Institutes for Research (AIR), Windwalker Corporation, Horizon Research Inc., Research Support Services (RSS), and MPR Associates (MPR) as subcontractors—under contract to the U.S. Department of Education (Contract number ED-04-CO-0036/0003).
The purpose of this Office of Management and Budget (OMB) submission is to supersede the earlier emergency clearance document with a full supporting statement that requests clearance for the instrumentation and data collection methods, as well as sampling and recruitment activities, for the HSLS:09 field test and main study. This submission expands on the earlier document by including, in particular, the data elements or items for each questionnaire, and their justifications. We request that the standard 60-day notice for changes after the field test be waived.
In this supporting statement for Standard Form (SF) 83-I, we report the purposes of the study, review the data elements for which clearance is requested, and describe how the collected information addresses the statutory provisions of Section 153 of the Education Sciences Reform Act of 2002 (P.L. 107-279). Subsequent sections of this document respond to the OMB instructions for preparing supporting statements to SF 83-I. Section A addresses OMB’s specific instructions for justification and provides an overview of the study’s design and data elements. Section B describes the collection of information employing statistical methods.
A.Justification
1.Circumstances Making the Collection of Information Necessary
a.Purpose of This Submission
The materials in this document support a request for clearance for the field test and main study of HSLS:09. The basic components and key design features of HSLS:09 are summarized below:
Base Year
baseline survey of high school ninth-graders, in fall term, 2009;
cognitive test in mathematics;
parents and mathematics and science teachers to be surveyed in the base year; school administrator and school counselor information will also be collected;
administrative records collected on coursetaking behavior in grades 8 and 9;
sample sizes of 800 schools and more than 21,000 students (schools are the first-stage unit of selection, with ninth-graders randomly selected within schools); and
oversampling of private schools and Asians/Pacific Islanders.
First Follow-up
Specifications have not yet been provided for follow-ups to the base-year study, although the following have been discussed:
follow-up in 2012 in the spring term, when most sample members are high school juniors, but some are dropouts or in other grades;
student questionnaires, mathematics assessment, and school administrator questionnaires to be administered;
returning to the same schools, but separately following transfer students; and
high school transcript component in 2013 (records data for grades 9–12).
Second Follow-up
post–high school follow-ups by web survey and computer-assisted telephone interview.
HSLS:09 will provide a link to its predecessor longitudinal studies, which address many of the same issues of transition from high school to postsecondary education and the labor force. At the same time, HSLS:09 will bring a new and special emphasis to the study of youth transition by exploring the path that leads students to pursue and persist in courses and careers in the fields of science, technology, engineering, and mathematics (STEM). HSLS:09 will measure math achievement gains in the first 3 years of high school, but also will relate tested achievement to students’ choice, access, and persistence—both in mathematics and science courses in high school, and thereafter in the science, technology, engineering, and mathematics pipelines in postsecondary education and in STEM careers. That is to say, the HSLS:09 assessments will serve not just as an outcome measure, but also as a predictor of readiness to proceed into STEM courses and careers. Questionnaires will focus on factors that motivate students for STEM coursetaking and careers.
Additionally, HSLS:09 will focus on students’ decisionmaking processes. Generally, the study will question students on when, why, and how they make decisions about courses and postsecondary options, including what factors, from parental input to considerations of financial aid for postsecondary education, enter into these decisions.
HSLS:09 supports two of the three goals of the American Competitiveness Initiative (ACI), which aims to strengthen math and science education, foreign language studies, and the high school experience in the United States. Information collected from students, parents, teachers, counselors, and school administrators will help to inform and shape efforts to improve the quality of math and science education in the United States, increase our competitiveness in STEM-related fields abroad, and improve the high school experience.
There are several reasons the transition into adulthood is of special interest to federal policy and programs. Adolescence is a time of physical as well as psychological changes. Attitudes, aspirations, and expectations are sensitive to the stimuli that adolescents are exposed to, and environments influence the process of choosing among opportunities. Parents, educators, and those involved in policy decisions in the educational arena all share the need to understand the effects that the presence or absence of good educational guidance from the school, in combination with that from the home, can have on the educational, occupational, and social success of youth.
These patterns of transition cover individual as well as institutional characteristics. At the individual level the study will look into educational attainment and personal development. In response to policy and scientific issues, data will also be provided on the demographic and background correlates of educational outcomes. At the institutional level, HSLS:09 will focus on school effectiveness issues, including tracking, promotion, retention, and curriculum content, structure, and sequencing, especially as these affect students’ choice of and assignment to different mathematics and science courses and achievement in these two subject areas.
By collecting extensive information from students, parents, teachers, school counselors, school administrators, and school records, it will be possible to investigate the relationship between home and school factors and academic achievement, interests, and social development at this critical juncture. The school environment will be captured primarily through student, teacher, and administrator reports. The extent to which schools are expected to provide special services to selected groups of students to compensate for limitations and poor performance (including special services to assist those lagging in their understanding of mathematics and science) will be examined. Base-year teachers will report on sampled students’ specific classroom environment and supply information about their own background and training. Moreover, the study will focus (in particular through the base-year parent survey) on basic policy issues related to parents’ role in the educational success of their children, including parents’ educational attainment expectations for their children, beliefs about and attitudes toward curricular and postsecondary educational choices, and the correlates of active parental involvement in the school; these are among the many questions HSLS:09 will address about the home education support system and its interaction with the student and the school.
Additionally, since the survey will focus on ninth-graders, it will also permit the identification and study of high school dropouts and underwrite trend comparisons with dropouts identified and surveyed in the High School and Beyond Longitudinal Study (HS&B), the National Education Longitudinal Study of 1988 (NELS:88), and the Education Longitudinal Study of 2002 (ELS:2002).
In sum, through its core and supplemental components, HSLS:09 data will allow researchers, educators, and policymakers to examine motivation, achievement, and persistence in STEM coursetaking and careers. More generally, HSLS:09 data will allow researchers from a variety of disciplines to examine changes in young people’s lives and their connections with communities, schools, teachers, families, parents, and friends along a number of dimensions, including the following:
academic (especially in math and science), social, and interpersonal growth;
transitions from high school to postsecondary education, and from school to work;
students’ choices about, access to, and persistence in math and science courses, majors, and careers;
the characteristics of high schools and postsecondary institutions and their impact on student outcomes;
family formation, including marriage and family development, and how prior experiences in and out of school correlate with these decisions; and
the contexts of education, including how minority and at-risk status is associated with education and labor market outcomes.
b.Legislative Authorization
HSLS:09 is sponsored by the National Center for Education Statistics (NCES), within the Institute of Education Sciences (IES), in close consultation with other offices and organizations within and outside the U.S. Department of Education (ED). HSLS:09 is authorized under Section 153 of the Education Sciences Reform Act of 2002 (P.L. 107-279, Title 1 Part C), which requires NCES to
“collect, report, analyze, and disseminate statistical data related to education in the United States and in other nations, including —
(1) collecting, acquiring, compiling (where appropriate, on a State-by-State basis), and disseminating full and complete statistics … on the condition and progress of education, at the preschool, elementary, secondary, postsecondary, and adult levels in the United States, including data on—
(A) State and local education reform activities; …
(C) student achievement in, at a minimum, the core academic areas of reading, mathematics, and science at all levels of education;
(D) secondary school completions, dropouts, and adult literacy and reading skills;
(E) access to, and opportunity for, postsecondary education, including data on financial aid to postsecondary students; …
(J) the social and economic status of children, including their academic achievement…
(2) conducting and publishing reports on the meaning and significance of the statistics described in paragraph (1);
(3) collecting, analyzing, cross-tabulating, and reporting, to the extent feasible, information by gender, race, ethnicity, socioeconomic status, limited English proficiency, mobility, disability, urbanicity, and other population characteristics, when such disaggregated information will facilitate educational and policy decisionmaking; …
(7) conducting longitudinal and special data collections necessary to report on the condition and progress of education…”
Section 183 of the Education Sciences Reform Act of 2002 further states that
all collection, maintenance, use, and wide dissemination of data by the Institute, including each office, board, committee, and Center of the Institute, shall conform with the requirements of section 552A of title 5, United States Code [which protects the confidentiality rights of individual respondents with regard to the data collected, reported, and published under this title].
c.Prior and Related Studies
In 1970 NCES initiated a program of longitudinal high school studies. Its purpose was to gather time-series data on nationally representative samples of high school students that would be pertinent to the formulation and evaluation of educational polices.
Starting in 1972 with the National Longitudinal Study of the High School Class of 1972 (NLS:72), NCES began providing educational policymakers and researchers with longitudinal data that linked educational experiences with later outcomes, such as early labor market experiences and postsecondary education enrollment and attainment. The NLS:72 cohort of high school seniors was surveyed five times (in 1972, 1973, 1974, 1979, and 1986). A wide variety of questionnaire data were collected in the follow-up surveys, including data on students’ family background, schools attended, labor force participation, family formation, and job satisfaction. In addition, postsecondary transcripts were collected.
Almost 10 years later, in 1980, the second in a series of NCES longitudinal surveys was launched, this time starting with two high school cohorts. High School and Beyond included one cohort of high school seniors comparable to the seniors in NLS:72. The second cohort within HS&B extended the age span and analytical range of NCES’s longitudinal studies by surveying a sample of high school sophomores. With the sophomore cohort, information became available to study the relationship between early high school experiences and students’ subsequent educational experiences in high school. For the first time, national data were available showing students’ academic growth over time and how family, community, school, and classroom factors promoted or inhibited student learning. In a leap forward for educational research, researchers, using data from the extensive battery of cognitive tests within HS&B, were also able to assess the growth of cognitive abilities over time. Moreover, data were now available to analyze the school experiences of students who later dropped out of high school. These data became a rich resource for policymakers and researchers over the next decade and provided an empirical base to inform the debates of the educational reform movement that began in the early 1980s. Both cohorts of HS&B participants were resurveyed in 1982, 1984, and 1986. The sophomore cohort was also resurveyed in 1992. Postsecondary transcripts also were collected for both cohorts.
The third longitudinal study of students sponsored by NCES was the National Education Longitudinal Study of 1988. NELS:88 further extended the age and grade span of NCES longitudinal studies by beginning the data collection with a cohort of eighth-graders. Along with the student survey, it included surveys of parents, teachers, and school administrators. It was designed not only to follow a single cohort of students over time (as had NCES’s earlier longitudinal studies, NLS:72 and HS&B), but also, by “freshening” the sample at each of the first two follow-ups, to follow three nationally representative grade cohorts over time (8th-grade, 10th-grade, and 12th-grade cohorts). This provided not only comparability of NELS:88 to existing cohorts, but it also enabled researchers to conduct both cross-sectional and longitudinal analyses of the data. In 1993, high school transcripts were collected, further increasing the analytic potential of the survey system. Students were interviewed again in 1994 and 2000, and in 2000–2001 their postsecondary educational transcripts were collected. In sum, NELS:88 represents an integrated system of data that tracked students from middle school through secondary and postsecondary education, labor market experiences, and marriage and family formation.
The Education Longitudinal Study of 2002 was the fourth longitudinal high school cohort study conducted by NCES. ELS:2002 started with a sophomore cohort and was designed to provide trend data about the critical transitions experienced by students as they proceed through high school and into postsecondary education or their careers. Student questionnaires and assessments in reading and mathematics were collected along with surveys of parents, teachers, and school administrators. In addition, a facilities component and school library/media studies component were added for this study series. Freshening occurred at the first follow-up in 2004 to allow for a nationally representative cohort of high school seniors, which was followed by the collection of high school transcripts. An additional follow-up was conducted in 2006.
These studies have investigated the educational, personal, and vocational development of students, and the school, familial, community, personal, and cultural factors that affect this development. Each of these studies has provided rich information about the critical transition from high school to postsecondary education and the workforce. HSLS:09 will continue on the path of its predecessors while also focusing on the factors associated with choosing, persisting in, and succeeding in STEM coursetaking and careers.
2.Purpose and Use of Information Collection
HSLS:09 is intended to be a general-purpose dataset; that is, it will be designed to serve multiple policy objectives. Policy issues to be studied through HSLS:09 include the identification of school attributes associated with achievement (especially in mathematics); the influence that parent and community involvement have on students’ achievement and development; the factors associated with dropping out of the educational system; changes in educational practices over time; and the transition of different groups (for example, racial and ethnic, gender, and socioeconomic status groups) from high school to postsecondary institutions and the labor market, and especially into STEM curricula and careers. HSLS:09 will inquire into students’ values and goals, investigate factors affecting risk and resiliency, gather information about the social capital available to sample members, inquire into the nature of student interests and decision-making, delineate students’ curricular and extracurricular experiences, and catalogue their school programs and coursetaking experiences and results. HSLS:09 will obtain teacher evaluations of the effort and ability of each student as well as information about the classroom and teacher background. HSLS:09 will include measures of school climate, each student’s native language and language use, student and parental educational expectations, attendance at school, course and program selection, planning for college, interactions with teachers and peers, perceptions of safety in school, parental income, resources, and home education support system. The HSLS:09 data elements will support research that speaks to the underlying dynamics and educational processes that influence student achievement, growth, and personal development over time.
The objectives of HSLS:09 also encompass the need to support both longitudinal and cross-cohort analyses and to provide a basis for important descriptive cross-sectional analyses. HSLS:09 is first and foremost a longitudinal study; hence survey items will be chosen for their usefulness in predicting or explaining future outcomes as measured in later survey waves. Compared to its earlier counterparts, there are considerable changes to the design of HSLS:09 that will have some impact on the ability to produce trend comparisons. NELS:88 began with an eighth-grade cohort in the spring term; while this cohort is not markedly different from the fall-term ninth-grade cohort of HSLS:09 in terms of student knowledge base, it differs at the school level in that the HSLS:09 time point represents the beginning of high school rather than the point of departure from middle school. HSLS:09 includes a spring-term 11th-grade follow-up (even though none of the predecessor studies do) because only modest gains have been seen on assessments in the final year of high school and the 11th-grade follow-up minimizes unit response problems associated with testing in the spring term of the senior year. The design of HSLS:09 calls for information to be collected from parents of 12th-graders and the use of transcripts to provide continuous data for grades 9–12. These data elements will provide the basis for trend analysis between HSLS:09 and its predecessor studies.
NCES will conduct a pilot test prior to the field test to determine the feasibility of using school computers and to test out the computer-based assessment. The survey questions would not be included in this pilot test. The purpose of this pilot test would be to help understand the issues associated with using school computers for the student assessment and to test out issues associated with programming the assessment items. Pilot testing these two issues before the field test allows the field test to be dedicated to testing the efficacy of the items, which could be compromised if unexpected difficulties with the computers themselves or with how the assessment screens were programmed occur.
As part of the pilot test, a series of questions will be asked to a convenience sample of about 20 schools to identify the issues associated with using the school’s computer laboratories and computer equipment for the student component of HSLS:09. At each school, questions will be asked about the availability of a computer lab or a location at the school with computers that might be available for the sessions. For schools that have computers available, questions will be asked about the capacity of the computer lab (or other location with a set of computers) with regard to number of computers and Internet connectivity, the security of the computers at the school, and whether NCES will be permitted to use the computer lab (or comparable location with a set of computers) to conduct HSLS:09. As a backup, five laptops per school may be brought in to conduct the student assessment and survey. The questions to be addressed to the school are
Do you have a computer lab in your school or other location with multiple computers?
How many computers are there in the computer lab (or comparable location) that can be connected to the Internet?
What type of Internet connections do you have in the computer lab (or comparable location)?
High-speed connection
Dial-up connection
None
Which operating system (Windows 2000/XP, Mac O/S, Linux, etc.) runs on these computers?
What web browser(s) (name and version) are installed on these computers (i.e., Internet Explorer 6.0, Mozilla Firefox 2.0, Netscape 6, etc.)?
Is the Internet activity of these computers recorded and/or monitored in any way?
How many students and/or classes per day use the computers in the computer lab?
Can NCES use the computers at the school for conducting the web-based student assessment and survey for students participating in the High School Longitudinal Study?
Are the school computers protected by
antivirus software;
antispyware software; or
Internet firewall?
Will you allow NCES to run checks on the school computers to verify that they are not infected with viruses or spyware?
Will you allow NCES to remove viruses and spyware found as the result of the check proposed in Question 10?
In addition to asking questions of the school, NCES will ask approximately 3–5 schools to allow pilot tests of the computer-based assessment. NCES will ask students from these schools to complete preliminary assessment screens to identify issues such as the presentation or display of the items which could impact the responses provided by students.
The content of the assessment battery and the questionnaires will be discussed in a later OMB submission, and data elements for the questionnaires will be explicitly presented at that time.
a.Content Justifications
Overview. This section contains justifications for the HSLS:09 field test instruments. Clusters of related items appear with their justifications in the document that follows. Several appendices have also been produced, in order to supply further documentation relevant to the field test instrument development task. Specifically, the summary of the November 2007 and January 2008 HSLS:09 Technical Review Panel (TRP) meetings has been appended, to provide information about the panel’s deliberations, including recommendations concerning study and instrument design. In addition, the six draft field test questionnaires—student, parent, administrator, mathematics teacher, science teacher, and counselor—have been included, each in its own appendix. The draft questionnaires show the basic skip logic as the respondent navigates through the questionnaire; however, note that in all instances this is an electronic instrument, with no paper version, so the respondent will be automatically routed through the instrument without seeing actual skip instructions. In web versions of the questionnaire, when a respondent violates the logic of the questionnaire, the respondent will receive an error message (for example, if the respondent marks multiple options in a “mark one” format the error message will explicitly inform the respondent of the problem and ask for resolution). In the CATI interview, the error or inconsistency prompt comes from the interviewer. Error messages have not been included in this clearance package, but will be developed when the final items are programmed.
All questionnaires and the assessment serve to support the overall purposes of HSLS:09, which are to understand the factors (e.g., experiences, behaviors, attitudes, interactions with people) that influence students’ decision-making process about high school courses, postsecondary options, and occupational goals (especially within the STEM pipeline), and to understand how these decisions evolve through secondary school and beyond.
Student Survey: Purposes and Content Justification
Four primary research questions drive the student questionnaire:
How do students decide what courses to take in high school and what to pursue after their time in high school concludes (e.g., college, work, careers, the military)? What factors affect their decision-making, particularly factors that are malleable to school or parent influence?
Opportunities
Barriers
Attitudes
Past behaviors
Plans
What factors lead students toward or away from STEM courses and careers?
How do students’ attitudes and learning approaches evolve in the course of high school?
Confidence, efficacy
Motivation, engagement, belonging
Reasoning and problem-solving
How do students prioritize various commitments or influences in high school? How do they balance social and academic engagement?
The draft field student questionnaire attempts to provide information that will help to address these and related questions from the student’s perspective.
Justifications for Student Questionnaire Content. Items selected for the student questionnaire have been developed and reviewed by the NCES project officer, project staff at NCES’s HSLS:09 contractor, RTI International, and the HSLS:09 Technical Review Panel. The questionnaire will collect information on seven domains: (1) student locating and contact information; (2) student background and previous experiences; (3) students’ social context and interpersonal influences; (4) students’ values and their determinants; (5) students’ expectations for the future and their determinants; (6) student decisions; and (7) the environment of students’ ninth-grade math and science classes. Each domain and the questions that comprise it are detailed below.
Part I: Student Locating and Contact Information
Questions 1–24. In support of the longitudinal design of the study, the first section of the questionnaire gathers information so that students and their parents/guardians can be located for future follow-up interviews. To facilitate the locating process, the precedent of previous NCES longitudinal studies will be followed by collecting names, addresses, and telephone numbers for the student, their parents, and one relative or acquaintance likely to know the student’s future whereabouts. Given the vast changes in technology and communication over the past decade, e-mail addresses and cell phone numbers will be collected, an approach first used successfully in ELS:2002.
Name
Address
Home telephone number
Cell phone number
E-mail address
Nickname
Mother’s name
Mother’s address
Mother’s home telephone number
Mother’s cell phone number
Mother’s work number
Mother’s e-mail address
Father’s name
Father’s address
Father’s home telephone number
Father’s cell phone number
Father’s work number
Father’s e-mail address
Name of relative or close friend
Address of relative or close friend
Telephone number of relative or close friend
Cell phone number of relative or close friend
Work phone number of relative or close friend
Relationship of relative or close friend
Part II: Student Background and Previous Experiences
Because this study focuses on students’ trajectories through high school and the decisions they make along the way, it is important to gather information on their background and experiences before they enter high school. This is crucial in establishing baseline information on students so that effects of school, family, and peers on educational trajectories and decisions can be effectively identified apart from preexisting characteristics of sample members. Once collected, these measures will be used mostly as predictors of high school and postsecondary outcomes.
Questions 25–29. The information generated will help shed light on important issues such as sex, race/ethnicity, and socioeconomic differences in educational outcomes. Although the best source of socioeconomic status and family structure data will be the parent questionnaire, if past experience is a guide, about 7 percent of parents will be nonrespondents. Because it is critical to have socioeconomic status and family structure information for all respondents, we also ask students about their parents’ occupation and household composition. All of these questions have been used in NELS:88 and/or ELS:2002.
Are you Hispanic or Latino/Latina?
If you are Hispanic or Latino/Latina, please select one of the following choices:
Mexican, Mexican-American, Chicano
Other Hispanic or Other Latino/Latina
(Note: all students will next be routed to the race question, 25b.)
b. Please select one or more of the following choices to best describe your race.
White
Black/African American
Asian
Native Hawaiian or Other Pacific Islander
American Indian or Alaska Native
c. If you marked Asian, which of the following are you?
Chinese
Filipino
Southeast Asian (Vietnamese, Thai, etc.)
South Asian (Asian Indian, Sri Lankan, etc.)
Other Asian (Korean, Japanese, etc.)
Note: the oversampling of Asians will produce analyzable numbers of Chinese, Filipino, and Southeast Asian ninth-graders. “South Asian” and “Other Asian” are residual categories.
What is your sex?
Male
Female
When were you born? (Month-Day-Year)
Which of the following people live in the same household with you? (Mark all that apply)
Father (biological or adoptive)
Other male guardian (stepfather or foster father)
Mother (biological or adoptive)
Other female guardian (stepmother or foster mother)
Brother(s) (including step- or half- or foster)
Sister(s) (including step- or half-)
Grandparent(s)
Other relative(s) (children or adults)
Nonrelative(s) (children or adults)
Please describe the current or most recent job of your father (mother) or male (female) guardian. If you have both a father (mother) and a male (female) guardian, answer for the one with whom you are currently living.
a. Is he (she) currently:
Working
Full-time homemaker
Unemployed (looking for work)
Retired
Disabled
b. What is his (her) job title? (Verbatim)
c. What does he (she) normally do in that job? That is, what are some of his (her) main duties? (Verbatim)
Note: this question is repeated in the same form, for mother’s occupation.
Questions 30–32. These questions are aimed at students for whom English is not their native language. The items will be used to address issues of language acquisition, usage, and the extent to which students’ English skills affect academic achievement and later postsecondary educational and occupational outcomes. The ample projected sample of Hispanic students as well as the oversampling of Asian students will make these particular items valuable in understanding the educational consequences of contemporary demographic changes in the United States. To measure language use, questions are drawn from ELS:2002 and the National Household Education Survey (NHES).
What was the first language you learned to speak when you were a child?
English
Spanish
Another language
English and Spanish equally
English and another language equally
(If another language) What is the other language/languages you learned to speak as a child?
Spanish
Another European language (for example, French, German, Russian, etc.)
A Chinese language
A Filipino language
A Southeast Asian language (for example, Vietnamese, Cambodian, etc.)
A South Asian Language (for example, Hindi, Tamil)
Another Asian Language (for example, Japanese, Korean)
A Middle Eastern Language (for example, Arabic, Farsi)
Other
How often do you speak (non-English language) with your mother or female guardian? Your friends?
(Never, Sometimes, About half of the time, Always or most of the time, Does not apply)
Questions 33–34. Since HSLS:09 will begin with a sample of ninth-graders, a grade with one of the highest levels of retention, we are expecting the sample will include a substantial portion of students who are repeating the 9th grade. It is important to identify these students so that analysts can make sense of students’ responses to questions about the previous year in school. As a number of repeaters will have transferred schools (i.e., start 9th grade at school A, fail, and then repeat 9th grade at school B), this information will be of value later in the study when collecting student transcripts.
In what grade were you enrolled during the previous school year (2007–08)?
7th grade
8th grade
9th grade
During the previous school year (2007–08) when you were in the ___ grade, where were you enrolled? (School Name-City-State) (fill in open field)
Question 35. This question is included to assess the extent to which parents are able to monitor and regulate their children’s behavior. Students’ reports are necessary as parents tend to overstate their parenting skills (e.g., “presentation bias”). This question has been developed and used by Jacqueline Eccles, a leading psychological expert on youth development in schools (see Downey, Eccles, and Chatman, 2005; Eccles, 2005; Wigfield and Eccles, 2001).
35. How much of the time do your parents know:
Where you are?
Who you are with?
(Never, Rarely, Sometimes, Often, Always)
How much do you agree or disagree with the following statements:
The rules in my family are quite clear.
My parents know if I follow family rules.
(Strongly Agree, Agree, Disagree, Strongly Disagree)
Questions 36–37. Students entering the ninth grade carry with them a growing résumé of activities both in and outside of school. These activities have been identified in the developmental psychological literature as crucial in establishing and maintaining self-identity and in the sociological literature as key behaviors connecting youth with their schools and communities. Since the HSLS:09 base-year interview occurs at the start of a new school-year in (for most) a brand-new school, there will be little history of activity participation during the current school year. Therefore, the survey will ask students about their participation between the start of the eighth grade and the present. To do so, the question wording of similar items used in NELS:88 and in ELS:2002 will be modified. A few new response items will be included that are specific to math and science activities to support the STEM focus that is unique to the present study. Question 36 is based on questions used in the Survey of Chemistry and Physics Graduate Students conducted by the University of Virginia on behalf of the National Science Foundation.
36. Have you ever participated in the following activities between the start of 8th grade and now? (Mark all that apply)
a. Sports teams that compete within the same school or sports teams that compete between different schools
b. Band, orchestra, chorus, choir
c. School play or musical
d. Student government/council
e. Volunteer work
f. Church groups
g. Math club, science club, technology/computer club
h. Math competition, science fair, technology/computer fair
i. Math camp, science camp, technology/computer camp
j. Math, science study groups or tutoring programs
37. Between the start of 8th grade and now, how often have you…
(Never, Rarely, Sometimes, Often)
a. Read science books and magazines?
b. Watched science-related television shows (e.g., CSI, NUMB3RS, Mythbusters, Animal Planet or Discovery channel?)
c. Read science fiction?
d. Visited a science museum (e.g., aquarium, natural history)?
Questions 38–41. One of the strongest predictors of later academic success is earlier performance in school. Ninth-graders enter high school with a history of academic successes and/or failures. In determining whether the effects of experiences in high school (e.g., courses, extracurricular activities, relationships with teachers, etc.) on later outcomes can be attributed to those experiences or to selection processes (e.g., preexisting differences among students that select them into different classrooms, activities, etc.), it is essential that information on previous grades and courses is collected. The TRP expressed concern that students may be unable to accurately report their eighth-grade courses, so their responses will be validated with information on their eighth-grade transcripts as received by the ninth-grade school (consistency). The results of this cross-check analysis will inform whether this question will be used in the full-scale study. While the eighth-grade school transcript is regarded as the validity criterion, it is possible that there may be more missingness in the administrative records, given that they will be collected from the student’s new (high) school. It is therefore all the more important to compare student report with school records to see which will be the best source of this critical information for the main study.
38. What math course did you take in the 8th grade?
Remedial Math 8
Math 8
Advanced or Honors Math 8 (not including Algebra)
Algebra
Geometry
Other (write in)
I was not taking math
Note: since the student questionnaire is electronic, a tailored alternative set of items can be addressed to those who in the Grade in Previous Year question identified themselves as ninth-grade repeaters (i.e., they will be asked about both eighth and ninth grade). The ninth grade math subject list is as follows:
Algebra I
Algebra IA
Algebra IB
Algebra II
Discrete Math
Geometry
Integrated Math I
Integrated Math II
Integrated Math III
Integrated Math IV
Probability/Statistics
Review/Remedial Math
Other Math Courses: _________________
I was not taking math
39. What was your final grade in this course?
a. A or a numerical average of 90–100
b. B or a numerical average of 80–89
c. C or a numerical average of 70–79
d. D or a numerical average of 60–69
e. Below D or a numerical average less than 60
f. Does not apply to me – my classes are not graded
g. Does not apply to me – I didn’t take math
40. What science course did you take in the 8th grade? (Mark all that apply)
a. General Science 8
Earth Science
Biology
d. Life Science
e. Physical Science
f. Chemistry or Physics
g. Other (write in)___________
h. Does not apply to me – I was not taking science
Note: since the student questionnaire is electronic, a tailored alternative set of items can be addressed to those who in the Grade in Previous Year question identified themselves as ninth-grade repeaters (i.e., they will be asked about both 8th and 9th grade). The ninth-grade science subject list is as follows:
Anatomy/ Physiology
Biology I
Biology II
Chemistry I
Chemistry II
Earth Science
Environmental Science
Integrated Science I
Integrated Science II
Integrated Science III
Integrated Science IV
Physical Science
Physics I
Other Science Courses: ________________
Does not apply to me – I was not taking science
41. What was your final grade in this course?
a. A or a numerical average of 90–100
b. B or a numerical average of 80–89
c. C or a numerical average of 70–79
d. D or a numerical average of 60–69
e. Below D or a numerical average less than 60
f. Does not apply to me – my classes are not graded
g. Does not apply to me – I was not taking science
42. What computer course did you take in the 8th grade?
Computer education/Computer science
Keyboarding
Other computer course
I was not taking a computer course
43. What was your final grade in this course?
a. A or a numerical average of 90–100
b. B or a numerical average of 80–89
c. C or a numerical average of 70–79
d. D or a numerical average of 60–69
e. Below D or a numerical average less than 60
f. Does not apply to me – my classes are not graded
Part III: Social Context and Interpersonal Influences
In accord with a host of past research conducted by NCES, students’ daily interactions with their parents, teachers, and peers play a large role in their transition to high school, their current academic investments, as well as their future plans. Like the previous section, this section will collect information that will mostly serve as predictors of later outcomes. Since many of the items measured here will also be collected in later rounds, this section provides a baseline for analyses of change and stability in social context and interpersonal influences, and how this overlaps with academic success and social development. Specifically, it will allow analysts to explore issues of school climate and support networks as students progress through high school.
Question 44. Whether students feel safe in their schools affects their connections with their teachers, their peers, and ultimately their sense of commitment to their education. Therefore, this question uses one of the key indicators used in NELS:88 and in ELS:2002 to capture students’ sense of safety.
44. How much do you agree or disagree with the following statement about your current school? I feel safe at this school.
(Strongly Agree, Agree, Disagree, Strongly Disagree)2
Questions 45–47. A core finding within developmental psychology as well as in the sociology of education is that the ongoing relationships students have with their peers, families, teachers, and community members influence their beliefs about their future and their subsequent behaviors. HSLS:09 provides a unique opportunity to link these influences with critical decisions that students make about their schooling and their career. This question set adopts questions used in NELS:88 that gauge these influences. Additionally, the set includes one new question (47) developed by Jacqueline Eccles (see Downey, Eccles, and Chatman, 2005; Eccles 2005;
Wigfield and Eccles, 2001) that has students compare their friends with their parents when dealing with life after high school.
45. Between the start of 8th grade and now, which of the following people have you talked to about the following…
(Mother or female guardian, Father or male guardian, Your friends, A favorite teacher, A school counselor, Haven’t discussed it with anyone)
a. What courses to take this school year
b. What math courses to take this school year
c. What science courses to take this school year
d. Going to college
e. Possible jobs/careers when you are an adult
f. Personal problems
Who provided the most helpful advice? (Mark one for each line)
(Mother or female guardian, Father or male guardian, Your friends, A favorite teacher, A school counselor, Haven’t discussed it with anyone)
a. What courses to take this school year
b. What math courses to take this school year
c. What science courses to take this school year
d. Going to college
e. Possible jobs/careers when you are an adult
f. Personal problems
46. Please think of your closest friend in this school who is a ninth-grader. As far as you know, are the following statements true or false for him/her?
a. Gets good grades
b. Is interested in school
c. Wants to finish high school
d. Attends classes regularly
e. Plans to go to college
f. Is popular with others
47. When I want to talk about my future plans, I talk (Mark one)
mostly to my parents
more to my parents than my friends
to my parents and my friends about the same
more to my friends than my parents
mostly to my friends
Part IV Students’ Values and Their Determinants
Research in social psychology, economics, and sociology identifies values as key determinants of behavior. For example, if students value going to college, their behaviors will be directed toward that end. In a departure from previous NCES longitudinal studies, HSLS:09 will include items that directly measure the value students place on school in general and in particular, math and science, their future careers, and the factors that shape these values. These factors include belonging, motivation, identity, perceived opportunities, and costs. These items can be used as outcomes predicted by student background, previous experiences, social context, and interpersonal influences, or they can be used as predictors of educational and occupational decisions measured in later rounds.
Questions 48–50. As key determinants of behavior, the HSLS:09 base-year interview will focus on everyday values regarding school as well as their future values regarding work and their careers. Everyday school values will be measured by Question 48, which was previously used in ELS:2002. To capture students’ propensities toward different subject matter, Question 49, which was developed and used by Jacqueline Eccles that asks students to report their favorite subject in school, will be used. Future work and career values will be measured by Question 50, an item used in the Sloan Foundation’s Study of Youth and Social Development. This question has been analyzed in a recent influential book by Barbara Schneider at the University of Chicago and the late David Stevenson at the National Council on Education Standards and Testing that documented the weakening linkages between students’ visions of the future and the reality of preparation and job performance.
48. How important is each of the following to you?
(Very important, somewhat important, not at all important)
a. Getting good grades in school generally
b. Getting good grades in my current math course
c. Getting good grades in my current science course
49. What is your favorite school subject (not including lunch or study periods)?
English
Non-English Language
History
Science
Art
Music
Math
Gym
Religion
Health
Computer
Government
Other (please specify)
50. For the job you expect to have at age 30, how important are the following to you:
(Very important, Somewhat important, Not at all important)
a. Helping people
b. Working to improve society
c. Having lots of free time
d. Working closely with people
e. Building or creating things
f. Having lots of money
g. Working outdoors
h. Learning new things
i. Expressing yourself
j. Working with animals
k. Not having to sit at a desk all day
l. Using no math
m. Using a computer
n. Working in a lab
o. I haven’t thought about it
p. I’m not really sure yet
Question 51. As mentioned earlier, the connections students have with their teachers and their peers shape their commitment to their education and ultimately their success in school. Therefore, the survey uses a widely accepted scale of school belongingness in the education research community developed by Kristin Voekl at Canisius College using the Tennessee Project STAR evaluation.
51. How much do you agree or disagree with the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I feel proud being part of my school.
b. I am treated with as much respect as other students in my class.
c. Most of the time I would like to be any place other than school.
d. There are always teachers or other adults in my school that I can talk to if I have a problem.
e. School is often a waste of time.
Questions 52–55. Intrinsic motivation is a well-studied concept among educational psychologists and is a strong predictor of academic success. The survey measures intrinsic motivation separately for math and for science using a scale developed by Elliot and colleagues. The survey also employs a further motivational scale from the Sloan Foundation’s Study of Youth and Social Development.
52. How much do you agree or disagree with the following statements about your current math class?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I am enjoying my math class very much.
b. I think this math class is a waste of my time.
c. I think my math class is fun.
d. I think this math class is boring.
e. I don’t like my math class at all.
53. How much do you agree or disagree with the following statements about your current science class?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I am enjoying my science class very much.
b. I think this science class is a waste of my time.
c. I think my science class is fun.
d. I think this science class is boring.
e. I don’t like my science class at all.
54. I do my assignments in my math classes because…
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I enjoy what I am learning.
b. I am interested in math.
c. I like learning something new.
d. I like taking on a challenge.
e. I want to be good at math.
f. My parents make me.
g. I want to keep up my grades.
55. I do my assignments in my science classes because…
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I enjoy what I am learning.
b. I am interested in science.
c. I like learning something new.
d. I like taking on a challenge.
e. I want to be good at science.
f. My parents make me.
g. I want to keep up my grades.
Questions 56–59. While STEM careers in adulthood are often prestigious, success in science/math while in high school does not typically confer high social status on adolescents. Educational psychologists contend that identification with STEM pathways while in high school, when the process of self-awareness and personality development is believed to crystallize, is crucial in understanding the foundation of later success in math and in science. To assess the extent to which this identity formation process shapes later educational and occupational decisions, a set of new items developed in part by Marie Claire-Shanahan at the University of Alberta will be used.
56. How much do you agree or disagree with the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I see myself as a math person.
b. Others see me as a math person.
c. I want others to see me as a math person.
56. How much do you agree or disagree with the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I see myself as a science person.
b. Others see me as a science person.
c. I want others to see me as a science person.
58. How much do you agree or disagree with the following statements? (Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I am good at math.
b. My math teacher thinks I am good at math.
c. My parents think I am good at math.
d. My friends think I am good at math.
59. How much do you agree or disagree with the following statements? (Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I am good at science.
b. My science teacher thinks I am good at science.
c. My parents think I am good at science.
d. My friends think I am good at science.
Questions 60–62. In simple terms, utility value refers to the importance of a current task for a future goal. In many cases, the future goal may be unrelated to the process nature of the current task. Seeded in the field of behavioral economics, the concept of utility value is used by educational psychologists to understand how youth make decisions about their current efforts in school. For example, if youth feel that doing homework will have some value that will bring satisfaction to them later in life by getting into college and getting a good job (e.g., doing homework has utility value) they will be more likely to complete their homework. Given that measures of utility value have been rarely used in national studies such as this, measures developed by Jacqueline Eccles will be tested in the field test. The TRP considered these new measures to be critical in establishing the underpinnings of student decision making.
60. How much do you agree or disagree with the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. The information we learn in school is useful for everyday life.
b. I like to get by in school doing as little work as possible.
c. The information we learn in school will be useful for college.
d. The information we learn in school will be useful for my career.
61. How much do you agree or disagree with the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. The information we learn in (current math class) is useful for everyday life.
b. The information we learn in (current math class) will be useful for college.
c. The information we learn in (current math class) will be useful for my career.
d. I really feel that I am wasting my time in (current math class).
e. I like to get by in (current math class) with doing as little work as possible.
62. How much do you agree or disagree with the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. The information we learn in (current science class) is useful for everyday life.
b. The information we learn in (current science class) will be useful for college.
c. The information we learn in (current science class) will be useful for my career.
d. I like to get by in (current science class) with doing as little work as possible.
e. I really feel that I am wasting my time in (current science class).
Questions 63–67. A central focus of the HSLS:09 is how opportunities, as well as perceptions of opportunities, influence the development of educational and career trajectories. In the ninth grade, one of the most immediate and salient forms of opportunity are courses. If youth are explicitly swayed from taking core foundational courses in math and science and see limited options for more advanced courses, their chances for taking courses that will propel them toward STEM career paths are substantially diminished. On a more macro level, if youth envision the larger opportunity structure as unfair, they will be less likely to exert effort in school that they see as having little payoff. Perceived opportunities will be measured using new questions that ask students their plans for future coursetaking and their motives behind these plans. Perceived barriers are then measured using questions that gauge students’ concrete attitudes toward educational opportunity from a more macro vantage point. These macro-level questions come from Roslyn Mickelson’s study of public high schools in Los Angeles that identified racial differences in attitudes toward educational opportunity as explaining why African American youth perform poorly in school.
63. How many years of math do you plan to take in high school?
64. Do you intend to enroll in: (Yes/No)
a. an Advanced Placement (AP) calculus course?
b. an International Baccalaureate (IB) calculus course?
65. How many years of science do you plan to take in high school?
66. Do you intend to enroll in: (Yes/No)
a. an Advanced Placement (AP) science course?
b. an International Baccalaureate (IB) science course?
(The computer program will assign the following to all respondents planning to take 3 or more years of math or science or AP/IB math or science.)
c. Mark all the reasons you plan taking more math. What is the most important reason? (Asked separately for math and science)
a. I need it in order to graduate
b. My parents will want me to
c. My teachers and/or guidance counselor will want me to
d. I am good at math/science
e. I will need advanced math/science for the type of career I want
f. Most students like me take advanced courses
g. I enjoy studying math/science
h. That kind of math will be useful for getting me into college
i. That kind of math will be useful in college
j. I don’t know why, I just probably will
k. It’s required
l. My friends are going to take it
m. I really hadn’t thought about it
67. How much do you agree or disagree with the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. Studying in school rarely pays off later with good jobs.
b. Even if I study, I will not be able to get into college.
c. Even if I study, my family cannot afford to pay for me to attend college.
Questions 68–69. Costs and benefits are central components to understanding behavior from the vantage of economics. While the benefits of educational investments (e.g., studying, engagement, coursetaking, etc.) are often reaped later in life via educational and occupational attainment, costs often impinge upon daily experience. Since costs are rarely examined explicitly in educational research using large scale surveys, there is little precedent for well-developed questions. The questionnaire asks two questions regarding costs. The first, Question 68, is a new question created by RTI International’s survey development team in consultation with NCES and the TRP. This question addresses the immediate costs of educational investments. The second, Question 69, is a scale created and validated by educational psychologists at Arizona State University (Jennifer Husman, Christa Lynch, Jonathon Hilpert, and Mary Ann Duggan) that measures the future time perspective of students.
68. How much do you agree or disagree with each of the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
If I spend too much time and effort in my math and science classes…
a. I won’t have enough time for other classes that I enjoy.
b. I won’t have enough time for hanging out with my friends.
c. I won’t have enough time for extracurricular activities.
d. I won't be popular.
e. I’ll be made fun of.
69. How much do you agree or disagree with each of the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I don’t think too much about the future.
b. It’s really no use worrying about the future.
c. What will happen in the future is important in deciding what I should do now.
d. What I do today will have little impact on what happens 5–10 years from now.
Part V: Students’ Expectations for the Future and their Determinants
In addition to values, past research using NCES longitudinal studies and other data sources have demonstrated that expectations for the future play a role in shaping behavior. For example, if students expect to work in their family business when they grow up, their behaviors will be directed toward that end. To further explore the role that these expectations play in the decision making processes of contemporary adolescents, HSLS:09 will include a host of items that directly measure the expectations students hold for postsecondary education and careers, as well as the factors that influence these expectations. These include attributions, self efficacy, deterrents, and negative experiences. As with students’ values, these items can be used as outcomes predicted by student background, previous experiences, social context, and interpersonal influences, or they can be used as predictors of later educational and occupational decisions. Additionally, given the historical use of expectations questions on NCES longitudinal surveys, analysts can make comparisons with previous high school cohorts.
Questions 70–72. The first set of expectation questions will assess the steps that students plan to take in order to fulfill their postsecondary expectations. As the economic and social benefits of a college degree have risen over the past two decades, the competition and preparation for college enrollment has become more entrenched in the high school years, with students knowledgeable about the college enrollment process being at a distinct advantage. The HSLS:09 will track the evolution of these preparatory steps during the first three years of high school. Two sets of questions will be asked regarding expectations for these preparatory steps. The first, Question 67, is based on items used in ELS:2002. The second, Questions 68-69, are new questions that ask about the steps that students will take to prepare for college and work. These will be tested using cognitive labs.
70. Have you taken or are you planning to take:
(I don’t know the test; Yes; No; I haven’t thought about it)
a. The PSAT
b. The College Board Scholastic Assessment Test (SAT)
c. American College Testing Service (ACT)
d. An Advanced Placement (AP) test
e. A test for the International Baccalaureate (IB)
71a. Have you put together a “college plan” or a series of activities and courses that you will need to complete in order to get into college?
(No, yes, I haven’t thought about it)
71b. If yes: Who helped you put the “college plan” together? (Mark all that apply)
(A counselor, a teacher, my parents, Other—specify)
72a. Have you put together a “career plan”—for example, a series of activities and courses that you will need to complete in order to be successful in your eventual career?
(No, yes, I haven’t thought about it)
72b. If yes: Who helped you put the “career plan” together? (Mark all that apply)
(A counselor, a teacher, my parents, Other—specify)
Questions 73–78. Postsecondary expectations have been core questions asked in all of the previous longitudinal studies. These questions have a long history within the sociology of education, stemming back to the influential Wisconsin Model of Status Attainment (the first empirical model in the United States that linked social origins with educational and occupational attainment in young adulthood). In the present study, these expectation questions will be critical in measuring how youths’ ideas about their future accord with their present-day decisions. These will be taken from a variety of sources: Question 73 is new; 74 is from ELS:2002; 75 is from NLS:72; 76 is new; 77 is from ELS:2002; and 78 is from Jacqueline Eccles (Eccles, 2005). Although Questions 73 and 76 are new, No problems are anticipated with them given their similarity to previously used items and their alignment with the surrounding items. NCES will pay close attention to these items when analyzing the field test data and recalibrate them if appropriate.
73. Do you expect to still be enrolled in school six months from now?
(No, Yes, I haven’t thought about it)
74. As things stand now, how far in school do you think you will get?
Less than high school
High school graduation or GED only
Attend or complete a 2-year program in a community college or vocational school
Attend college, but not graduate
Graduate from college with a 4-year degree
Obtain a Master’s degree or equivalent
Obtain a Ph.D., M.D., or other advanced degree
Don’t know
75. Would you be disappointed if at age 30 you did not have a bachelor’s degree? (Yes/No)
76. What is the principal activity that you plan to do right after high school?
(Mark one)
Enroll in a 4-year college or university
Enroll in a 2-year community college
Enroll in a vocational, technical, or trade school
Join the armed services
Get a job
Start a family
Travel
Do volunteer or missionary work
Not sure what I want to do
77. As things stand now, write in the name of the job or occupation that you expect or plan to have at age 30.
(Verbatim or
a. I don’t know)
78. How much have you thought about this choice?
(Likert scale (1-5), A little…Somewhat…A lot)
Questions 79–80. Self-concept reflects an individual’s knowledge and understanding of his or her self. In accord with the identity items asked earlier, measures of self-concept gauge the process of self-awareness and personality development in the early years of high school. Since youth base their expectations for the future on a current self-assessment, repeated measures of self-concept over time will help elucidate the process through which individuals modify or refine their sense of self. To measure self-concept, the survey draws on a well-tested scale developed and validated by Herbert Marsh at Oxford University.
79. How much do you agree or disagree with each of the following statements?
Compared to others my age….
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I am good at math.
b. Work in math class is easy for me.
c. I have to study hard in math.
80. How much do you agree or disagree with each of the following statements?
Compared to others my age….
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I am good at science.
b. Work in science class is easy for me.
c. I have to study hard in science.
Questions 81–82. Self-efficacy is the belief that one has the ability to accomplish a task. Students who feel they are able to succeed in certain areas will likely anticipate success later in life in similar areas. For purposes of HSLS:09, the focus is on self-efficacy in math and in science as links with later attempts to succeed in STEM courses and careers. The questionnaire adopts a scale used in ELS:2002 to measure self-efficacy in these two subject areas.
81. How much do you agree or disagree with each of the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I am confident that I can do an excellent job on my math tests.
b. I’m certain I can understand the most difficult material presented in my math textbooks.
c. I’m certain I can master the skills being taught in my math class.
d. I’m confident I can do an excellent job on my math assignments.
82. How much do you agree or disagree with each of the following statements?
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. I am confident that I can do an excellent job on my science tests.
b. I’m certain I can understand the most difficult material presented in my science textbook.
c. I’m certain I can master the skills being taught in my science class.
d. I’m confident I can do an excellent job on my science assignments.
Questions 83–85. A key focus of the previous high school longitudinal studies conducted by NCES (e.g., NELS:88, ELS:2002) has been on the factors that promote academic success and well-being. Very little attention has been paid to negative experiences that might deter academic success and well-being. Understanding these deterrents will provide valuable information on how students progress in the academic pipeline. The survey focuses on two forms of deterrents. The first is whether students are externally deterred from taking certain courses. This will be captured using a new measure developed by Jeremy Finn at the University of Buffalo which directly asks students about courses they were persuaded against taking and why they were deterred. The second is a measure developed and used by Jacqueline Eccles to gauge youth’s perceptions of gender differences in math and science (see Downey, Eccles, and Chatman, 2005; Eccles 2005; Wigfield and Eccles, 2001).
Gender beliefs may serve as an internal deterrent for young women, particularly in the hard sciences. Given the large policy focus on gender differences in math and in science, this item will be valuable in understanding this important component of STEM career formation. NCES plans a methodological experiment using this item:
How would you compare boys and girls in...? {Girls are much better, Girls are somewhat better, Girls and boys are the same, Boys are somewhat better, Boys are much better}
a. Reading, b. Math, c. Writing, d. Science
The intent of asking about reading and writing, areas in which girls stereotypically excel, is to provide balance to the question so as to minimize potential social desirability biases. The hypothesis is that parents, particularly those reporting on a daughter, may be more inclined to state their belief that boys are superior to girls in math and science if they are able to answer favorably about girls in other subject areas. The same may be true for students.
For the experiment, the subject areas would be randomly ordered such that reading or writing would appear first for some respondents while math or science would appear first for others. The analysis would compare responses based on the order in which the subject areas were listed, the sex of the parent, and the sex of the student. For the parent interview, the results will also be analyzed by mode of administration.
83. Was there any course you wanted to take this term but couldn’t? (yes/no)
What course? (if yes: verbatim)
84. (if yes to 80) Why? (Mark One)
Couldn’t fit into schedule
Wasn’t available
Needed to take required courses instead
I was discouraged from taking it by my parents
I was discouraged from taking it by a teacher or guidance counselor
I was discouraged from taking it by my friends
Other (specify)________
85. How would you compare boys and girls in …?
(Girls are much better, Girls are somewhat better, Girls and boys are the same, Boys are somewhat better, Boys are much better)
a. Reading
b. Math
c. Writing
d. Science
Part VI: Student Decisions
The central outcomes measured in the student questionnaire are the decisions made by the students. These include coursetaking (specifically in mathematics and science), student engagement, student time use, college, and careers. The information on these decisions will allow analysts to examine their relationship with constructs measured in the previous substantive sections. These include, but are not limited to, the relationship between sociodemographic background and coursetaking patterns; the role of parent and teacher support in student engagement; linkages between early orientations toward math and science and pursuit of a STEM career; and how values and expectations influence postsecondary decisions. In this base-year survey, the focus is on three decisions: coursetaking, engagement, and time use.
Questions 86–90. As a means to anchor the base-year data collection, information will be collected on the specific math and science courses in which students are enrolled. The quality of student reports can be compared to school administrative records to determine the best source for the main study. The teachers of these courses will be interviewed and the students will provide perceptions of their teachers. Having this information from a variety of sources (along with the assessment scores) will provide rich data on math and science education at the start of high school. The reasons that students report for taking a course are key items for future analysis.
86. What math courses are you taking this semester? Mark all that apply.
Algebra I
Algebra IA
Algebra IB
Algebra II
Discrete Math
Geometry
Integrated Math I
Integrated Math II
Integrated Math III
Integrated Math IV
Probability/Statistics
Review/Remedial Math
Other Math Courses: _________________
87. (For all math courses currently enrolled): I am taking this course because… (Mark all that apply)
a. I really enjoy math
b. I like to be challenged
c. I had no choice, it is a school requirement
d. The school guidance counselor suggested I take it
e. My parent(s) encouraged me to take it
f. A teacher or other school official encouraged me to take it
g. There were no other math courses offered
h. My friends were taking it
i. I will need it for college
j. I will need it for my career
k. I heard it was the easiest math class
l. I don’t know why I am taking this course
m. It was assigned to me.
88. (For all math courses currently enrolled): I am taking this course because… (Mark the most important or main reason)
a. I really enjoy math
b. I like to be challenged
c. I had no choice, it is a school requirement
d. The school guidance counselor suggested I take it
e. My parent(s) encouraged me to take it
f. A teacher or other school official encouraged me to take it
g. There were no other math courses offered
h. My friends were taking it
i. I will need it for college
j. I will need it for my career
k. I heard it was the easiest math class
l. I don’t know why I am taking this course
m. It was assigned to me.
89. What science courses are you taking this year? (Mark all that apply)
a. Anatomy/Physiology
b. Biology I
c. Biology II
d. Chemistry I
e. Chemistry II
f. Earth Science
g. Environmental Science
h. Integrated Science I
i. Integrated Science II
j. Integrated Science III
k. Integrated Science IV
l. Physical Science
m. Physics I
n. Other Science Courses: ________________
90. (For all science courses currently enrolled): I am taking this course because… (Mark all that apply and mark the most important or main reason)
a. I really enjoy science
b. I like to be challenged
c. I had no choice, it is a school requirement
d. The school guidance counselor suggested I take it
e. My parent(s) encouraged me to take it
f. A teacher or other school official encouraged me to take it
g. There were no other science courses offered
h. My friends were taking it
i. I will need it for college
j. I will need it for my career
k. I heard it was the easiest science class
l. I don’t know why I am taking this course
Questions 91–96. The decision to engage in school is fundamental in understanding youths’ commitment toward their education. The survey uses three sets of questions to measure three different components of engagement: Question 87 is a scale used in NELS:88 and ELS:2002 that measures daily preparation for school; Question 88 is an item from the Alliance for Excellent Education’s “Survey of Student Culture” that measures disengagement during class; and Questions 89–92 were developed by Heinrich Mintrop (UC-Berkeley) and Tina Trujillo (UCLA) to measure self-regulation.
91. How often do you go to class: (Never, Rarely, Sometimes, Often)
a. Without your homework done
b. Without pencil or paper
c. Without books
d. Late
92. How often: (Never, Rarely, Sometimes, Often)
a. Do you really pay attention during class
b. Does your mind wander in class
c. Do you joke around in class
d. Do you talk with your friends during class?
93. When you finish a math assignment, how often do you check to make sure it is done correctly? (Never, Rarely, Sometimes, Often)
94. When you are working on a math assignment, how often do you think about whether you understand what you are doing? (Never, Rarely, Sometimes, Often)
95. When you finish a science assignment, how often do you check to make sure it is done correctly? (Never, Rarely, Sometimes, Often)
96. When you are working on a science assignment, how often do you think about whether you understand what you are doing? (Never, Rarely, Sometimes, Often)
Questions 97–98. Developmental psychologists and sociologists contend that how youth spend their time is an expression of how they see their “future selves.” Youth who envision themselves holding professional jobs that require advanced degrees tend to spend more time studying and participating in academic-related activities, while those who do not tend to spend more time working and participating in nonacademic-related activities. The questionnaire uses a checklist from the National High School Survey of Student Engagement to measure the intensity of time use for key activities in the adolescent world. It will also use a question from ELS:2002 that measures special program participation.
97. About how many hours do you spend in a typical 7-day week during the school year doing each of the following?
(asked in continuous format, fill in number of hours)
a. Working on homework and studying for all of your classes
b. Working on math homework and studying for math class
c. Working on science homework and studying for science class
d. Watching television/movies
e. Working for pay (not including chores or jobs you do around your house)
f. Chatting or surfing online
g. Hanging out/socializing with friends outside of school
h. Playing video games
i. Talking on the phone/text messaging
j. Exercising
k. Sports
98. Are you participating in any of these activities? (Yes/No)
a. Talent Search
b. Upward Bound
c. Gear Up
Part VII: The Environment of Students’ Math and Science Classes
A unique feature of HSLS:09 is the in-depth examination of the math and science courses that students take in the ninth grade. In the survey, students will identify their current math and science courses and will answer a battery of questions about their perceptions of their teachers’ effectiveness. This information will be augmented by surveys of the teachers. Using this information in tandem with the other components of the student survey and the assessment, analysts will have rich information that will help identify the influence that teachers and ninth-grade math and science courses have on achievement. The focus is on two key components of the classroom: the teacher’s approach to students and the perceived effectiveness of the teacher.
Questions 99–100. These questions were developed by Jacqueline Eccles to assess the overall relationship that students have with their teachers. These will test the proposition that the more engaged the teacher is with the students, the more effective they are at teaching.
99. How much do you agree or disagree with the following statements about your math teacher? Remember, none of your teachers or your principal is going to see any of the answers you provide. (Strongly Agree, Agree, Disagree, Strongly Disagree)
My math teacher…
a. Values and listens to students’ ideas
b. Treats students with respect
c. Treats me like an adult
d. Treats every student fairly
e. Thinks every student can be successful
f. Thinks mistakes are okay as long as all students learn
g. Grades our math work fairly
h. Treats some kids better than other kids
i. Tries to make math interesting
j. Treats boys and girls differently
100. How much do you agree or disagree with the following statements about your science teacher? (Strongly Agree, Agree, Disagree, Strongly Disagree)
My science teacher…
a. Values and listens to students’ ideas
b. Treats students with respect
c Treats me like an adult
d Treats every student fairly
e. Thinks every student can be successful
f. Thinks mistakes are okay as long as all students learn
g.. Grades our science work fairly
h. Treats some kids better than other kids
i. Tries to make science interesting
j. Treats boys and girls differently
>> END OF STUDENT INTERVIEW <<
References
Converse, Jean M., and Presser, Stanley (1986). Survey Questions: Handcrafting the Standardized Questionnaire. Newbury Park, CA: Sage
Downey, G., Eccles, J.S., and Chatman, C. (2005). Navigating the Future: Social Identity, Coping, and Life Tasks. New York: Russell Sage.
Eccles, J. (2005). Studying gender and ethnic differences in participation in math, physical science, and information technology. New Directions for Child and Adolescent Development, 110.
Marsh, H.W., and Hattie, J. (1996). Theoretical perspectives on the structure of self-concept. In B. A. Bracken (Ed.), Handbook of Self-Concept (pp. 38-90). New York: Wiley.
Mickelson, R. (1990). The attitude-achievement paradox among black adolescents. Sociology of Education, 63: 44-61.
Schneider, B., and Stevenson, D.L. (1999). The Ambitious Generation. New Haven, CT: Yale University Press.
Voelkl, K.E. (1996). Measuring students’ identification with school. Educational and Psychological Measurement, 56: 760-770.
Wigfield, A.L., and Eccles, J.S. (2001). Development of Achievement Motivation. San Diego, CA: Academic Press.
Parent Survey
The parent questionnaire complements the student questionnaire by providing information on the student’s context and history, reporting on parental school involvement, and describing the home environment (e.g., values, expectations, and opportunities). Three research questions frame the parent questionnaire:
What social capital resources are available in the home environment to support children’s academic development and decision making (e.g., parent involvement in child’s decision making; course selection; planning for college or the labor market; shifts in involvement around key transitions – middle to high school, high school to postsecondary life; child’s involvement in extracurricular activities; child’s involvement in community activities [e.g., Girl Scouts, church groups])?
What human capital resources are available in the home environment to support children’s academic development and decision making (e.g., parents’ background in mathematics; parents’ background in science; parents’ attitudes about the importance of math, science, and education in general; parents’ expectations for children’s educational achievement; and parents’ expectations for their child’s career)?
What financial capital resources are available in the home environment to support children’s academic development and decision making (e.g., household income, savings, savings set aside for college education)?
Question 1. The first section of the parent interview will begin with questions about the student’s family situation. The first question will establish how the respondent is related to the ninth-grader. The parent or guardian who is most knowledgeable about the ninth-grader’s schooling will be asked to respond. Experience with this approach in the NCES longitudinal studies has shown that the vast majority of respondents to the parent interview will in fact be parents, most often mothers. However, given the diversity of family structures today a sizable number will be grandparents, other relatives, or guardians.
1. What is your relationship to [9th-grader]? Are you (his/her) biological (mother/father), adoptive (mother/father), step(mother/father) or someone else?
Biological mother/father
Adoptive mother/father
Stepmother/stepfather
Foster mother/father
Female partner of parent/guardian
Male partner of parent/guardian
Grandmother/father
Other relative
Other guardian
Questions 2–6. Follow-up questions will identify any other parents or guardians in the household and the marital status of the respondent. These questions which were used in ELS:2002 will be inputs to a composite family structure variable. Many studies have found a relationship between students’ family structure and academic outcomes including academic performance, dropping out of school, and continuing education after high school (for example, see Pong & Ju, 2000). Some factors that have been shown to moderate or amplify the relationship are whether the single parent was ever married (Jeynes, 2000), the presence of other guardians such as a grandparent in a single parent home (Deleire & Kalil, 2002), whether divorce occurred prior to the student’s adolescence (Theroux, 2000), and remarriage (Jeynes, 1999). So as to detect divorce, remarriage, or the death of a spouse during the student’s adolescence, parents when reinterviewed 3 years later will again be asked to report their marital status and identify any other parents and/or guardians living in the household.
2. (If respondent is not a parent) Does one or both of [9th-grader’s] biological or adoptive parents live in the same household as you and [9th-grader]?
Yes, two parents
Yes, one parent;
No, neither parent
3. (If respondent is a parent or neither parent lives in 9th-grader’s household) Do you have a spouse or partner who lives in the same household as you and [9th-grader]? Yes, a spouse
Yes, a partner
No, neither a spouse nor a partner)
4. (If respondent is a parent who has a spouse or partner OR respondent is not a parent but one or both parents live in household) What is (your spouse’s/your partner’s/this parent’s/these parents’) relationship to [9th-grader]?
Biological mother/father
Adoptive mother/father
Stepmother/stepfather
Foster mother/father
Grandmother/father
Other female relative
Other male relative
Other female guardian
Other male guardian
5. (If respondent is only parent in household) What is your current marital status? Married
Divorced
Separated
Never married
Widowed
Don’t know
OR
6. (If respondent is not a parent) What is [9th-grader’s] parents’ current marital status? Are they…?
Married
Divorced
Separated
Never married
Widowed
Don’t know
Questions 7–8. Additionally, many children have two homes, most often the two households of divorced parents. Respondents will be asked how often the ninth-grader lives with them and where the ninth-grader lives when not in their household. These questions, which were used in ELS:2002 and NELS:88, are important to assess the degree of contact the student has with the respondent as well as the stability of the student’s living situation.
7. How much of the time does [9th-grader] live with you (and your spouse/and your partner/and (his/her) parents)?
All of the time
More than half of the time
Half of the time
Less than half of the time
None of the time
8. With whom does [9th-grader] live most of the time when (he/she) does not live with you?
With his/her other parent
With another adult relative
With a friend
At boarding school
With a nonrelated adult guardian
By himself/herself
Other
Questions 9–10. The family structure section will close with a question about the age distribution of household members, and two questions about the ninth-grader’s siblings; the total number of siblings and the number of older siblings. A robust finding in the literature shows that as family size increases educational performance decreases (see for example Blake, 1989). These NELS:88 questions have been used in analyses that suggest that as family size increases parents’ social and economic resources are increasingly divided, which is detrimental to educational performance (Downey, 1995). While Question 9b provides information about birth order, Question 10 (taken from the NELS:88 second follow-up parent questionnaire) informs economic and poverty measures.
9a. How many brothers and sisters does [9th-grader] have? Consider all siblings, including half- and step- and adoptive brothers and sisters, regardless of where they live.
Full and/or adoptive brother(s)
Half-brother(s)
Step-brother(s)
Full and/or adoptive sister(s)
Half sister(s)
Step-sister(s)
9b. How many of these siblings are older than [9th-grader]? {open-ended}
10. Including yourself and your 9th grader, how many people living in your household are…
a. Under the age of 18
b. 18 years of age or older
Questions 11–34. The next section of the parent interview will collect demographic data, immigrant status, and language use variables. Each parent’s race and ethnicity will be collected following OMB guidelines. Since parents may be of different races and/or ethnicities, this information is collected from each parent or guardian. Parents’ ages will also be gathered. For biological parents, the parents’ ages in conjunction with the student’s age (as reported in the student questionnaire) may be used to determine at what stage in the parents’ life course the student was born.
Many analysts have used NELS:88 data to learn about the educational outcomes of first- and second-generation immigrant students (Hagy & Staniec, 2002; Kao & Tienda, 1995; Kaufman, Chavez, & Lauen, 1999; Mau, 1997) and bilingual students (Mouw & Xie, 1999). HSLS:09 will continue in this tradition. The questions about country of nativity are new to the NCES high school cohort studies, but have been used successfully in the National Household Education Survey (NHES).
11. Are you Hispanic or Latino/Latina? (yes, no)
12. (If yes) Which of the following are you?
a. Mexican, Mexican-American, Chicano
b. Other Hispanic
13. Please select one or more of the following choices to best describe your race.
a. White
b. Black/African American
c. Asian
d. Native Hawaiian or Other Pacific Islander
e. American Indian or Alaska Native
14. (If Asian) Which of the following are you?
a. Chinese
b. Filipino
c. Southeast Asian (Vietnamese, Thai, etc.)
d. South Asian (Indian, Sri Lankan, etc.)
e. Other Asian
15. Is your (spouse/partner) Hispanic or Latino/Latina? (yes, no)
16. If yes: If your (spouse/partner) is Hispanic or Latino/Latina, please select one of the following choices:
a. Mexican, Mexican-American, Chicano
b. Other Hispanic or Other Latino/Latina
17. Please select one or more of the following choices to best describe your (spouse’s/partner’s) race. Is (he/she)…?
White
Black/African American
Asian
Native Hawaiian or other Pacific Islander
American Indian or Alaska Native
18. If you marked Asian, which of the following is your spouse/partner?
a. Chinese
b. Filipino
c. Southeast Asian (Vietnamese, Thai, etc.)
d. South Asian (Asian Indian, Sri Lankan, etc.)
e. Other Asian
19. In what country (were you/was [9th-grader’s] mother) born?
20. (If not born in U.S.) At what age did (you/she) come to the United States to stay? {open-ended}
21. In what country (were you/was [9th-grader’s] father) born?
22. (If not born in U.S.) At what age did (you/he) come to the United States to stay? {open-ended}
23. In what country was [9th-grader] born?
24. (If not born in U.S.) At what age did [9th-grader] come to the United States to stay? {open-ended}
25. What grade was (he/she) placed in when (he/she) started school in the United States? {Pre-K, K, 1-9}
26. Is any language other than English spoken in your home? {yes, no}
27. (If yes) What languages other than English are spoken in your home?
Spanish
Another European language (for example, French, German, Russian etc.)
A Chinese language
A Filipino language
A Southeast Asian language (for example, Vietnamese, Thai, etc.)
A South Asian Language (for example, Hindi, Tamil)
Another Asian Language (for example, Japanese, Korean)
A Middle Eastern Language (for example, Arabic, Farsi)
Other
28. Is English also spoken in your home? {yes, no}
29. (If yes) What language does [9th-grader] usually speak to you in your home?
Spanish
Another European language (for example, French, German, Russian etc.)
A Chinese language
A Filipino language
A Southeast Asian language (for example, Vietnamese, Thai, etc.)
A South Asian Language (for example, Hindi, Tamil)
Another Asian Language (for example, Japanese, Korean)
A Middle Eastern Language (for example, Arabic, Farsi)
Other
30. (If yes) What language do you usually speak to [9th-grader] in your home?
Spanish
Another European language (for example, French, German, Russian etc.)
A Chinese language
A Filipino language
A Southeast Asian language (for example, Vietnamese, Thai, etc.)
A South Asian Language (for example, Hindi, Tamil)
Another Asian Language (for example, Japanese, Korean)
A Middle Eastern Language (for example, Arabic, Farsi)
Other
31. Has [9th-grader] ever been enrolled in an English as a Second Language (ESL) program? (Yes, No, Don’t know)
32. (If yes) Is [9th-grader] currently enrolled in an English as a Second Language (ESL) program? (Yes, No, Don’t know)
33. In what year (were you/was [9th-grader’s] mother) born? {open-ended}
34. In what year (were you/was [9th-grader’s] father) born? {open-ended}
Questions 35–49. The next series of questions will gather information that will be used to construct a measure of socioeconomic status (SES). The level of education, income, and occupation of each parent in the responding household will be collected. The SES composite will incorporate the 1989 General Social Survey (Nakao and Treas) occupational prestige scores (which update the Duncan SEI). In analyzing the reliability, predictive validity, and efficiency of the NELS:88 base-year SES composite, Freidlin and Salvucci (1995) concluded that occupation was of, at best, marginal interest for the composite and that father’s and mother’s education and family income were far more critical. However, the survey maintains in its entirety the traditional scheme. Data on parents’ occupations have an additional purpose in HSLS:09 given its emphasis on STEM. Parents who work in STEM fields may serve as role models to their children, be more likely to encourage their children to pursue a STEM career, be more likely to be proactive in the placement of their children in math and science courses, and be better equipped to help their children with math and science homework and activities. Expert coders will identify parents who work in STEM fields based on their job title and the description of their duties. Based on the suggestion of the Technical Review Panel, NCES also proposes asking parents how often they use advanced math and science on their job. However, since these are new items, they will be subject to cognitive testing. Parents who have completed some higher postsecondary education will also be asked to name their major field of study. Respondents who complete the survey online or with a telephone interviewer will code the field of study using an assisted coder. This process has been used successfully with students in a number of NCES studies including ELS, NPSAS, and BPS.
Conley (1999) suggests that parental assets may be more important than income to children’s educational outcomes. Since time constraints will not allow collection of extensive asset information, a brief question about home ownership will serve as a simple indicator of wealth. This item has been used successfully in the National Postsecondary Student Aid Study (NPSAS) and the Beginning Postsecondary Students Longitudinal Study (BPS).
35. What is the highest level of education you and your spouse/partner have reached? (Mark one for spouse, one for partner)
Did not finish high school
Graduated from high school or equivalent (GED)
Graduated from high school and attended a 2-year school (such as a vocational or technical school, a junior college, or a community college) but did not complete a degree
Graduated from a 2-year school (such as a vocational or technical school, a junior college, or a community collete)
Graduated from high school and went to college but did not complete a 4-year degree
Graduated from college
Started in a Master’s degree program but didn’t finish
Completed a Master’s degree or equivalent
Started in a Ph.D., M.D., or other advanced professional degree program but didn’t finish
Completed a Ph.D., M.D., or other advanced professional degree
Don’t know
Does not apply. I do not have a spouse or partner
36. (If at least associate’s degree) What was (your/her) major area of study for (your/her) [highest degree]?
37. What is the highest level of education (your spouse/your partner/[9th-grader’s] father) has completed?
38. (If at least associate’s degree) What was (his/her) major area of study for (his/her) [highest degree]?
39. During the past week, did (you/[9th-grader’s] mother) hold a job for pay or income? {yes, no}
40. (If yes) About how many total hours per week do (you/[9th-grader’s] mother) usually work for pay or income, counting all jobs? {open-ended}
41. What is (your/her) job title? If (you/she) had more than one job, describe the one at which (you/she) worked the most hours. {verbatim}
42. What (do you/does she) actually do in that job? That is, what are (your/her) main activities or duties? {verbatim}
43. During the past week, did [ninth-graders father) hold a job for pay or income?
44. If yes: About how many total hours per week (do you/does [9th-grader’s] father) usually work for pay or income, counting all jobs? {open-ended}
45. What is (your/his) job title? If (you/he) had more than one job, describe the one at which (he/she) worked the most hours. {verbatim}
46. What (do you/does he) actually do in that job? That is, what are (your/his) main activities or duties? {verbatim}
47. What was your total household income from all sources (including income from work, investment income, alimony, etc.) prior to taxes and deductions in calendar year 2008? {open-ended}
48. (If refuse) Please indicate the range that best estimates your total household income in calendar year 2008. {categorical with 12 options}
49. Do you own or pay mortgage on your home, rent your home, or have some other arrangement? {Own home, rent home, other arrangement}
Questions 50–57. The interview will then turn to the ninth-grader’s educational experiences, beginning with experiences prior to ninth grade which cannot be ascertained from high school records. Parents will be asked whether their student has ever repeated or skipped a grade in school and if so, in what grades retention and promotion occurred. Students who have been held back are significantly more likely to drop out of high school than their peers and those who do persist have significantly lower odds of enrolling in a postsecondary institution (Goldenring & Davis, 2003). Parents will also be asked how many times their ninth-grader has changed schools due to a residential relocation. The impact of residential moves on students’ education has been the focus of some inquiry using NELS:88 data (Pettit & Durfee, 2002; Pribesh & Downey, 1999; Swanson & Schneider, 1999). As such, the question is replicated here. Questions will also identify at-risk students with poor attendance records and behavior problems.
50. Since starting kindergarten, has [9th-grader] repeated any grades? (yes, no)
51. (If yes) What grades did (he/she) repeat? (K-9)
52. Since starting kindergarten, has [9th-grader] skipped any grades? (yes, no)
53. (If yes) What grade(s) did (he/she) skip? (K-8)
54. How many times has [9th-grader] changed schools since he/she entered school? Do not count changes that occurred as a result of promotion to the next grade or level (for example, a move from an elementary school to a middle school or from a middle school to a high school in the same district). (open-ended)
55. Since the first grade, has [9th-grader] ever stopped going to school for a period of a month or more other than for illness, injury or vacation? (yes, no)
56. Since the first grade, has [9th-grader] ever been suspended (not counting detentions) or expelled from school? (yes, no)
57. From the beginning of 8th grade up to now, how many times did [9th-grader’s] school contact you (or your spouse/partner) about [9th-grader’s] behavior in school? [9th-grader’s] attendance record at school? (Never, Rarely, Sometimes, Often) (K-12)
Question 58. Two questions which also appeared in the NELS:88 parent questionnaire focus on algebra coursetaking. Research has demonstrated that students who take algebra before high school have a distinct educational advantage (Atanda, 1999; Lee, Burkam, Chow-Hoy, Smerdon, & Goverdt, 1999; Rock & Pollack, 1995). Given the importance of algebra as a gatekeeper course and HSLS:09’s focus on STEM, parents are asked whether their ninth-grader took algebra before starting high school and who was most influential in this decision.
58. Did [9th-grader] take algebra in either the 7th or 8th grade? (Yes, No, Don’t know)
Questions 59–65. Parents’ involvement in their child’s education may be particularly important during the transition from middle school to high school. Parents’ advocacy in their child’s education at this transitional period will be measured by several questions. The question about the frequency with which parents talk with other parents about matters related to their child’s eighth- and ninth-grade schools is a modified version of a question from NELS:88. The NELS:88 question was more limited in its scope as it asked about advice provided by parents of the child’s friends. The question presented here pertains to exchange of information as well as advice. It also is not limited to conversations with friends’ parents since in some cases students may have friends who do not attend the same school. A new item asks parents if they attended an open house at their ninth-grader’s high school before the school year began. An item used in NHES:2003 asks parents whether they requested their ninth-grader’s placement with a particular ninth-grade teacher or course. The question about parent-child discussions of school-related matters is a staple to the NCES high school cohort studies. A new item asks parents to assess the degree of influence they believe they have on their child’s school-related decisions. There is some evidence that high levels of parent-child interaction are related to shared educational aspirations which in turn promote achievement (Hao & Bonstead-Bruns, 1998). ELS:2002 and NELS:88 are the source of questions about the degree to which parents monitor their child’s academic work and social life.
59. How often did you talk with other parents about teachers, courses, or school events at [9th-grader’s] 8th grade school? at [9th-grader’s] current school? (Never, Rarely, Sometimes, Often)
60. Did you (or your spouse/partner) attend an open house or orientation at [9th-grader’s] high school before this school year began? (yes, no)
61. Did you request that [9th-grader] get or not get a particular 9th grade teacher or course at (his/her) high school? (yes, no)
62. In the past year, how often did you (and your spouse/partner) discuss the following subjects with [9th-grader]? (Never, Rarely, Sometimes, Often)
a. Selecting courses or programs at school
b. School activities or events
c. Things [9th-grader] has studied in class
d. [9th-grader’s] grades
e. Your [9th-grader’s] plans after leaving high school
f. Applying to colleges or other schools after high school
g. Jobs [9th-grader] might like to have when (he/she) grows up
63. How much influence do you think you have on the choices [9th-grader] makes about school? (None, A little, Some, A lot)
64. Are there family rules that are enforced for [9th-grader] about the following activities? (yes/no)
a. Earning acceptable grades in school
b. Doing homework
65. In a typical week, what is the latest [9th-grader] can stay out on school nights (Sunday - Thursday)?
Not allowed out
No later than 8:00pm
No later than 9:00pm
No later than 10:00pm
No later than 11:00pm
No later than midnight
As late as (he/she) wants
Questions 66–71. Students with special needs may or may not receive the services they need to maximize their potential. Parents will be asked if their student has an Individualized Education Plan (IEP), if he or she is enrolled in a Gifted and Talented (GATE) program or an English as a Second Language (ESL) program. Parents will also be asked if they believe their son or daughter has a learning disability.
66. In your opinion, does [9th-grader] have a specific learning disability? (Yes, No, don’t know)
67. Does your [9th-grader] currently have an Individualized Education Plan (IEP)? {Yes, No, Don’t know}
68. If yes, what is the primary category? (List the 13 federal disability categories: specific learning disability, speech or language impairments, etc.)
69. Has [9th-grader] ever been diagnosed by a doctor as having a specific learning disability? (Yes, No, don’t know)
70. Is [9th-grader] currently enrolled in a gifted and talented education (GATE) program? (Yes, No, Don’t know)
Questions 71–74. On the advice of the TRP, a new question asking parents about whether their son or daughter has had any academic instruction outside of school has been included. Follow-up questions identify the subject matter of the instruction and whether the study was remedial. Parents will also be asked about their child’s participation in organized activities outside of school.
71. From the start of 8th grade until now, did [9th-grader] have any academic instruction outside of school such as from a Saturday Academy, learning center, personal tutor or summer school program? (yes, no)
72. (If yes) In what subjects was this instruction?
Reading
Math
Science
Writer
Other (specify) ___________
73. Was this remedial instruction? (yes, no)
74. During the last 12 months, has [9th-grader] participated in any of the following activities outside of school? (yes, no)
a. Music, dance, art, or theater
b. Organized sports supervised by an adult
c. Religious youth group or religious instruction
d. Scouting or another group or club activity
e. A math or science camp
f. Another camp
g. Any other regular activities or lessons
Question 75. Given the focus of HSLS:09 on STEM, the TRP strongly recommended that a new question about STEM-related family activities be included. Since this item is newly written for the field test, parents will be able to identify other unlisted STEM-related activities in which they have participated. These verbatim responses will identify other common activities that should be included in the full-scale version of this question.
75. In the past year, that is, since {month, day}, has anyone in your family done the following activities with your 9th-grader? (Yes, No)
a. Visited a zoo, planetarium, natural history museum, transportation museum, or a similar museum?
b. Worked or played on a computer together?
c. Built or fixed something such as a vehicle or appliance?
d. Attended a school science fair?
e. Helped [9th-grader] with a school science fair project?
f. Discussed a program or article about math, science, or technology?
g. Another science, technology, engineering or math-related activity (specify)
Questions 76–77. Since females are less likely than males to persist in the STEM pipeline, there is research interest in identifying the causes of this disparity (Huang, Taddese, & Walter, 2000). Gender stereotypes about ability in science and math may play a role. Therefore, parents are asked about how they compare boys and girls in their math and science ability. Parents are also asked about reading and writing, domains stereotypically favorable to women, to provide balance to the question and to minimize social desirability biases. This item is new to the NCES high school cohort studies, but has been used in the 13-year University of Michigan Childhood and Beyond study.
76. How would you compare boys and girls in...? (Girls are much better, Girls are somewhat better, Girls and boys are the same, Boys are somewhat better, Boys are much better)
a. Reading
b. Math
c. Writing
d. Science
77. How important do you think the following subjects are for your ninth-grader to meet (her/his) educational and occupational goals? (Extremely important, Important, Somewhat important, Unimportant)
a. Reading
b. Math
c. Writing
d. Science
Questions 78-81. The final substantive portion of the interview includes questions about parents’ expectations and aspirations for the ninth-grader’s education. Parents’ educational aspirations for their children are often adopted by the children (Hao & Bonstead-Bruns, 1998; Trusty & Pirtle, 1998). A distinction is made between aspirations and expectations to account for barriers, perceived or real, that the parent may anticipate the child will face. Parents who expect that their child will continue his or her education after high school will be asked what type of postsecondary institution is the likely point of entry. A response option will allow parents to indicate that they have not yet thought about this. Parents will also be asked about how they plan to finance their ninth-grader’s education after high school. If the parent does not expect the teenager to continue his or her education after high school, the respondent will be asked for the reasons for this expectation.
78. How far in school do you want [9th-grader] to go? (Mark one)
Not finish high school
Graduate from high school or equivalent (GED)
Graduate from high school and attend a 2-year school (such as a vocational or technical school, a junior college, or a community college) but not complete a degree
Graduate from a 2-year school (such as a vocational or technical school, a junior college, or a community college)
Graduate from high school and go to college but not complete a 4-year degree
Graduate from college
Obtain a Master’s degree or equivalent
Obtain a Ph.D., M.D., or other advanced degree
79. As things stand now, how far in school do you think [9th-grader] will actually get? (Mark one)
Not finish high school
Graduate from high school or equivalent (GED)
Graduate from high school and attend a 2-year school (such as a vocational or technical school, a junior college, or a community college) but not complete a degree
Graduate from a 2-year school (such as a vocational or technical school, a junior college, or a community college)
Graduate from high school and go to college but not complete a 4-year degree
Graduate from college
Obtain a Master’s degree or equivalent
Obtain a Ph.D., M.D., or other advanced degree
80. (If thinks 9th grader will continue education after high school) Do you think [9th- grader] will start (his/her) college education at a:
Vocational or trade school
2-year school
4-year school
I have not thought about this yet
81. Do you or does anyone in your family plan to help [9th-grader] pay for (his/her) education after high school? (Yes, no)
Finally, parents will be asked to provide information that will assist NCES in locating them and their 9th-grader in future follow-ups of the study. Locating items will include: name, address, home and work telephone numbers, e-mail address, Social Security number, ninth-grader’s Social Security number, spouse/partner’s name, if ninth-grader has parent/guardian outside the interviewed household, name and address, home and work telephone number. Also: name, address and telephone number of close relative or friend, person’s relationship to the respondent, and their name, address and phone number.
Date of interview is also recorded, as well as source of any assistance in completing the questionnaire. All of these elements—locating information, date of interview, information about assistance in interview completion—are taken from the ELS:2002 base-year survey of parents.
>> END OF PARENT INTERVIEW<<
References
Atanda, R.T. (1999). Do gatekeeper courses expand options? Washington, DC: National Center for Education Statistics. Statistics in Brief. [NCES 1999-303]
Blake, J. (1989). Number of Siblings and Educational Attainment. Science, 245(4913): 32-37.
Conley, D. (1999). Being Black, Living in the Red: Race, Wealth and Social Policy, Los Angeles: University of California Press.
Deleire, T., Kalil, A. (2002). Good things come in threes: Single-parent multigenerational family structure and adolescent adjustment. Demography, 39(2): 393-414.
Downey, Douglas B. (March 1995). Understanding Academic Achievement among Children in Step-households: The Role of Parental Resources, Sex of Stepparent, and Sex of Child. Social Forces, 73(3): 875–894.
Friedlin, B., and Salvucci, S. (March 1995). Empirical Evaluation of Social, Psychological, And Educational Construct Variables In NCES Surveys. National Center for Education Statistics Working Paper, No. 94.14.
Goldenring Fine, J., and Davis, J.M. (2003). Grade retention and enrollment in post-secondary education. Journal of School Psychology, 41(6): 401-412.
Hagy, A.P., and Staniec, J.F.O. (2002). Immigrant status, race, and institutional choice in higher education. Economics of Education Review, 21(4): 381-392.
Hao, L., and Bonstead-Bruns, M. (1998). Parent-child differences in educational expectations and the academic achievement of immigrant and native students. Sociology of Education, 71(3): 175-198.
Huang, G., Taddese, N., and Walter, El. (2000). Entry and Persistence of Women and Minorities in College Science and Engineering Education. Washington DC: National Center for Educational Statistics. Research and Development Report. [NCES 2000-601]
Jeynes, WH. (1999). Effects of remarriages following divorce on the academic achievement of children. Journal of Youth and Adolescence, 28(3): 385-393. [EJ600506]
Jeynes, W.H. (2000). The effects of several of the most common family structures on the academic achievement of eighth-graders. Marriage and Family Review, 30(1-2): 73-97.
Kao, G., and Tienda, M. (1995). Optimism and achievement: The educational performance of immigrant youth. Social Science Quarterly, 76(1): 1-19.
Kaufman, P., Chavez, L., and Lauen, D. (1998). Generational Status and Educational Outcomes Among Asian and Hispanic 1988 8th-Graders. Washington, DC: National Center for Education Statistics. Statistical Analysis Report. [NCES 1999-020]
Lee, V.E., Burkam, D.T., Chow-Hoy, T., Smerdon, B.A., and Goverdt, D. (1999). High School Curriculum Structure: Effects on Coursetaking and Achievement in Mathematics for High School Graduates. Washington, DC: National Center for Education Statistics. Working Paper. [NCES 98-09]
Mau, W.-C. (1997). Parental influences on the high school students. Academic achievement: A comparison of Asian immigrants, Asian Americans, and white Americans. Psychology in the Schools, 34(3): 267-277.
Mouw, T., and Xie, Y. (1999). Bilingualism and the academic achievement of first- and second-generation Asian Americans: Accommodation with or without assimilation? American Sociological Review, 64(2): 232-252.
Pettit, B., and Durfee, A. (2002). The Consequences of Residential Mobility During Childhood on Early Adult Outcomes. Paper presented at the Annual Meeting of the American Sociological Association, Chicago, IL.
Pong, S., and Ju, D. (2000). The Effects of Change in Family Structure and Income on Dropping Out of Middle and High School. Journal of Family Issues, 21: 147-169.
Pribesh, S., and Downey, D.B. (1999). Why are residential and school moves associated with poor school performance? Demography, 36(4): 521-534.
Rock, D.A., and Pollack, J.M. (1995). Mathematics Course-Taking and Gains in Mathematics Achievement. Washington, DC: National Center for Education Statistic. Statistics in Brief. [NCES 95-714, RIEJAN96, ED386499, TM024075]
Swanson, C.B., and Schneider, B. (1999). Students on the move: Residential and educational mobility in America’s schools. Sociology of Education, 72: 54-67.
Theroux, P.J. (2000). Transitioning Families, Adjusting Adolescents: Linking Parent Partnering Patterns to Teen School Performance. Ph.D. dissertation, Columbia University.
Trusty, J., and Pirtle, T. (1998). Parents’ transmission of educational goals to their adolescent children. Journal of Research and Development in Education, 32(1): 53-65.
School Administrator Questionnaire
The school administrator provides contextual information about him- or herself, school climate, staffing, and resources.
The purpose of the HSLS:09 7School Administrator Questionnaire is to support the study’s main research objectives: How do young adults choose the pathways they do, particularly pathways into science, technology, engineering, and mathematics (STEM) careers? What role does high school (or the high school years) play in students’ ultimate decisions? And, what role does “algebra learning” in high school play in students’ decisions to pursue a career in STEM specifically, and more globally, in providing students with the ability to reason, persist, and achieve throughout life?
To achieve its purpose, the HSLS:09 School Administrator Questionnaire has been designed to provide school-level contextual data and control variables (e.g., public/private high school; tracking) for examining and interpreting students’ decision making and planning processes. And, because HSLS:09 schools will comprise a representative sample, questionnaire data may also be used to draw a descriptive profile of the course and program offerings, reform efforts, and math and science focus of American high schools with 9th and 11th grades.
Although questionnaire items were selected to achieve the overall goals and purposes of the study as mentioned above, selection was guided primarily by the desire to address the following questions specific to schools:
What school structures, policies, practices, and offerings facilitate or inhibit different high school trajectories and decisions (e.g., coursetaking, dropping out, going on to work or college)?
What programs and policies do schools offer to assist students at risk of school failure, including students at risk of dropping out, students transitioning from middle school to high school, and students struggling in math and science?
What are the school-level correlates of high-achieving schools in math and science (e.g., principal training and experience, climate, ease of hiring and retaining qualified math and science teachers, program offerings in math and science, and supports for struggling students)?
What is the math and science focus of schools (e.g., what explicit activities, if any, are schools engaged in to raise students’ interest and performance in math and science)?
Is the math and science focus of schools associated with students’ subsequent performance in math and science and decisions to pursue careers in math and science?
Items were also selected based on the need to collect certain data in students’ 9th- versus 11th-grade year. For example, the question on block scheduling is proposed for the 9th-grade School Administrator Questionnaire because it is subject to change. Asking the item later would not indicate whether the school had block scheduling previously to which the HSLS:09 student was exposed and may have benefited. School practices and policies that are less likely to change over time will be asked in the 11th-grade School Administrator Questionnaire scheduled for the spring of 2012. This division of items also keeps the burden of the questionnaire to 30 minutes. Four sets of items have already been identified for the 11th-grade HSLS School Administrator Questionnaire: (1) student evaluation items; (2) accountability and standardized assessment items, (3) extracurricular activities items, including career exploration; and (5) an additional item on climate.
The School Administrator Questionnaire collects information on the school in five domains: (1) school and student characteristics; (2) teaching staff characteristics; (3) school policies, practices, and programs; (4) school governance and climate; and (5) principal background and experiences. Data gathered in the School Administrator Questionnaire can be merged with data from the student, counselor, and teacher questionnaires and the student cognitive assessment. This linkage will allow researchers to determine the school structures, policies, and practices that facilitate or inhibit different high school trajectories and decisions, such as coursetaking, dropping out of school or going on to college or work, and specifically with respect to decisions concerning the pursuit of STEM careers.
Questions 1–3. Questions 1 and 2 are measures of school size. Question 3 measures grade span. These questions are taken from the ELS:2002 Base-Year School Administrator Questionnaire without modification. School size has been related to educational outcomes with medium-sized schools having the highest levels of student engagement and achievement (Bracey, 1998; Lee & Smith, 1997). Data on overall school size and the grade levels served by the school will enable researchers to determine school cohort size.
As of October 1, 2009 (or the most recent date for which data are available), what was the total student enrollment in your school? (Write in #)
As of October 1, 2009 (or the most recent date for which data are available), what was the total 9th grade student enrollment in your school? (Write in #)
Mark each grade level included in your school. (K-12)
Questions 4–5. These questions measure school type and structure. Question 4 is from the 2009 NAEP School Questionnaire, without modification, and Question 5 is from the ELS:2002 Base-Year School Administrator Questionnaire, without modification. The nation’s educational system comprises a wide array of high school types. Public policy makers, researchers, and parents all share an interest in schools that effectively educate high school students (Schneider, Teske, & Marschall, 2000). There is evidence that certain subsets of students attending magnet schools and Catholic schools have higher levels of achievement in some academic subjects (this is not, for example, true for science in Catholic schools, but is the case for English and math) than their counterparts in public comprehensive schools (Gamoran, 1996; Sebring & Camburn, 1992). There is also tremendous interest in the growth of charter schools, and new school-reform structures, such as schools within schools, career academies, and early-college high schools. While HSLS:09 is not designed to draw direct comparisons among different types of schools per se, these questions coupled with the study’s representative school sample can be used to draw a descriptive profile of American high schools in 2009.
Is your school a public charter school? (Yes, No)
What type of school is this? (Mark all that apply)
Comprehensive public school (not including magnet school or school of choice)
Public magnet school (e.g., whole school, magnet program, school within a school)
Public magnet school with a specialized academic, career or technical theme (e.g., a high school for agricultural sciences, International Baccalaureate program)
Public school of choice (open enrollment/nonspecialized curriculum)
Year-round school
High school served by an area or regional vocational school/center (part-time or part-day)
Full-time technical or vocational school
Other technical or vocational school
Catholic school
Other private school, religious affiliation
Private school, no religious affiliation
Boarding school
Indian reservation school
Military academy
Alternative/stay-in-school/dropout prevention school/continuation school
Coeducational
Same sex school
Early college high school
Autonomous small schools sharing a principal
Autonomous small school with own principal
School-within-a-school (SWS) sharing a principal with other SWSs or academic programs in same building, but not a career academy
Question 6. This question measures school type and structure. The question is newly developed for this survey. Factors that may contribute to low performance in urban high schools include offering curricula that do not support the skills students need after high school and offering narrowly defined vocational training for jobs with little future. Many schools, however, are offering career academies, which offer training in a career field growing in demand, developed with guidance from representatives from the industry and the community. Positive results of participation in career academies include higher graduation rates and scholarship rates among participating students and improved teacher-student relationships that extend beyond the school day (Shorr & Hon, 1999).
Does your school have one or more career academies (defined as a school-within-a-school that focuses on career preparation)? (Yes, No)
Question 7. This question on school capacity is intended to measure the constructs of overcrowding and underutilization. The question is newly developed for this survey. School overcrowding is measured as a percentage, using the number of students enrolled in a school and the number of students the facility is designed to serve. Inadequate school facilities signal limited educational opportunities and resources for students (Ready, Lee, & Welner, 2004).
What is your school’s current student capacity expressed as a percent, for example, 120 percent filled, 46 percent filled? (Write in percent)
Questions 8–11. These questions measure a school’s calendar. Questions 8 and 9 are from the ELS:2002 First Follow-up School Administrator Questionnaire, without modification. Questions 10 and 11 are from the ELS:2002 Base-Year School Administrator Questionnaire, with modification. Many researchers have been looking into alternatives to traditional class scheduling and how they might be related to student achievement. Block scheduling, for example, generally organizes courses around one semester of 90-minute classes, rather than two semesters of 50-minute classes. When block scheduling is adopted, a student can complete a greater number of classes within a 4-year period. Longer class periods may also promote learning because teachers can explore more material and more topics with students in more depth than they could in a shorter period. The amount of instruction that students receive has been directly related to their academic success (Hernandez-Gantes, 1993; Purkey & Smith, 1983). As a result, graduation rates increase and discipline referrals and dropout rates are lowered (Queen, 2000). Therefore, longer school days and academic calendars, and alternative class scheduling, may facilitate student learning.
What kind of academic calendar does your high school (9–12) have?
Semester system
Trimester system
Quarter system
Block schedule (such as 4x4 or A/B)
Other
Are any of the following types of courses offered through block scheduling?
Academic courses, Y/N minutes/block
Vocational/technical courses, Y/N minutes/block
Other courses, Y/N minutes/block)
If your high school uses a traditional (nonblock) schedule or also offers courses on a traditional schedule in addition to a block schedule, how many minutes long are courses on the traditional schedule? (Write in minutes)
On average, how many hours of instruction per day do students receive at your high school? (Write in hours and minutes)
Question 12. This question measures student turnover. The question is newly developed for this survey. Studies of student mobility indicate that up to 50 percent of school changes are not due to a change in students’ residences. Rather, they are due to students leaving one school for another. School policies, practices, and characteristics (e.g., composition and size) have an influence on student turnover by affecting conditions that keep students engaged in school (Rumberger & Thomas, 2000). Data on how many 9th-grade students returned to the school the following year may indicate which schools have ineffective or harmful policies and practices.
What percentage of 9th-grade students who were enrolled in your school in September of 2008 returned to your school in September of 2009? (Write in percent)
Question 13. This question measures 9th-grade student retention. The question is newly developed for this survey. Grade retention rates vary considerably across states and among minority and nonminority students (Jackson, 1975). Jimerson (2001) notes that students who are retained a grade are at greater risk for future academic failure and are more likely to drop out of high school. Specifically, this item is intended to measure a school’s capacity to assist students with the transition from middle school to high school, and the school’s approach to helping students who are struggling academically.
What percentage of the 2009–10 ninth-grade class is repeating ninth grade? (Write in percent)
Question 14: This question was developed to measure a number of constructs in a minimally burdensome way. One construct is the poverty level of the school and/or degree of economic and educational advantage. The second construct relates to the preventive and enrichment programs schools offer. The question is from the ELS:2002 First Follow-up School Administrator Questionnaire, without modification. This question will allow researchers to create a profile of schools’ enrichment and preventive programs for at-risk and high-achieving students. In addition, these data will indicate how many students are making use of these programs. The question will also allow characterization of the school as affluent or advantaged or disadvantaged and poor.
What percentage of the total student body in your school (Write in percent; Check box, if school does not offer):
receive free or reduced-price lunch
are English language learners (ELL)
receive Special Education services for students with disabilities
are enrolled in an alternative program
are enrolled in a dropout prevention program
are enrolled in College Board Advanced Placement (AP) courses
are enrolled in an International Baccalaureate program (IB)
Questions 15–16. These questions measure school-level race/ethnicity. The questions are from the NELS:88 Base-Year School Administrator Questionnaire, with modification. These data, in combination with other school-level data, will provide researchers with information about how the racial and ethnic makeup of a school is related to student outcomes and achievements.
What percentage of the total student body in your school is of Hispanic or Latino origin? (Write in percent)
What percentage of the total student body in your school are members of the following groups? (Write in percent)
Asian
American Indian or Alaska Native
Black or African American
Native Hawaiian or Pacific Islander
White
Don’t know students’ race
Question 17. Together with other questions in the School Administrator Questionnaire, this item is designed to capture several concepts—the academic press of the school, and the schools’ predominant educational mission. Schools with a high percentage of students going on to a 4-year degree-granting institution are directed toward college preparation, whereas schools with a high percentage of students entering the workforce or going on to a technical/trade school after high school are directed toward career training.
What percentage of last year’s 12th-grade class: (write in percent)
a. were accepted by 4-year degree-granting institutions
were accepted by 2-year institutions
were accepted by technical/trade schools
entered the workforce
joined the military
Question 18. This question measures student absenteeism. The question is from the NELS:88 First Follow-up School Administrator Questionnaire, with modification. Moos and Moos (1978) found that student absenteeism is related to classroom social climate. Roby (2004) also found that there was a statistically significant relationship between student attendance and student achievement. He also found that this relationship was most significant at the ninth-grade level. A school with a high average daily attendance rate is likely to have a positive social climate and higher student achievement.
What was the average daily percentage attendance rate for students in your school last year? (Write in average daily attendance rate)
Question 19. This question is a measure of school-level dropout. The question is newly developed for this survey. Alternative programs or schools are ways of addressing the needs of students who are at risk for dropping out of high school or have been suspended or expelled. Schools that lose students to alternative programs generally have students with low math and reading achievement, low socioeconomic backgrounds, and are racial or ethnic minorities (DeRidder, 1991). This question will provide information about whether schools lose students to alternative programs and why.
What percentage of students attending your school last year (2008–09) were transferred out to an alternative program or school? (Write in percent)
Teacher Staff Characteristics
Questions 20–21. These questions measure teacher characteristics. The questions are from the ELS:2002 Base-Year School Administrator Questionnaire, with modification. The size of the faculty in conjunction with the size of the student body will allow researchers to calculate the student-teacher ratio in math and science classes. Zahorik (1999) notes that classrooms with a smaller class size have three significant advantages: more instruction due to fewer discipline problems, more student knowledge, and higher teacher enthusiasm for teaching. These questions also provide the denominator or base for interpreting data collected through teacher characteristic questions (e.g., Questions 25–28).
For each of the areas listed below, please indicate the number of full-time or part-time teachers. Please give your best estimate. (If a teacher works full-time in your school, but divides his/her time between two or more subject areas, consider that teacher as part-time in each subject or area.)
Math
Science
Art
Music
English
Foreign Language
Social science/social studies
History
Vocational/technical education
Physical education
Special education
All other subject areas in your school
Questions 21–22. These questions measure teacher staffing. The questions are from the 2007–08 SASS School Questionnaire, with modification. Research by Darling-Hammond and Hudson (1990) indicates that when schools experience a shortage of well-qualified teachers, they often compensate by increasing class sizes, increasing the number of courses teachers teach, cancelling certain courses, or filling positions with less qualified teachers, all of which reduce the quality of education. Additionally, poor students and students of color are more likely to be assigned to teachers with the least preparation and the weakest academic backgrounds (Murnane & Steele, 2007). The following two questions will provide information about whether schools have difficulty hiring qualified math and science teachers and therefore must compensate in ways that may lower the quality of math and science instruction.
For last school year (2007–08), were there teaching vacancies in either your math or your science departments for which teachers were recruited and interviewed? (NSkip to Q23 YGo to Q22)
How easy or difficult was it to fill the teaching vacancies in the mathematics or the science departments in your school? (Mark one response in each column: Math department/Science department)
No vacancies in this department
Easy
Somewhat difficult
Very difficult
Could not fill the vacancy
Questions 23–24. These questions also measure teacher staffing. The questions are newly developed for this survey. Chamberlin et al. (2002) note that performance incentives assist in the recruitment and retention of high-quality staff. Offering signing bonuses and other incentives to qualified candidates is one way school districts are working to make up for teacher shortages, especially in the areas of math and science (also foreign language and special education) (Pricola, 2001) and in high-needs schools and schools with lower achievement (Keller, 2006).
Does your school/district offer incentives or signing bonuses to attract qualified full-time math teachers (e.g., monetary bonuses, tuition aid, or tuition tax credits)? (No, Yes incentives, Yes signing bonuses)
Does your school/district offer incentives or signing bonuses to attract qualified full-time science teachers (e.g., monetary bonuses, tuition aid or tuition tax credits)? (No, Yes incentives, Yes signing bonuses)
Questions 25–26. These questions measure school-level teacher and staff qualifications. The questions are from the ELS:2002 Base-Year School Administrator Questionnaire, with modification. Only limited work has been done with national data on the potentially important relationship between teacher licensure and student achievement or other outcomes. Using NELS:88 data, however, Goldhaber and Brewer (2000) found that math teachers who are certified have a statistically significant impact on student math test scores relative to teachers who are not certified in their subject area. (However, they also found that students who have teachers with emergency credentials do no worse than students whose teachers have standard certification credentials.) In ELS:2002, as it will be in HSLS:09, the certification question was asked of both teachers and school administrators, because one cannot generalize from teachers to the situation of the school as a whole. Including this question in the school administrator questionnaire will allow this distinction to continue being made.
Of the full-time teachers who instruct math in your school, how many are certified by your state to teach math at the secondary school (9–12) level? (Write in number)
Of the full-time teachers who instruct science in your school, how many are certified in the state to teach science at the secondary school level? (Write in number)
Questions 27–28. These questions are also measures of teacher and staff qualifications. The questions are newly developed for this survey. Studies have found that teacher turnover is high in schools with low-income, low-achieving, and minority students, and teachers in those schools generally leave for schools that are more economically and educationally advantaged. Schools with high turnover rates face lack of continuity in instruction, lack of teaching expertise to make curriculum decisions, and lost time and resources for training new teachers (Loeb, Darling-Hammond & Luczak, 2005).
How many full-time math teachers who taught in your school last year (2008–2009), did not return to teach at your school this year (2009–2010)? (Write in number)
How many full-time science teachers who taught in your school last year (2008–2009), did not return to teach at your school this year (2009–2010)? (Write in number)
Questions 29–30. These questions measure teacher and staff absenteeism. They are from the 2009 NAEP School Questionnaire, with modification. There are two notable negative implications of high teacher absenteeism. First, teachers who receive low performance ratings are associated with higher levels of absenteeism. Second, high rates of teacher absenteeism are associated with lower student achievement (Norton, 1998).
Of your school’s full-time math teachers, about how many are absent on an average day? (Write in number)
Of your school’s full-time science teachers, about how many are absent on an average day? (Write in number)
School Policies, Practices, and Programs
Questions 31–34. These questions measure students’ opportunity to learn. The questions are from the NELS:88 First and Second Follow-up School Administrator Questionnaires, with modification. Finn et al. (2002) note that the number of courses offered at a school determines what courses students take and therefore what they learn. Students have higher achievement and are more encouraged to pursue higher education if the school provides the opportunity to take advanced classes and there are increased requirements for graduation. For example, when schools offered a wider variety of math courses, including advanced-level courses, students in academic and general tracks were more likely to take advanced-level courses.
Which of the following science courses does your school offer? Which of the following are open to 9th-graders?
Anatomy/Physiology
Biology I
Biology II
Biology AP
Chemistry I
Chemistry II
Chemistry AP
Earth Science
Environmental Science
Environmental Science AP
Integrated Science I
Integrated Science II
Integrated Science III
Integrated Science IV
Physical Science
Physics I
Physics II
Other Science Courses: ________________
Which of the following math courses does your school offer? Which of the following are open to 9th-graders?
Algebra I
Algebra IA
Algebra IB
Algebra II
Calculus
Calculus Prep
Calculus AP (AB)
Calculus AP (BC)
Discrete Math
Geometry
Integrated Math I
Integrated Math II
Integrated Math III
Integrated Math IV
Probability/Statistics
Probability/Statistics AP
Computer Science AP
Review/Remedial Math
Other Math Courses: _________________
Does your school offer an International Baccalaureate (IB) program? (Yes, No)
Which of the following mathematics courses are required for graduation? (Mark all that apply)
Algebra I
Geometry
Algebra II
Probability/Statistics
Data Analysis
Other Math Courses: __________________
No specific courses required
Question 35. This question measures teacher assignment to classes. The question is from the 2009 NAEP School Questionnaire, without modification. The question is meant to provide information about how teachers are assigned to teach students of different ability levels. As previously noted, poor students, students of color, and disadvantaged students are more likely to be assigned to teachers with the least preparation and the weakest academic backgrounds (Murnane & Steele 2007). If class assignments are made based on teacher seniority or performance, for example, the highest quality teachers with the most experience will more likely be assigned to the highest performing classes, leaving the less qualified or inexperienced teachers to teach the lower performing students.
How does your school assign teachers to math/science classes? (Mark all that apply)
Requests are granted based on teacher seniority
Class assignments are made based on teacher performance
Class assignments are rotated
Class assignments are made based on balancing teaching loads
Class assignments are made based on experience
The principal/school administrator/department head decides which teacher fits best with a particular class or group of students
Questions 36–37. These questions measure instructional strategies (differentiated instruction/ tracking). The questions are from the ELS:2002 Base-Year School Administrator Questionnaire, with modification. Tomlinson (2001) notes that students today have vastly different learning styles and come from a variety of backgrounds. Students have differing levels of ability across and even within subjects. The following questions identify a school’s approach to the challenge of educating students with differing learning rates and styles, and whether students’ rates and styles of learning are addressed effectively. The data will be of interest to researchers interested in equity of outcomes for students with different racial/ethnic backgrounds.
Which of the following best describes your school’s approach to providing instruction in math to students who come to you with different abilities, learning rates, interests, or motivations in math (do not include Special Education students when considering your answer)?
We offer differentiated courses in math but students have open access to any course provided they have taken the required prerequisite(s)
We offer differentiated courses and do differentiated grouping in math
We offer a variety of undifferentiated courses in math, and students have open access to any course provided they have taken the required prerequisite(s)
Other (specify)
Which of the following statements best describes your school’s approach to providing instruction in science to students who come to you with different abilities, learning rates, interests, or motivations in science (do not include Special Education students when considering your answer)?
We offer differentiated courses in science but students have open access to any course provided they have taken the required prerequisite(s)
We offer differentiated courses and do differentiated grouping in science
We offer a variety of undifferentiated courses in science, and students have open access to any course provided they have taken the required prerequisite(s)
Other (specify)
Questions 38-40. These questions measure tracking and stratification and equal opportunity to learn. The questions are newly developed for this survey. If schools assign students to different courses or sections of courses, also known as organizational differentiation, assignments are generally made based on past performance, current achievement, and/or cognitive skills (Sorensen, 1970). Sorensen notes that as a result of student homogeneity in differentiated courses, achievement levels of high-ability classes may increase, while achievement levels of lower ability classes may decrease.
Does your school offer different levels of Algebra I for students who vary in ability or in academic background (e.g., prior 8th-grade coursework in math)? (Yes, No)
Are students assigned to math courses, or sections of math courses, by ability? (Yes, No)
Are students assigned to science courses, or sections of science courses, by ability? (Yes, No)
Questions 41–42. These questions measure the availability of a dropout prevention program. The questions are from the ELS:2002 First Follow-up School Administrator Questionnaire, without modification. Cavanagh (2002) indicates that successful school dropout prevention programs help at-risk students succeed academically and socially, reducing the risk that they will drop out of high school. The following two questions will provide information about whether a school is working to reduce dropout rates and how students are identified and referred to such a program. These questions are intended to address the issue of whether schools offer programs that support struggling and at-risk students, and what kind of programs.
Does your school have a formal dropout prevention program? (Yes/Noskip)
On what basis are students recommended for your dropout prevention program? (Mark all that apply)
Absentee record
Academic performance
Teacher’s referral
Counselor’s referral
Parental request
Student request
Disciplinary problems
Questions 43–44. These questions measure schools’ assistance for struggling students – math and science. The questions are newly developed for this survey. A study by Ladouceur and Armstrong (1983) determined that a program that included study assistance for struggling students increased performance in students exhibiting the potential for academic problems. School-based programs for academically at-risk students, particularly those at risk of dropping out of high school, can be effective in helping students overcome their failings by focusing on a positive interpersonal climate to promote growth and by using a variety of instructional strategies (Alfassi, 2003). Similar to schools that engage in a formal dropout prevention program, schools that provide assistance for struggling students help to increase student academic achievement and reduce student dropout rates.
Are any of the following programs offered at your school to assist students who are struggling with mathematics? (Mark all that apply)
Evening high school credit recovery program
Homework assistance program
School-run tutoring program
Peer tutoring
Other tutoring program
Extra subject period
Off-track credit recovery program
Summer school credit recovery program
Supplementary instruction after regular school hours or on Saturdays
Other, please specify: verbatim
Are any of the following programs offered at your school to assist students who are struggling with science? (Mark all that apply)
Evening high school credit recovery program
Homework assistance program
School-run tutoring program
Peer tutoring
Other tutoring program
Extra subject period
Off-track credit recovery program
Summer school credit recovery program
Supplementary instruction after regular school hours or on Saturdays
Other, please specify: verbatim
Question 45. This question measures school programs that assist with the transition from middle school to high school/9th grade (elementary/middle school outreach). The question is from the NELS:88 Base-Year School Questionnaire – Survey of Middle Grade Practices, without modification. Three characteristics of schools that offer transition programs for students leaving middle school and entering high school are higher socioeconomic status, more positive learning relationships between students and teachers, and schools with a larger cohort of eighth-graders entering high school (Smith, 2001). In addition, Smith determined that students who participated in high school transition programs were less likely to drop out of school and more likely to succeed academically in high school.
How does your school assist students in the transition from middle school to high school? (Mark all that apply)
a. No transition, high school grades continue in K–12, 7–12, or other program
b. No special activities until students arrive at high school in the fall
c. High school students present information at the middle schools
Middle grade students invited to social event at high school before school year starts
Middle grade students attend regular classes at high school
Buddy or big brother/sister programs that pair new student with older ones at entry
Parents visit high school for orientation in the fall after children have entered
Summer meetings at the high school
High school counselors meet with students while they are still in the middle grades
Middle grade and high school teachers meet together on courses and requirements
Middle grade and high school administrators meet together on articulation and programs
Middle grade counselors meet with high school counselors or staff
Ninth-graders are placed in small learning communities or 9th Grade Academies
Other (specify) __________________________________
Questions 46–47. These questions also measure school programs that assist students with the transition from middle school to high school/ninth grade, specifically ninth-graders who struggle with the transition from middle to school high school. Question 46 is newly developed for this survey. Question 47 is from the ELS:2002 First Follow-up School Administrator Questionnaire, with modification. Students whose academic difficulties are not addressed have decreased interest in academics, and their expectations and likelihood for future academic success decreases (Miller & Atkinson, 2001). Schools offer various services intended to improve struggling student achievement, but whether schools can adequately address the issues of struggling students depends on the services offered relative to students’ needs.
Does your school offer any of the following programs to assist ninth-graders who are struggling academically? (Mark all that apply)
Summer program prior to entry into high school that provides supplemental instruction in reading and math
Small learning communities or Achievement Academies for over-aged students who have not met high school entry criteria
Double-blocked class schedules
Catch-up courses
After-school tutoring
Weekend tutoring
On what basis are ninth-graders who are struggling academically recommended to receive assistance? (Mark all that apply)
Absentee record
Academic performance
Teacher’s referral
Counselor’s referral
Parental request
Student request
Disciplinary problems
School Governance and Climate
Question 48. This question is a measure of a school’s climate. The question is from the ELS:2002 Base-Year School Administrator Questionnaire, without modification. A similar scale has been used in the Schools and Staffing Survey (SASS). Understandably, more learning takes place and student achievement is higher when schools are not experiencing school disorder (e.g., violence, crime, classroom disruption) (Chen 2007).
To the best of your knowledge how often do the following types of problems occur at your school? (happens daily, happens at least once a week, happens at least once a month, happens on occasion, never happens)
Tardiness
Absenteeism
Class cutting
Physical conflicts among students
Robbery or theft
Vandalism
Use of alcohol
Use of illegal drugs
Students under the influence of drugs/alcohol while at school
The sale of drugs on the way to or from school and/or on school grounds
Possession of weapons
Physical abuse of teachers
Student racial tensions
Student bullying
Student verbal abuse of teachers
In-class misbehavior
Student acts of disrespect for teachers
Gang activities
Question 49. This question is a measure of discipline. The question is from the ELS:2002 Base-Year School Administrator Questionnaire, without modification. The question is one measure of the school’s supervision of students and efforts to involve parents. A school’s early detection of a habitual absence and subsequent intervention may divert at-risk adolescents from dropping out. Likewise, parental involvement with the school community has been found to be associated with students’ academic success (Lee, 1995; Newmann & Wehlage, 1995). This question will reveal the efforts of schools to include parents in handling student disengagement from school.
When students are absent without an excuse, are parents notified? (Yes, No)
Principal Background and Activities
Question 50. This item measures principals’ goals. The item is from the ECLS-K Base-Year School Administrator Survey, with modification. Principals have an influence on school effectiveness based on decisions they make related to student learning (Hallinger et al., 1996). In addition, these effects are most likely to occur as a result of a principal’s efforts to influence those who come in direct contact with students, mainly teachers. For example, effective schools have clearly defined and communicated goals focusing on student achievement (Hallinger & Murphy 1985).
How much emphasis do you place on the following goals and objectives for your teachers? (No or minor emphasis; Moderate emphasis; Major emphasis)
Assisting all students to achieve high standards
Using curricula aligned with high standards
Maintaining a quiet and orderly classroom environment
Providing challenging material, activities, and assignments for higher achieving students
Using instructional strategies aligned with high standards (e.g., differentiated instruction)
Communicating well with parents
Working well with other staff
Openness to new ideas and methods
Participating in professional development activities
Questions 51–53. These questions measure principals’ credentials/qualifications/experience. Question 51 is from the SASS School Questionnaire, without modification. Questions 52 and 53 are from the ELS 2002 Teacher Questionnaire, with modification. In recent years, high school principals have been given more complex and demanding responsibilities, and continuing education helps principals encourage effective renewal and innovation within their schools. In the 1999–2000 school year, 99 percent of secondary school principals held master’s or doctorate degrees. Principal educational attainment was also higher in urban and suburban schools than in rural schools (Rodriguez-Campos et al., 2005).
What is the highest degree you have earned?
Do not have a degree
Associate’s degree
Bachelor’s degree
Master’s degree
Educational specialist or professional diploma
Doctorate or first professional degree
What were your major and minor (or 2nd major) fields of study for your undergraduate degree? (Mark one response in each column) (Major, Minor, 2nd major)
Education administration/instructional leadership
Education
English
Mathematics
History/social studies/social science
Natural/physical sciences
Foreign languages
Physical education
Vocational education
Business
Other (Please Specify)
Does not apply
What were your major and minor (or 2nd major) fields of study for your graduate degree? (Mark one response in each column) (Major, Minor, 2nd major)
Education administration/instructional leadership
Education
English
Mathematics
History/social studies/social science
Natural/physical sciences
Foreign languages
Physical education
Vocational education
Business
Other (Please Specify)
Does not apply
Questions 54–55. These questions also measure principals’ credentials/qualifications/ experience. The questions are from the SASS School Questionnaire, without modification. Archer (2002) notes that the New York City public schools, like school districts across the nation, are seeing long-time school principals replaced with new principals who have less than 3 years of school leadership experience. In New York, principals with 30 years of experience feel unable to cope with school leadership as new systems develop. On the other hand, inexperienced principals require support to help them obtain the necessary skills for leading a school. The following two questions will provide information about how a principal’s level of experience affects the climate and effectiveness of a school.
Prior to this school year, how many years did you serve as the principal of this or any other school? (Write number of years)
Prior to this school year, how many years did you serve as the principal of this school? (Write number of years)
Questions 56–59. These questions also measure principals’ credentials/qualifications/ experience. The questions are from the SASS Principal Questionnaire. Halawah (2005) notes that rather than focusing on administrative issues, effective principals manage their staff with a focus on education-related issues such as providing knowledge, information, materials, and supplies that support the work of teachers. In addition, years of teaching experience is an important determinant in successful principalship (Rodriguez-Campos et al., 2005). The following set of questions provides information about whether principals have the necessary classroom experience to support the instructional goals of teachers and other academic staff in their school.
Before you became a principal, how many years of elementary or secondary teaching experience did you have? (Write number of years separately for elementary [K–5], middle [6–8], and secondary [9–12]).
57. ADD: What was the main subject that you taught? (Fill in)
Since becoming a principal, how many years of elementary, middle, or secondary teaching experience have you had? (Write number of years separately for elementary, middle, and secondary)
In addition to serving as principal, are you currently teaching in this school? (Yes, No)
Questions 60–61. These questions measure principals’ credentials/qualifications/experience. The questions are newly developed for this survey. Forty-nine states have certification requirements for school administrators. These requirements were developed with the hope of producing quality school leaders. Requirements for certification can include currently holding a valid teaching certification, holding valid teaching certificate in the past, and/or completing a state-approved preparation program (LeTendre & Roberts, 2005). Data on state certification of school principals can provide information about whether principals have the required management and instructional knowledge to lead effective schools. Information will also be provided about whether certification requirements are appropriately connected with the characteristics of effective school leaders.
Are you certified as a principal in your state? (Yes, No)
Did you become a principal through alternative prep programs (e.g., New Leaders for New Schools)? (Yes, No)
Question 62. This question measures principal activities; how principals spend their time. The question is from the ECLS-K Base-Year School Administrator Survey, with modification. Graham (1997) conducted a survey of 500 elementary and secondary school principals. The average school principal indicated that he or she spent between 46 and 60 hours per week on school-related activities. A significant amount of this time was taken up by administrative activities, including paperwork and phone calls. An additional large portion of time was spent supervising students and faculty, attending meetings, and dealing with discipline situations. Fewer than 5 hours per week were devoted to each additional activity, including curriculum development, teacher evaluation, scheduling, and meeting with parents.
Please estimate the percentage of time you spend in an average week on the following activities: (Write in percent time spent totaling to 100%)
Working with teachers on instructional issues
Internal school management (e.g., weekly calendars, vendors, office, memos, etc.)
External school management (e.g., district/superintendent meetings, financial operations, public relations, communicating with decision-makers outside the school community, etc.)
Student discipline/attendance
Monitoring hallways, campus, lunchroom
Teaching
Talking and meeting with parents
Meeting with students
Paperwork required by local, state, or federal authorities
Question 63. This question is a measure of professional development. The item is from the SASS School Questionnaire, without modification. Halawah (2005) also notes that the effectiveness of the school principal has a profound and positive effect on a school’s climate. In order for a principal to be effective, he or she must possess a wide range of leadership and communication abilities. Beck and Murphy (1993) indicate that effective principals act as business executives, using good management and social science research to run schools. The following question will provide data about whether principals have acquired managerial leadership and communication skills outside the field of education.
Before you became a principal, did you have any management experience outside of the field of education? (Yes, No)
Questions 64-66. These questions measure characteristics of principals. The questions are from the ECLS-K Base-Year School Administrator Survey, with modification. Research indicates that gender, for example, influences instructional leadership among principals. Female principals are more likely to view themselves as curriculum and instructional leaders, while their male counterparts tend to view themselves as general managers (Hallinger et al., 1996). Also, research on principals’ race by McCray et al. (2007) indicates that a principal’s race often determines what type of school he or she leads. African American principals are most likely to be placed in schools with predominantly African American students, and Mexican American principals are most likely to be placed in schools with predominantly Mexican American students, while White principals are placed in schools with varying levels of diversity.
What is your sex? (Male/Female)
Are you of Hispanic or Latino Origin (Yes, No)
Please select one or more of the following choices to best describe your race. (Mark all that apply)
American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or other Pacific Islander
White
>> END OF SCHOOL ADMINISTRATOR INTERVIEW <<
References
Alfassi, M. (2003). Promoting the will and skill of students at academic risk: An evaluation of an instructional design geared to foster achievement, self-efficacy, and motivation. Journal of Instructional Psychology, 30(1), 28-40.
Archer, J. (2002). Novice principals put huge strain on N.Y.C. schools. Education Week, 21(38), 1-2.
Beck, L.G., and Murphy, J. (1993). Understanding the Principalship: Metaphorical Themes, 1920’s-1990’s. New York: Teachers College Press.
Bracey, G.E. (1998). An optimal size for high schools? Research. Phi Delta Kappan, 79(5), 406.
Cavanagh, S. (2002). Best programs empower students. Education Week, 21(43), 46.
Chamberlin, R., Wragg, T., Haynes, G., and Wragg, C. (2002). Performance-related pay and the teaching profession: A review of the literature. Research Papers in Education, 17(1), 31-49.
Chen, G. (2007). School disorder and student achievement: A study of New York City elementary schools. Journal of School Violence, 6(1), 27-43.
Darling-Hammond, L., and Hudson, L. (1990). Precollege science and mathematics teachers: Supply, demand, and quality. Review of Research in Education, 16, 223-264.
DeRidder, L.M. (1991). How suspension and expulsion contribute to dropping out. Education Digest, 56(6), 44-47.
Finn, J.D., Gerber, S.B., & Wang, M.C. (2002). Course offerings, course requirements, and course taking in mathematics. Journal of Curriculum and Supervision, 17(4), 336-366.
Gamoran, A. (1996). Student achievement in public magnet, public comprehensive, and private city high schools. Educational Evaluation & Policy Analysis, 18(1), 1-18.
Goldhaber, D.D., and Brewer, D.J. (2000). Does teacher certification matter? High school teacher certification status and student achievement. Educational Evaluation and Policy Analysis, 22(2), 129-145.
Graham, M.W. (1997). School Principals: Their Roles and Preparation. Oklahoma City, OK: National Conference on Creating the Quality School (ERIC Document Reproduction Service No. ED411576).
Halawah, I. (2005). The relationship between effective communication of high school principal and school climate. Education, 126(2), 334-345.
Hallinger, P., and Murphy, J. (1985). Assessing the instructional management behavior of principals. The Elementary School Journal, 86(2), 217-247.
Hallinger, P., Bickman, L., and Davis, K. (1996). School context, principal leadership, and student reading achievement. The Elementary School Journal, 96(5), 527-549.
Hernandez-Gantes, V.M. (1993). Direct and indirect influences of school learning variables on Hispanic-American eighth-grade students’ academic achievement. Dissertation Abstracts International, 54(2-A), 463.
Jackson, G.B. (1975). The research evidence on the effects of grade retention. Review of Educational Research, 45(4), 613-635.
Jimerson, S.R. (2001). Meta-analysis of grade retention research: Implications for practice in the 21st century. School Psychology Review, 30(3), 420-437.
Keller, B. (2006). Teacher-incentive plans geared to bonuses for individuals. Education Week, 26(12), 7.
Ladouceur, R., and Armstrong, J. (1983). Evaluation of a behavioral program for the improvement of grades among high school students. Journal of Counseling Psychology, 30(1), 100-103.
Lee, S. (1995). Family-School Connections and Student’s Education: Continuity and Change of Family Involvement from the Middle Grades to High School. Doctoral dissertation, Johns Hopkins University, Baltimore, MD.
Lee, V.E., and Smith, J.B. (1997). High school size: Which works best and for whom? Educational Evaluation and Policy Analysis, 19(3), 205-227.
LeTendre, B.G., and Roberts, B. (2005). A National View of Certification of School Principals: Current and Future Trends. Nashville, TN: University Council for Educational Administration.
Loeb, S., Darling-Hammond, L., and Luczak, J. (2005). How teaching conditions predict teacher turnover in California schools. Peabody Journal of Education, 80(3), 44-70.
Massachusetts Department of Education (2006). Massachusetts Charter School Achievement Comparison Study: An Analysis of 2001-2005 MCAS Performance. Retrieved February 4, 2008 from http://www.doe.mass.edu/charter/reports/datastudy/report.pdf.
McCray, C.R., Wright, J.V., and Beachum, F.D. (2007). Beyond Brown: Examining the perplexing plight of African American principals. Journal of Instructional Psychology, 34(4), 247-255.
Miller, S.D., and Atkinson, T.S. (2001). Cognitive and motivational effects of seeking academic assistance. The Journal of Educational Research, 94(6), 323-334.
Moos, R.H., and Moos, B.S. (1978). Classroom social climate and student absences and grades. Journal of Educational Psychology, 70(2), 263-269.
Murnane, R.J., and Steele, J.L. (2007). What is the problem? The challenge of providing effective teachers for all children. The Future of Children, 17(1), 15-43.
New Hampshire Department of Education (2002). Majority of Seniors Satisfied with High School Experience and Planning on Pursuing a Postsecondary Education: Results of 2002 New Hampshire High School Senior Pilot Survey Released. Concord, NH: New Hampshire Department of Education.
Newmann, F.M., and Wehlage, G.G. (1995). Successful School Restructuring: A Report to the Public and Educators. Madison, WI: Center on Organization and Restructuring of Schools, Wisconsin Center for Education Research.
Norton, S.M. (1998). Teacher absenteeism: A growing dilemma in education. Contemporary Education, 69(2), 95-99.
Pricola, J. (2001). Hooking new teachers. Teacher Magazine, 12(8), 12.
Purkey, S.C., and Smith, M.S. (1983). Effective schools: A review. Elementary School Journal, 83(4), 427-52.
Queen, J.A. (2000). Block scheduling revisited. Phi Delta Kappan, 82(3), 214-222.
Ready, D.D., Lee, V.E., and Welner, K.G. (2004). Educational equity and school structure: School size, overcrowding, and schools-within-schools. Teachers College Record, 106(10), 1989-2014.
Roby, D.E. (2004). Research on school attendance and student achievement: A study of Ohio schools. Educational Research Quarterly, 28(1), 3-14.
Rodriguez-Campos, L., Rincones-Gomez, R., and Shen, J. (2005). Secondary principals’ educational attainment, experience, and professional development in the USA. International Journal of Leadership in Education, 8(4), 309-319.
Rumberger, R.W., and Thomas, S.L. (2000). The distribution of dropout and turnover rates among urban and suburban high schools. Sociology of Education, 71, 39-67.
Schneider, M., Teske P., and Marschall, M. (2000). Choosing Schools. Princeton, NJ: Princeton University Press.
Sebring, P.A., and Camburn, E.M. (1992). How Teachers are Engaging Reform in Chicago: Differences among Schools. San Francisco, CA: American Educational Research Association. (ERIC Document Reproduction Service No. ED349713).
Shorr, A.A., and Hon, J.E. (1999). They said it couldn’t be done: Implementing a career academy program for a diverse high school population. Journal of Education for Students Placed at Risk, 4(4), 379-391.
Smith, J.B. (2001). Effects of eighth grade transition programs on high school retention and experiences. The Journal of Educational Research, 90(3), 144-152.
Sorensen, A.B. (1970). Organizational differentiation of students and educational opportunity. Sociology of Education, 43(4), 355-376.
Tomlinson, C.A. (2001). How to Differentiate Instruction in Mixed-Ability Classrooms (2nd ed). Alexandria, VA: Association for Supervision and Curriculum Development.
Zahorik, J.A. (1999). Reducing class size leads to individualized instruction. Educational Leadership, 57(1), 50-53.
Teacher Survey: Mathematics Teacher Survey, Science Teacher Survey
The teacher surveys complement the student survey by providing school context information and data about the opportunities and resources available to support student achievement. Key questions that can be answered by the teacher component are as follows:
What do math and science teachers do in the classroom that engages and encourages students to pursue STEM pathways, or alternatively, disengages and discourages students from choosing STEM pathways?
Interactions with students
Approaches to teaching (not exact practices)
Expectations, efficacy, beliefs about student potential
Personal background/history (e.g., reasons for entering profession, knowledge and comfort with math/science)
How do math and science teachers perceive the quality and supply of the resources and support they have available to teach effectively?
Induction practices (transition from career training to career)
Textbooks (usage linked to HSLS sample member’s classroom), manipulatives, supplies
Curriculum specialists
Current teacher assignment (e.g., how many different classes the teacher is instructing); how many planning periods does the teacher have (if any); what is the average class size (does he/she feel the classes are too big?).
The Teacher Questionnaires (mathematics and science) collect information on teachers in five areas: teacher background, teacher attitudes/beliefs, instructional policies/programs, textbook use, and school/departmental climate. Data gathered from the teacher questionnaire can be merged with data from the student assessment and survey. This linkage of the data will allow researchers to use the teacher data contextually with the student as the primary unit of analysis, and include linkages between the teacher’s/student’s specific classroom and the textbook materials used in each ninth-grader’s mathematics and science courses.
Justifications for the HSLS:09 Teacher Questionnaire
Questions 1–8. There have been a number of studies that have found positive relationships between student achievement and both teacher educational background and certification status, particularly in mathematics (Wayne & Youngs, 2003). Questions 1–5 ask about teachers’ educational background. Questions 1–3 come from the 2007–08 Schools and Staffing Survey (SASS) with very slight modifications to ask (a) about undergraduate minors and (b) the specific field of science in which teachers with undergraduate science degrees majored. Questions 4 and 5 ask about mathematics/science content courses taken by teachers and are newly developed for this study. Questions 6–8 ascertain teacher certification status and come from the SASS 2007-08 questionnaire.
For Question 1b, an easily accessible menu of options for respondent identification of postsecondary institution can be made available in the web version of the teacher questionnaire (and can be employed by the CATI interviewer as well). Coding to IPEDS would be automatic in the electronic version.
Do you have a bachelor’s degree?
a. In what year did you receive your bachelor’s degree?
b. What is the name of the college or university where you earned this degree?
In what city and state is it located?
c. Was this degree awarded by a college/university’s school or department of education?
d. What was your major field of study?
e. Did you have a second major or minor field of study?
f. What was your second major or minor field of study?
Do you have a master’s degree?
a. In what year did you receive your master’s degree?
b. Was this degree awarded by a university’s school or department of education?
c. What was your major field of study?
Have you earned any of the degrees or certificates listed below?
a. Vocational certificate
b. Associate’s degree
c. SECOND Bachelor’s degree
d. SECOND Master’s degree
e. Educational specialist or professional diploma (at least one year beyond master’s level)
f. Certificate of Advanced Graduate Studies
g. Doctorate or first professional degree
Which of the following college courses have you completed? (Mark all that apply)
Mathematics version:
a. Calculus
b. Abstract algebra
c. Linear algebra
d. Non-Euclidean geometry
e. Probability and statistics
f. Discrete or finite mathematics
g. Other upper division mathematics
Science version:
a. Chemistry
b. Earth/Space Sciences
c. Environmental Sciences
d. Life Sciences
e. Physics
f. Other
How many college mathematics/science courses have you completed?
Which of the following describes the teaching certificate you currently hold in THIS state?
a. Regular or standard state certificate or advanced professional certificate
b. Certificate issued after completion of all requirements except a probationary period
c. Certificate that requires some additional coursework, student teaching, or test score before regular certification can be obtained
d. Certificate issues to persons who must complete a certification program in order to continue teaching
e. I do not hold any of the above certifications in THIS state (Skip to question 8)
In which grades does this certificate allow you to teach mathematics in THIS state? (Select all that apply) (Response options: individual grades, K–12).
Did you enter teaching through an alternative certifications program? (Yes, No)
Questions 9–11. These questions measure teacher gender, race, and ethnicity. All questions in this section come from the 2007–08 SASS, without modifications. Studies conducted by Dee (2005) indicate that the racial, ethnic, and gender dynamics between students and teachers have consistently large effects on teacher perceptions of student performance.
Are you male or female?
Are you of Hispanic or Latino origin?
What is your race? (Mark all that apply)
a. White
b. Black/African American
c. Asian
d. Native Hawaiian or Other Pacific Islander
e. American Indian or Alaska Native
Questions 12–15. A positive relationship has been found between teacher experience and both teacher effectiveness and student learning; for example, one study found that teachers with less than 3 years’ experience are less effective than more senior teachers (Rosenholtz, 1986). More recent studies of the relationship between teacher experience and student achievement have been conducted by Greenwald, Hedges, and Laine (1996) and Rivkin, Hanushek, and Kain (2005). Questions 12 and 13 come from the ELS:2002 teacher questionnaire, without modification. Question 14 is a new item collecting data about the teacher experience in the current school. Question 15, asking about retirement status, comes from 2007–08 SASS, without modification.
Including this year,
a. How many years in total have you taught at the elementary level (K–6)?
b. How many years in total have you taught in the secondary level (7–12)?
c. How many years have you taught any grades (K–12)?
Counting this year, how many years have you taught mathematics/science at the secondary level? (Fill in)
Counting this year, how many years in total have you taught in this school? (Fill in)
Are you currently collecting a pension from a teacher retirement system or drawing money from a school/system sponsored 401(k) or 403(b) plan which includes funds you contributed as a teacher? (Yes, No)
Questions 16–18. Research has shown that understanding the belief structures of teachers is essential to improving their professional preparation and teaching practices (Ashton, 1990). Researchers have repeatedly related teacher efficacy to a variety of positive teaching behaviors and student outcomes (Woolfolk & Hoy 1990, 1993). Previous research has also found that teacher efficacy predicts teachers’ implementation of innovative programs and student achievement (Ross. 1992). Good and Brophy (1984) found that teacher expectations affect students’ self-concepts, levels of aspiration, and ultimately their achievement and other outcomes.
This section asks teachers about their attitudes and beliefs about effective teaching. Question 16 comprises a widely researched teacher efficacy scale: the Ohio State Teacher Efficacy Scale-Short Form developed by Hoy and Woolfolk (1993) which in past use has had a Cronbach’s alpha reliability of 0.90. Question 18 measures teacher expectations using items developed by the American Institutes for Research and SRI International (alpha = 0.90).
Indicate the extent to which you agree or disagree with each of the following statements as it applies to your mathematics/science instruction: (Strongly agree, Agree, Disagree, Strongly disagree)
a. The amount a student can learn is primarily related to family background.
b. If students aren’t disciplined at home, they aren’t likely to accept any discipline.
c. A teacher is very limited in what he/she can achieve because a student’s home environment is a large influence on his/her achievement.
d. If parents would do more for their children, I could do more for my students.
e. If a student did not remember information I gave in a previous lesson, I would know how to increase his/her retention in the next lesson.
f. If a student in my class becomes disruptive and noisy, I feel assured that I know some techniques to redirect him/her quickly.
g. If one of my students couldn’t do a class assignment, I could accurately assess whether the assignment was at the correct level of difficulty.
h. If I really try hard, I can get through to even the most difficult or unmotivated students.
i. When it comes right down to it, a teacher really can’t do much because most of a student’s motivation and performance depends on his or her home environment.
How would you compare boys and girls in...? (Girls are much better, Girls are somewhat better, Girls and boys are the same, Boys are somewhat better, Boys are much better)
a. Reading
b. Math
c. Writing
d. Science
Indicate the extent to which you agree or disagree with each of the following statements about mathematics/science teachers at this school: “Mathematics/science teachers in this school:
(Strongly Agree, Agree, Disagree, Strongly Disagree)
a. Set high standards for teaching
b. Set high standards for students’ learning
c. Believe all students can do well
d. Explain learning expectations to students
e. Give up on some students
f. Care only about smart students
g. Expect very little from students
h. Work hard to make sure all students are learning
Questions 19–25. Instructional materials often provide a variety of exposure to content, such as lab activities. In an analysis of widely used textbooks, Project 2061 found that textbooks vary greatly in the opportunity to learn they provide students (Kesidou & Roseman, 2002). Schmidt and colleagues (2001) found a statistically significant relationship between achievement gain in the subject area and many textbook variables including extent and depth of textbook coverage.
Questions 19–21 seek to understand the role textbooks play in teachers’ instructional practices, and ultimately student educational experiences. Question 19 is a newly developed item asking teachers to identify the course for which they are providing information about their textbook. Question 20 is also a newly developed item, asking teachers to identify the textbook/program primarily used in the course.
Questions 21 and 22 are drawn from the Reys’ Middle School Mathematics Survey and measure how much of the textbook teachers plan to cover and the extent to which teachers use the textbook, as well as other instructional materials, as the basis of instruction. Question 23 is a new item meant to assess the appropriateness of the difficulty level of the primary textbook/program for students in the class. Question 24 is also drawn from the Reys’ survey and probes teachers’ regarding their anticipated textbook usage. Question 25 is newly developed for this survey and asks about the extent to which teachers base their summative assessments on items provided with the textbook versus other sources.
It should be noted that the textbook information will be linked, via specific class, to the specific HSLS sample members. This same model was used in collecting textbook information class-by-class in the NELS:88 first follow-up (see Ingels et al., 1992). HSLS student’s enrollment will generate a list of math or science classes associated with each teacher. An extensive course title list is requested in Question 19 to ensure that the class will link with the high school transcript as well, when transcripts are collected after the first follow-up. Questions 19–25 will be repeated for each class in which the teacher has at least one HSLS student. Typically the textbook series will be answered between one and five times.
Which of the following best describes the title of this mathematics/science course?
Mathematics version:
a. Algebra I
b. Algebra IA
c. Algebra IB
d. Algebra II
e. Calculus
f. Calculus Prep
g. Calculus AP
h. Discrete Math
i. Geometry
j. Integrated Math I
k. Integrated Math II
l. Integrated Math III
m. Integrated Math IV
n. Probability/Statistics
o. Probability/Statistics AP
p. Review/Remedial Math
q. Other Math Course (please specify)_______________
Science version:
a. Anatomy/Physiology
b. Biology I
c. Biology II
d. Biology AP
e. Chemistry I
f. Chemistry II
g. Chemistry AP
h. Earth Science
i. Environmental Science
j. Integrated Science I
k. Integrated Science II
l. Integrated Science III
m. Integrated Science IV
n. Physical Science
o. Physics I
p. Physics II
q. Physics AP
r. Other Science Course (please specify)______________________
What textbook/program is primarily used in this class? [publisher, title, edition] (fill in)
Approximately what percentage of the designated textbook/program do you plan to cover this school year? (fill in percent)
What percentage of the instructional time in this class do you anticipate will be based on each of the following: (fill in percent)
a. Primary mathematics/science textbook/program
b. Other textbook/program
c. Other commercially available instructional materials
d. Materials obtained at conferences/conventions
e. Materials created by you
f. A graphing calculator
g. Other (please specify) [field test only] format as list
Thinking of a typical student in this class, which of the following best describes the difficulty level of the primary textbook/program? (Mark one)
(It is much too easy, It is somewhat too easy, It is at the appropriate level, It is somewhat too difficult, It is much too difficult)
How often do you anticipate the designated mathematics/science textbook/program will be used in each of the following ways: (Never, Rarely, Sometimes, Often, Always)
a. I will follow the textbook/program page by page.
b. I will pick what is important from the textbook/program and skip the rest.
c. I will follow my district’s curriculum recommendations regardless of what is in the textbook/program.
d. The textbook/program will guide the structure (content emphasis) of my course.
e. I will incorporate activities from other sources to supplement what the textbook/program is lacking.
f. I will read and review suggestions in the textbook’s/program’s teacher guide to plan my lessons.
g. I will use the student textbook/program to plan my lessons.
h. I will assign homework from the textbook/program.
i. My students will use their textbook/program during the mathematics/science lesson.
j. My students will use their textbook/program for homework assignments.
Assessment material used in determining student grades can come from any of several sources. For this class, what percentage will be:
a. Drawn from the primary textbook/program
b. Drawn from other commercially available material
c. Developed by the school/district
d. Developed by you
e. Other (please specify) [field test only]
Question 26. Much has been published about the inequitable assignment of teachers, particularly that the least qualified teachers are assigned to those students with the greatest educational needs. Recently, the Tennessee Department of Education provided teacher effectiveness data that indicated students in high poverty/high minority schools have less access to the “most effective” teachers and more access to the “least effective” teachers than students in low poverty/low minority schools (Tower, 2007). This question, newly developed for this survey, seeks to gain insight into the course assignment policies in schools.
To what extent do you agree or disagree with each of the following statements about how high school mathematics/science teaching assignments are made in this school? (Mark all that apply)
a. Advanced courses are assigned to teachers with the most seniority.
b. Advanced courses are assigned to teachers with the strongest mathematics/science background.
c. All or most mathematics/science teachers are assigned at least one section of advanced courses.
d. Non-college prep courses are assigned to teachers new to the profession.
e. Non-college prep courses are assigned to teachers whose students don’t perform well on standardized tests.
f. All or most mathematics/science teachers are assigned at least one section of a non-college prep course.
Question 27 (Mathematics Teacher Questionnaire Only). An implementation study by Biesinger and Crippen (2008) shows remediation assisted students in significantly outperforming those who did not use the program and virtually eliminated the performance gaps between white students and minority students. This question on the quality of remediation services available to students struggling with math comes from the National Math Panel Survey of Algebra Teachers, without modification.
How do you rate the remedial help in your school for grades 9-12 students who are struggling in Algebra I? [Poor, Fair, Good, Excellent]
Availability of tutoring or other remedial assistance
Quality of tutoring or other remedial assistance
Questions 28–29 (Mathematics) & 27-28 (Science). In their synthesis of research, Druian and Butler (1987) found effective schools exhibit equity in terms of learning, where learning takes place in a safe, orderly environment. A review of studies examining how availability of resources for instruction affects student performance (Hedges, Laine, & Greenwald, 1994) found that the majority of studies found a positive correlation. More recently, Archibald (2006) found that resource availability was positively related to student achievement, although this study focused on reading achievement (Archibald, 2006). Research from Apple Computer, Inc. (2002) demonstrated that access to technology increases student engagement and provides a broader range of resources, which can motivate students to continue their education in college. These questions gather information on school climate and factors that may negatively affect instruction. The questions are from the 2007–08 SASS and the 2003 Trends in International Mathematics and Science Study (TIMSS) teacher questionnaire, both without modification.
28/27. To what extent is each of the following a problem in this school? (Not applicable, Not at all, a little, some, a lot)
a. Student tardiness
b. Student absenteeism
c. Student truancy
d. Teacher absenteeism
e. Students dropping out
f. Student apathy
g. Lack of parental involvement
h. Poverty
i. Students coming to school unprepared to learn
j. Poor student health
k. Lack of resources and materials for teachers
29/28. In your view, to what extent do the following limit how you teach? (Not applicable, Not at all, a little, some, a lot)
a. Students with different academic abilities in the same class
b. Students who come from a wide range of socioeconomic backgrounds
c. Students who come from a wide range of language backgrounds
d. Students with special needs (e.g., hearing, vision, speech impairment, physical disabilities, mental or emotional/psychological impairment)
e. Uninterested students
f. Low morale among students
g. Disruptive students
h. Shortage of computer hardware/software
i. Shortage of support for using computers
j. Shortage of textbooks for student use
k. Shortage of other instructional equipment for student use
l. Shortage of equipment for teacher use in demonstrations and other exercises
m. Inadequate physical facilities
n. High student/teacher ratio
o. Lack of planning time
p. Lack of autonomy in instructional decisions
q. Lack of parent/family support
r. Inadequate opportunities for professional development
s. Inadequate administrative support
Question 30 (Mathematics) and 29 (Science). Charlotte Advocates for Education (2004) found that principals who are committed to providing professional development opportunities, support, and resources are key to the success of the school. Research has also found that schools in which the principal has a clear vision for the school and focuses the school on achieving that vision show improvements in student achievement (Gordon, 2003). This question from NELS:88 teacher questionnaire ascertains teachers’ opinions of principal support.
30/29. Please indicate the extent to which you agree or disagree with each of the following statements:
a. The principal deals effectively with pressures from outside the school that might interfere with my teaching.
b. The principal does a poor job of getting resources for this school.
c. The principal sets priorities, makes plans, and sees that they are carried out.
d. The principal knows what kind of school he/she wants and has communicated it to the staff.
e. The principal lets staff members know what is expected of them.
f. The principal is interested in innovation and new ideas.
g. The principal usually consults with staff members before he/she makes decisions that affect us.
Question 31 (Mathematics) and 30 (Science). Studies suggest that well-developed professional learning communities have positive impacts on both teaching practice and student achievement (Vescio, Ross, & Adams, 2008).This question is designed to assess the extent to which a professional learning community exists within the mathematics/science department. The core of this question comes from the Bay Area School Reform Collaborative teacher survey developed by researchers at Stanford University (alpha = 0.89).
31/30. To what extent do you agree or disagree with each of the following statements about the mathematics/science department in this school. (Strongly disagree, disagree, agree, strongly agree)
a. Math/science teachers share ideas on teaching
b. Math/science teachers discuss what was learned at a workshop or conference
c. Math/science teachers share and discuss student work
d. Math/science teachers discuss particular lessons that were not very
successful
e. Math/science teachers in this department discuss beliefs about teaching and learning
f. Math/science teachers in this department share and discuss research on effective teaching methods
g. Math/science teachers in this department share and discuss research on effective instructional practices for English language learners
h. Math/science teachers in this department explore new teaching approaches for underperforming students
i. Math/science teachers in this department make a conscious effort to coordinate the content of courses with other teachers in the school
j. Math/science teachers in this department are effective at teaching students math/science
k. Math/science teachers in this department provide support to new math/science teachers
l. The math/science department’s chair or curricular area coordinator’s behavior toward the staff is supportive and encouraging
Question 32 (Mathematics) and 31 (Science). Lee and Smith (1996) found very consistent results showing achievement gains were significantly higher in schools where teachers take collective responsibility for students’ academic success. This question focuses on collective responsibility and is from the teacher survey developed by the Consortium on Chicago School Research (alpha = 0.92).
32/31. Indicate the extent to which you agree or disagree (examples of response options) with each of the following statements: (Strongly disagree, disagree, agree, strongly agree)
Teachers in this school help maintain discipline in the entire school, not just their classroom.
Teachers in this school take responsibility for improving the school.
Teachers in this school set high standards for their teaching.
Teachers in this school feel responsible for helping students develop self-control.
Teachers in this school feel responsible to help each other do their best.
Teachers in this school feel responsible that all students learn.
Teachers in this school feel responsible when students in the school fail.
>> END OF TEACHER QUESTIONNAIRE <<
References
Apple Computers, Inc. (2002). The impact of technology on student achievement: A summary of research findings on technology’s impact in the classroom. Retrieved February 20, 2008, from http://images.apple.com/education/research/pdf/EduResearchFSv2.pdf.
Archibald, S. (2006). Narrowing in on educational resources that do affect student achievement. Peabody Journal of Education, 81(4), 23-42.
Ashton, P. T. (1990). Editorial. Journal of Teacher Education, 44(1), 2.
Biesinger, K., and Crippen, K. (2008). The impact of a state-funded online remediation site on performance related to high school mathematics proficiency. Journal of Computers in Mathematics and Science Teaching, 27(1), 5-17.
Charlotte Advocates for Education (2004). Role of principal leadership in increasing teacher retention: Creating a supportive environment. The Advocate. Retrieved February 18, 2008, from http://www.advocatesfored.org/publications/Principal%20Final%20Report.pdf.
Dee, T. (2005). A teacher like me: Does race, ethnicity, or gender matter? The American Economic Review, 95(2), 158-166.
Druian, G., and Butler, J. (1987). Effective schooling practices and at-risk youth: What the research shows. NW Regional Educational Laboratory. Retrieved February 19, 2008, from http://www.nwrel.org/scpd/sirs/1/topsyn1.html.
Good, T.L., and Brophy, J.E. (1984). Teaching expectations. Chapter 4. Looking in Classrooms. New York: Harper & Row, pp. 93-121.
Gordon, G. (2003). Do principals make a difference? Unpublished manuscript.
Greenwald, R., Hedges, L.V., and Laine, R.D. (1996). The effect of school resources on student achievement. Review of Educational Research, 66(3): 361-396.
Hedges, L.V., Laine, R.D., and Greenwald, R. (1994). Does money matter? A meta-analysis of studies of the effects of differential school inputs on student outcomes. Educational Researcher, 23(3), 5-14.
Ingels, S.J., Scott, L.A., Lindmark, Judith T., Frankel, Martin R, and Myers, S.L. (1992). National Education Longitudinal Study of 1988 First Follow-Up: Teacher Component Data File User’s Manual. (NCES 93-085). Washington, DC: National Center for Education Statistics.
Kesidou, S., and Roseman, J.E. (2002). How well do middle school science programs measure up? Findings from project 2061’s curriculum review. Journal of Research in Science Teaching, 39(6), 522-549.
Lee, V., and Smith, J. (1996). Collective responsibility for learning and its effects on gains in achievement for early secondary school students. American Journal of Education, 104(2), 103-147.
Rivkin, S.G., Hanushek, E.A., and Kain, J.F. (2005). Teachers, schools and academic achievement. Econometrica, 73(2), 417-458.
Ross, J.A. (1992). Teacher efficacy and the effect of coaching on student achievement. Canadian Journal of Education, 17, 51-65.
Rosenholtz, S.J. (1986). The organizational context of teaching. Learning to Teach. Champaign-Urbana, IL: University of Illinois.
Schmidt, W.H., McKnight, C.C., Houang, R.T., Wang, H.C., Wiley, D.E., Cogan, L.S., et al. (2001). Why Schools Matter: A Cross-National Comparison of Curriculum and Learning. San Francisco: Jossey-Bass.
Tower, A. (2007). Tennessee’s most effective teachers: Are they assigned to the most schools that need them the most? Retrieved February 20, 2008, from http://tennessee.gov/education/nclb/doc/TeacherEffectiveness2007_03.pdf.
Vescio, V., Ross, D., and Adams, A. (2008). A review of research on the impact of professional learning communities on teaching practice and student learning. Teaching and Teacher Education: An International Journal of Research and Studies, 24(1), 80-91.
Wayne, A., and Youngs, P. (2003). Teacher characteristics and student achievement gains: A review. Review of Educational Research, 73(1), 89-122.
Woolfolk, A.E., and Hoy, W.K. (1990). Prospective teachers’ sense of efficacy and beliefs about control. Journal of Educational Psychology, 82, 81-91.
Woolfolk, A.E., and Hoy, W.K. (1993). Teachers’ sense of efficacy and the organizational health of schools. The Elementary School Journal, 93(4), 355-372.
Counselor Survey
The counselor component is targeted to the head counselor or whomever the head counselor designates as a knowledgeable source about the questionnaire contents. HSLS:09 is not a study of counselors and cannot generalize about counselors as a special population, but rather, a study that employs counselor data contextually to illuminate characteristics and practices of the school, particularly those related to student placement in mathematics and science, and the availability and role of counseling services vis-à-vis the transition into high school and out of high school. Key research questions that the counselor survey may help to address include the following:
How do students get placed into and out of classes?
What counseling resources are available to the students within the school (e.g., how many counselors; what is their student load; what do they do, what are their responsibilities (such as course placement, college planning, career planning; transitions from middle school, from high school to postsecondary)?
What are the tracking procedures and policies and graduation requirements (e.g., how many credits/courses in English, in math, in science, etc.)?
What college preparation is in place in the school (e.g., AP classes/AP exams, preparation for SATs)?
The School Counselor Questionnaire collects information on the school in six areas: (1) counseling services provided; (2) postsecondary counseling; (3) course placement policies; (4) school-based remediation and enrichment services offered (with a focus on STEM); (5) out-of-school learning experiences/opportunities; and (6) school climate. Data gathered from the counselor questionnaire can be merged with data from the student assessment and survey. This linkage of the data will allow researchers to determine to what degree disparities in educational aspirations, expectations, and outcomes of various student populations are accounted for by differences in their educational experiences. Questions selected for the counselor questionnaire were reviewed by the HSLS:09 Technical Review Panel in its November 2007 and January 2008 meetings.
Part I of the Counselor Questionnaire collects information on the counseling services available at the school, including the number of full-time and part-time counselors; certification of those counselors, and goals of the counseling program. This section also includes questions on the frequency of counselor/student interactions.
Part II focuses on services the school offers to assist students with the transition from high school to postsecondary education or work.
Part III of the Counselor Questionnaire addresses the procedures used to place students into their ninth-grade mathematics and science courses. In addition, this section includes questions about requirements for entry into advanced mathematics and science courses.
Part IV emphasizes alternative educational opportunities available for students during both the academic year and in summer, including programs/policies designed to encourage students in STEM, and whether the school offers vocational-technical programs. Also of interest are interventions and services available for both at-risk students and advanced/gifted students.
Part V addresses student opportunities to take STEM courses not offered at school.
Justifications for the HSLS:09 Counselor Questionnaire
Questions 1–11. This section of the Counselor Questionnaire collects information on the counseling services available at the school, including the number of full-time and part-time counselors, certification of those counselors, and goals of the counseling program. This section also includes items on the frequency of counselor/student interactions.
Questions 1 and 2 gather information on the number and certification status of counselors at the school; these questions come from the 2001 Fast Response Survey System (FRSS) on High School Guidance Counseling, without modification. Questions 3–5 gather background information on the respondent: years of counseling experience, educational background, and pathway into counseling. These questions were newly developed for this survey.
Question 6 comes from the High School and Beyond Guidance Questionnaire and was used again on the 2001 FRSS survey; it has not been modified for this study. This item examines the goals of the counseling program at a school. Question 7 is new and asks about the nature of counselor-student interactions; Question 8 was taken from the HS&B questionnaire and asks about the types of counseling activities offered at the school. According to the American School Counseling Association, school counselors ought to provide counseling programs in three domains: academic, career, and personal/social. Reviews of research on school counseling indicate that the services that school counselors provide have positive effects on students (Brigman & Campbell, 2003; Mosconi & Emmett, 2003).
Questions 9 and 10 are new. Question 9 asks about the nature of teacher-counselor communications regarding students. Question 10 asks about whether students are required to have a high school graduation plan and whether the plans can be modified over time. Lee and Ekstrom (1987) found support for the hypothesis that students who planned their high school program with a school counselor were more likely to be placed on the academic track and as a result of this placement they were also more likely to take advanced mathematics courses. The authors went on to conclude that counseling appeared to be an important conduit toward increased coursetaking.
Questions 11a-c ask about how students are assigned to counselors, average number assigned, and whether the counseling group addresses college application and work transitions, all central issues for studying youth transition into and out of high school.
Indicate the number of full-time and part-time counselors assigned to high school (9–12) students at your school.
2. Of those assigned, indicate the number of counselors that are certified as high school (9–12) counselors.
3. Please indicate how long you have been a:
a. school counselor for any grades K–12
b. school counselor for any grades 9–12
4. Do you have an undergraduate or graduate degree in: (Yes/No)
a. psychology
b. school counseling
c. education
d. social science
e. physical or biological science
5. Which of the following best describes your entry into the counseling profession? (Select one):
I become a school counselor immediately after college
a. I was first a teacher, prior to becoming a school counselor
b. I was in another education-related profession, prior to becoming a school counselor
c. I was another type of counselor
d. I was in a noneducation-related profession, prior to becoming a school counselor
e. Other, please specify: (for field test only); I was a counselor in a noneducation-related profession.
6. Please rank the extent to which the following goals are currently emphasized by the counseling program in your school (“1” indicates the goal with the most emphasis, “2” indicates the second most emphasis, and so on through “4” for the goal with the least emphasis.)
a. Help students plan and prepare for their work roles after high school
b. Help students with personal growth and development
c. Help students plan and prepare for postsecondary schooling
d. Help students improve their achievement in high school
7. Which of the following describe how school counselors meet with students to discuss their performance and/or plans for the future? (select all that apply)
a. Counselors are required to meet with students at the beginning of the school year
b. Counselors are required to meet with students at least once every term (e.g.,
semester, trimester)
c. Counselors are required to meet with students at the end of the year
d. Counselors in this school maintain an open door policy and students can meet
with their counselors as often as they like
e. Counselors request to meet with students as needed
f. Other, please specify: (for field test only).
8. Which of the following activities are offered to students in this school? (select all that apply)
a. School courses in career decision making
b. Occupational information units in subject-matter courses
c. Exploratory work experience programs (e.g., co-op, workstudy, EBCE)
d. Career days/nights
e. Vocational oriented assemblies and speakers in classes
f. Job site visits (field trips)
g. School arranged tours of postsecondary institutions
h. Job shadowing (extended observations of a worker)
i. Simulations (e.g., Singer, SRA Job experience kits)
j. Administering and interpreting tests for career planning purposes (e.g., interest inventories, vocational aptitude tests)
k. Group counseling sessions
l. Training in job seeking skills
m. Use of noncomputerized career information resources
n. Use of computerized career information resources
o. Access to college catalogs
9. Does the school’s counseling staff consult with teachers regarding students’: (Yes/No)
a. Future course placement
b. Mid-year course changes
c. Remediation/tutoring needs
d. Discipline
e. Participation in enrichment programs
f. College preparation
10a. Are students in your school required to have a high school graduation plan? (Yes/No)
10b. If Yes, can this plan be modified or updated throughout students’ high school years? (Yes/No)
11a. Which of the following best describes how counselors are assigned to students at this school? (Mark one)
a. Counselors are assigned to a specific grade level (e.g., a 9th grade counselor)
b. Counselors are assigned to an incoming class of 9th-graders and remain with them throughout their high school years (e.g., a counselor for the class of 2009)
c. Counselors are assigned to a group of students whose last names fall within a slice of the alphabet (e.g., all students with last names from “A to D”)
d. Other (please specify) ________________
11b. On average, how many students are assigned to a counselor at this school? [verbatim]
11c. Does your school have one or more counselors whose primary responsibility is: [Yes/No]
Assisting students with college readiness, selection, and applications?
Assisting students with preparation for and placement into the workforce?
Questions 12–13. Research has found that school counselors’ services are very effective in assisting students with career development. In a study of senior high school career planning, Witko and colleagues (2005) found that high school students valued career planning services, but wanted more specific career information and support to help them make more informed decisions. When students were asked about how their current education was preparing them for postsecondary contexts that required more education, Lapan et al. (2003) found that increased career development activities predicted greater student satisfaction with this aspect of their education.
The questions in this section focus on services the school offers to assist students with the transition from high school to postsecondary education or work. Questions 12 and 13 are new for this study.
12a. Does your school’s professional counseling staff assist with transitioning 8th-grade students into high school? (Yes/If No – skip next item)
12b.In which of the following ways does your school’s professional counseling staff assist with transitioning 8th-grade students into high school? [select all that apply]
a. Presenting information to 8th-grade students and/or parents/guardians about high school courses and registration
b. Assisting individual 8th-grade students with selecting 9th-grade courses based upon their interests and prior achievement
c. Placing 8th-grade students into 9th-grade courses based on school/district placement policies
12c. What practices does the school engage in to assist students with the transition from high school to college? (select all that apply)
a. Holding/participating in college fairs
b. Consulting with postsecondary school representatives about requirements and qualifications sought
c. Encouraging students to visit colleges
d. Offering special programs that help students plan or prepare for college, such as Upward Bound, college scholarships
e. Other (please specify):__________ [field test only]
13. How does the school assist students with the transition from high school to work? (Mark all that apply)
a. Offering internships with local employers
b. Holding/participating in job fairs
c. Arranging school/classroom presentations by local employers
d. Offering career awareness activities
e. Other (please specify):__________ [field test only]
Questions 14–17. Nelson et al. (1998) found that students in schools with higher levels of guidance were more likely to take advanced courses in mathematics and science in comparison with other students. Gamoran (1987) found that the number of advanced high school courses taken is a strong predictor of achievement in both science and mathematics.
This section addresses the procedures used to place students into their ninth-grade mathematics and science courses and requirements for entry into advanced mathematics and science courses. Questions 14–15 are new and inquire about what factors influence counselor recommendations for student course placement. Questions 16–17 are also new and ask about school requirements for entry into advanced mathematics and science courses.
14. For a typical student, which of the following influence his/her placement into 9th grade mathematics/science?
a. Recommended by middle school counselor.
b. Recommended by middle school mathematics/science teacher.
c. Based on courses taken/achievement in middle school courses
d. Recommended by high school counselor.
e. Based on results of placement tests.
f. Selected by student and/or parent/guardian.
g. Other (please specify):__________ [field test only]
15. Which of the following typically factor into counselor recommendations for mathematics/science courses for students entering 10th, 11th, and 12th grade? [two parts, one to respond for math, one for science]
a. Prior grades
b. Placement tests
c. Ninth-grade mathematics/science teacher recommendation
d. Student and/or parent/guardian preference
e. Master schedule considerations
f. Other (please specify):__________ [field test only]
16. Does your school have a mathematics pre-requisite for entry into advanced science courses (e.g., chemistry, physics)? (Yes/No)
17. If yes: In which of the following ways can a student not meeting this prerequisite enroll in the course? [select all that apply]
a. Teacher approval
b. Counselor approval
c. Principal approval
d. Parental request for waiver
e. There is no way the student can enroll in the course
f. Other (please specify):__________ [field test only]
Questions 18–23. With regard to STEM, Dunham and Frome (2003) found that students who received more guidance and encouragement from counselors to take more advanced courses in mathematics and science took more college-preparatory courses in those areas. Kyburg, Hertberg-Davis, and Callahan (2007) found that the more scaffolding schools provided to support and challenge students via activities such as extracurricular help, lunchtime discussion forums, and subsidized college visits, the greater the growth of academic talent, including among students of diverse backgrounds. This study also found that school environments that tended to be flexible and attuned to the varying and individual needs of culturally, linguistically, socially, and economically diverse students were ones in which learners were more likely to experience success. In addition, Borders and Drury (1992) found that counseling interventions tend to have substantial impact on students’ educational and personal development.
This section emphasizes alternative educational opportunities available for students during both the academic year and in summer, including programs/policies designed to encourage students in STEM, and whether the school offers vocational-technical programs. Also of interest are interventions and services available for both at-risk students and advanced/gifted students. All of the questions are new except for two; question 18 comes from the ELS:2002 Base-Year School Administrator Survey and question 22 comes from the Study for Instructional Improvement School Characteristics Inventory.
18. Does your school offer summer school enrichment courses that allow students to get ahead (e.g., a geometry class that would allow a student taking algebra in 9th grade to take calculus in the 12th grade)? (Yes/No)
19. If a student fails a state or district mathematics competency assessment, which of the following options are available to the student at the school and which are required of the student? [mark all that apply; one column to check available, one column to check required]
a. Retaking the test
b. Taking remedial classes in deficient subject areas
c. Repeating classes in deficient subject areas
d. Completing a general competency test preparation class
e. Tutoring
f. Individualized academic program
g. Summer school
h. Referral to an alternative or continuing education school
20. Does your school have any programs to:
a. Encourage students who might not be considering college to consider college.
b. Encourage underrepresented students to pursue mathematics/science.
c. Inform parents/guardians about mathematics/science higher education and/or career opportunities.
21. Is there a vocational-technical program offered at your school? (Yes/No)
22. Which of the following steps does this school take for students who need extra assistance in mathematics/science? [ask separately for math and science] (Mark all that apply)
a. Tutoring is available to low achieving students during the regular school day
b. Instructional aides or specialists work in classrooms to provide assistance to low-achieving students
c. Instructional aides provide low-achieving students with pull-out instruction during the regular school day
d. Additional support is provided to low-achieving students outside the regular school day (e.g., before- or after-school tutoring or special programs, summer school programs)
e. Other (please specify): _________ [field test only]
23. Which of the following are available in this school to support and encourage gifted students in mathematics and science? (Mark all that apply)
a. Technology and software to support curriculum specifically to meet the needs of the gifted students.
b. Instructional aides or specialists work in classrooms to provide enrichment to high-achieving students.
c. Enrichment experiences such as Odyssey of the Mind, Science Olympiad, Academic Decathlon.
d. Scholarships for gifted students to attend special events or classes.
e. Summer activities/programs appropriate for gifted students.
f. Other (please specify): _________ [field test only]
Question 24. Studies show that students in small schools, particularly those in rural areas, are less likely to have access to advanced courses in science and mathematics (U.S. Department of Education, 2005). Further, even if a school does offer advanced courses, scheduling conflicts and oversubscription to these courses can limit student opportunities to enroll in these courses (WestEd, 2007). Thus, nontraditional venues such as Internet-based courses offer students expanded access to advanced materials. This question is new and addresses student opportunities to take STEM courses not offered at school.
24. Which of the following options are available for students to take science, technology, engineering, or mathematics courses not offered by your school? (Mark all that apply)
a. Independent study
b. Online courses
c. Courses at another traditional high school in the district
d. Courses at a local career/technical school
e. Courses at a local community college
f. Courses at a nearby 4-year college/university
g. Other (please specify): _________ [field test only]
Questions 25–27. In a review of the research on the relationship between expectations and student outcomes, Cotton (1989) found that expectations that are communicated school-wide can and do affect student achievement and attitudes. Furthermore, this review indicates that high expectations are a critical component of effective schools, especially when these expectations are communicated through policies and practices which focus on academic goals.
This section focuses on school climate, in particular the expectations teachers, counselors, and the principal have for students in the school. All three questions were taken from the American Institutes for Research/SRI International Gates Foundation Evaluation survey.
25. Indicate the extent to which you agree or disagree with each of the following statements: (Strongly Agree, Agree, Disagree, Strongly Disagree)
a. Teachers in this school set high standards for teaching
b. Teachers in this school set high standards for students’ learning
c. Teachers in this school believe all students can do well
d. Teachers in this school have given up on some students
e. Teachers in this school care only about smart students
f. Teachers in this school expect very little from students
g. Teachers in this school work hard to make sure all students are learning
Note on Question 25: there is a parallel question on the teacher questionnaires, with two minor differences: (a) the TQ asks about mathematics (or science) teachers in the school specifically while the CQ asks more generally about teachers in the school; and (b) we dropped the sub-item “teachers make expectations for instructional goals clear to students” as we thought counselors would not be able to answer accurately (more so than the other sub-questions).
26. Indicate the extent to which you agree or disagree with each of the following statements: (Strongly Agree, Agree, Disagree, Strongly Disagree)
a. Counselors in this school set high standards for students’ learning
b. Counselors in this school believe all students can do well
c. Counselors in this school have given up on some students
d. Counselors in this school care only about smart students
e. Counselors in this school expect very little from students
f. Counselors in this school work hard to make sure all students are learning
27. Indicate the extent to which you agree or disagree with each of the following statements: (Strongly Agree, Agree, Disagree, Strongly Disagree)
a. The principal in this school sets high standards for students’ learning
b. The principal in this school believes all students can do well
c. The principal in this school has given up on some students
d. The principal in this school cares only about smart students
e. The principal in this school expects very little from students
f. The principal in this school works hard to make sure all students are learning
>> END OF COUNSELOR QUESTIONNAIRE <<
Administrative Records
In addition to the mathematics assessment and six questionnaires, information will be collected in a brief administrative records component. Given the importance of eighth-grade algebra and other pre-high school curricular experience to the transition into high school and future placement and coursetaking sequences, a major objective of the base year should be to obtain information about eighth-grade science and mathematics coursework and grades. The administrative records supplement seeks to do this by requesting eighth-grade science and math course title and letter or numerical grade from the high school in which HSLS sample members are now enrolled. We also ask this information on the student questionnaire, so that we can compare sources for the completeness and accuracy (using school report as validity criterion) of the data. If it is clear that one source will suffice for both accuracy and completeness, then this can be taken into account for the base year full-scale study design.
References
Borders, L.D., and Drury, S.M. (1992). Comprehensive school counseling programs: A review for policymakers and practitioners. Journal of Counseling and Development, 70: 487-498.
Brigman, G., and Campbell, C. (2003). Helping students improve academic achievement and school success behavior. Professional School Counseling, 7(2): 91-98.
Cotton, K. (1989). Expectations and student outcomes. NW Regional Educational Laboratory. Retrieved February 19 2008, from http://www.nwrel.org/scpd/sirs/4/cu7.html.
Dunham, C., and Frome, P. (2003). Guidance and Advisement: Influences on Students’ Motivation and Course-Taking Choices.
Gamoran, A. (1987). The stratification of high school learning opportunities. Sociology of Education, 60: 135-155.
Kyburg, R.M., Hertberg-Davis, H., and Callahan, C.M. (2007). Advanced placement and international baccalaureate programs: Optimal learning environments for talented minorities? Journal of Advanced Academies, 18(2): 172-215.
Lapan, R.T., Tucker, B., Kim, S., and Kosciulek, J. . (2003). Preparing rural adolescents for post-high school transitions. Journal of Counseling & Development, 81: 329-342.
Lee, V. L., and Ekstrom, R.B. (1987). Student access to guidance counseling in high school. American Education Research Journal, 24: 287-310.
Mosconi, J., and Emmett, J. (2003). Effects of values clarification curriculum on high school students’ definitions of success. Professional School Counseling, 7(2): 68-77.
Nelson, D.E., Fox, D.G., and Gardner, J.L. (1998). Study II: Contrasts Between Students in High Implementation and Low Implementation High Schools in the Utah Comprehensive Guidance Program. Salt Lake City, UT: The Institute for Behavioral Research in Creativity.
U.S. Department of Education, National Center for Education Statistics (2005). The Condition of Education 2005, NCES 2005-094, Washington, DC: U.S. Government Printing Office.
WestEd (2007). Connecting students to advanced courses online: Innovations in education. U.S. Department of Education, Office of Innovation and Improvement. Washington, DC: U.S. Government Printing Office.
Witko, K., Bernes, K.B., Magnusson, K., and Bardick, A.D. (2005). Senior high school career planning: What students want. Journal of Educational Enquiry, 6(1): 34-49.
3.Use of Improved Information Technology and Burden Reduction
For the first time in the series of NCES longitudinal studies, all questionnaire data will be collected in electronic media only. In addition, the student assessment will also be a computer-assisted two-stage adaptive test. For the student component, the school’s computer lab will be used when available, and, as a backup, multiple laptops will be supplied for use by the sampled students. A trained session administrator will be present to assist students with computer issues as needed.
School administrators, teachers, and parents will be given a username and password and will be asked to complete the questionnaire via the Internet. Follow-up for school administrators, teachers, and parents who do not complete the web questionnaire by self-administration will be in the form of computer-assisted telephone interviewing (CATI). Computer control of interviewing offers accurate and efficient management of survey activities, including case management, scheduling of calls, generation of reports on sample disposition, data quality monitoring, interviewer performance, and flow of information between telephone and field operations.
Additional features of the system include (1) online help for each screen to assist interviewers in question administration; (2) full documentation of all instrument components, including variable ranges, formats, record layouts, labels, question wording, and flow logic; (3) capability for creating and processing hierarchical data structures to eliminate data redundancy and conserve computer resources; (4) a scheduler system to manage the flow and assignment of cases to interviewers by time zone, case status, appointment information, and prior cases disposition; (5) an integrated case-level control system to track the status of each sample member across the various data collection activities; (6) automatic audit file creation and timed backup to ensure that, if an interview is terminated prematurely and later restarted, all data entered during the earlier portion of the interview can be retrieved; and (7) a screen library containing the survey instrument as displayed to the interviewer.
4.Efforts to Identify Duplication and Use of Similar Information
Since the inception of its secondary education longitudinal studies program in 1970, NCES has consulted with other federal offices to ensure that the data collected in this important series of longitudinal studies do not duplicate the information from any other national data sources within the U.S. Department of Education or other government agencies. In addition, NCES staff have regularly consulted with nonfederal associations such as the College Board, American Educational Research Association, the American Association of Community Colleges, and other groups to confirm that the data to be collected through this study series are not available from any other sources. These consultations also provided, and continue to provide through the HSLS:09 Technical Review Panel, methodological insights from the results of other studies of secondary and postsecondary students and labor force members, and they ensure that the data collected through HSLS:09 will meet the needs of the federal government and other interested agencies and organizations.
Other longitudinal studies of secondary and postsecondary students (i.e., NLS:72, HS&B, NELS:88, ELS:2002) have been sponsored by NCES in the past. HSLS:09 builds on and extends these studies rather than duplicating them. These earlier studies were conducted during the 1970s, 1980s, 1990s, and the early 2000s and represent educational, employment, and social experiences and environments different from those experienced by the HSLS:09 student sample. In addition to extending prior studies temporally as a time series, HSLS:09 will extend them conceptually. The historical studies do not fully provide the data that are necessary to understand the role of different factors in the development of student commitment to attend higher education and then to take the steps necessary to succeed in college (take the right courses, take courses in specific sequences, etc.). Using items and inventories, the study will enable researchers to move beyond the traditional covariates to ask, “How do students and parents construct their choice set?” Further, HSLS:09 will focus on the factors associated with choosing and persisting in mathematics and science coursetaking and STEM careers. These focal points present a marked difference between HSLS:09 and its predecessor studies.
The only other dataset that offers so large an opportunity to understand the key transitions into postsecondary institutions and/or the world of work is the Department of Labor (Bureau of Labor Statistics) longitudinal cohorts, the National Longitudinal Survey of Youth 1979 and 1997 cohorts (NLSY79, NLSY97). Clearly, however, the NLSY youth cohorts represent temporally earlier cohorts than HSLS:09. There are also important design differences between NLSY79/ NLSY97 and HSLS:09 that render them more complementary than duplicative. NLSY is a household-based longitudinal survey; HSLS:09 is school based. For both NLSY cohorts, baseline Armed Service Vocational Aptitude Battery (ASVAB) test data are available, but there is no longitudinal high school achievement measure. While NLSY97 also gathers information from schools (including principal and teacher reports and high school transcripts), it cannot study school processes in the same way as HSLS:09, given its household sampling basis. Any given school contains only one to a handful of NLSY97 sample members, a number that constitutes neither a representative sample of students in the school nor a sufficient number to provide within-school estimates. Thus, although both studies provide important information for understanding the transition from high school to the labor market, HSLS:09 is uniquely able to provide information about educational processes and within-school dynamics and how these affect both school achievement and ultimate labor market outcomes, including outcomes in science, technology, engineering, and mathematics education and occupations.
5.Impact on Small Businesses or Other Small Entities
This section has limited applicability to the proposed data collection effort. Target respondents for HSLS:09 are individuals (typically nested within an institutional context) of public and private schools; base-year data collection activities will involve no burden to small businesses or entities.
6.Consequences of Collecting the Information Less Frequently
This submission describes the field test and full-scale data collection for the base year of HSLS:09. Base-year data collection will take place in the fall of 2009, preceded by a field test in 2008. First follow-up data collection will take place 2½ years later, in the spring term of 2012, with a field test in 2011. The initial out-of-school follow-up is tentatively scheduled for 3 years thereafter.
The rationale for conducting HSLS:09 is based on a historical national need for information on academic and social growth, school and work transitions, and family formation. In particular, recent education and social welfare reform initiatives, changes in federal policy concerning postsecondary student support, and other interventions necessitate frequent studies. Repeated surveys are also necessary because of rapid changes in the secondary and postsecondary educational environments and the world of work. Indeed, longitudinal information provides better measures of the effects of program, policy, and environmental changes than would multiple cross-sectional studies.
To address this need, NCES began the National Longitudinal Studies Program more than 35 years ago with the National Longitudinal Study of 1972 (NLS:72). This study collected a wide variety of data on students’ family background, schools attended, labor force participation, family formation, and job satisfaction at five data collection points through 1986. NLS:72 was followed approximately 10 years later by High School and Beyond (HS&B), a longitudinal study of two high school cohorts (10th- and 12th-grade students). The National Education Longitudinal Study of 1988 (NELS:88) followed an eighth-grade cohort, which, upon completion in 2000, reflected a modal respondent age of about 26 years. The Education Longitudinal Study of 2002 (ELS:2002) followed a 10th-grade cohort and allows for the availability of a 32-year trend line.
The scheduled student follow-ups of HSLS:09 are less frequent than the 2-year interval employed with HS&B, NELS:88, and ELS:2002. The first follow-up takes place at 2½ years after the base year, and the second follow-up 3 years after the first follow-up. However, parent data may be collected at grade 12, and a high school transcripts study to be conducted soon after graduation will provide continuous coursetaking data for the cohort’s high school careers for all on-time or early completers. The initial data collection occurs at the start of the students’ high school careers and will allow researchers to understand decisionmaking processes as they pertain to the selection of STEM-related courses. By following up at the end of the students’ junior year, researchers will be able to measure achievement gain as well as postsecondary planning information. Collecting parent and transcript information in the 12th grade will minimize burden on schools and respondents, while also allowing for further intercohort comparability with the main transition themes of the prior studies. The second follow-up is scheduled to occur in the second year after high school, which is on track with the timing of the predecessor studies, thus facilitating comparisons in the domain of postsecondary access and choice. Despite the changes in grade cohorts and data collection time points for the first two rounds, general trends will still be measurable, since the same key transitions, albeit with slightly different data collection points, will be captured with the HSLS:09 data.
Probably the most cost-efficient and least burdensome method for obtaining continuous data on student careers through the high school years comes through the avenue of collecting school records. In most cases, transcript data are more accurate than self-report data as well. High school transcripts were collected for a subsample of the HS&B sophomore cohort, as well as for the entire NELS:88 cohort retained in the study after eighth grade and the entire ELS:2002 sophomore and senior cohorts. The collection of administrative records will take place at the onset of HSLS:09 to identify coursetaking behaviors in grades 8 and 9, and a full transcript study is tentatively scheduled to take place after high school graduation.
7.Special Circumstances Relating to Guidelines of 5 CFR 1320.5
All data collection guidelines in 5 CFR 1320.5 are being followed. No special circumstances of data collection are anticipated.
8.Consultations Outside NCES
Consultations with persons and organizations both internal and external to the National Center for Education Statistics, the U.S. Department of Education (ED), and the federal government have been pursued. In the planning stage for HSLS:09, there were many efforts to obtain critical review and to acquire comments regarding project plans and interim and final products. We are in the process of convening the Technical Review Panel, which becomes the major vehicle through which future consultation is achieved in the course of the project. Consultants outside ED and members of the Technical Review Panel include the following individuals:
Technical Review Panel
Dr. Clifford Adelman
The Institute for Higher Education Policy
1320 19th Street, NW, Suite 400
Washington, DC 20036
Phone: (202) 861-8223 ext. 228
Fax: (202) 861-9307
E-mail: [email protected]
Dr. Kathy Borman
Department of Anthropology, SOC 107
University of South Florida
4202 Fowler Avenue
Tampa, FL 33620
Phone: (813) 974-9058
E-mail: [email protected]
Dr. Daryl E. Chubin
Director
Center for Advancing Science & Engineering Capacity
American Association for the Advancement of Science (AAAS)
1200 New York Avenue, NW
Washington, DC 20005
Dr. Jeremy Finn
State University of New York at Buffalo
Graduate School of Education
409 Baldy Hall
Buffalo, NY 14260
Phone: (716) 645-2484
E-mail: [email protected]
Dr. Thomas Hoffer
NORC
1155 E. 60th Street
Chicago, IL 60637
Phone: (773) 256-6097
E-mail: [email protected]
Dr. Vinetta Jones
Howard University
525 Bryant Street NW
Academic Support Building
Washington DC 20059
Phone: (202) 806-7340 or (301) 395-5335
E-mail: [email protected]
Dr. Donald Rock
Before 10/15: K11 Shirley Lane
Trenton NJ 08648
Phone: 609-896-2659
After 10/15: 9357 Blind Pass Rd, #503
St. Pete Beach, FL 33706
Phone: (727) 363-3717
E-mail: [email protected]
Dr. James Rosenbaum
Institute for Policy Research
Education and Social Policy
Annenberg Hall 110 EV2610
Evanston, IL 60204
Phone: (847) 491-3795
E-mail: [email protected]
Dr. Russ Rumberger
Gevirtz Graduate School of Education
University of California
Santa Barbara, CA 93106
Phone: (805) 893-3385
E-mail: [email protected]
Dr. Philip Sadler
Harvard-Smithsonian Center
for Astrophysics
60
Garden St.,
MS 71
Office D-315
Cambridge, MA 02138.
Phone: (617) 496-4709
Fax: (617) 496-5405
E-mail: [email protected]
Dr. Sharon Senk
Department of Mathematics
Division of Science and Mathematics Education
D320 Wells Hall
East Lansing, MI 48824
Phone: (517) 353-4691 (office)
E‑mail: [email protected]
Dr. Timothy Urdan
Santa Clara University
Department of Psychology
500 El Camino Real
Santa Clara, CA 95053
Phone: (408) 554-4495
Fax: (408) 554-5241
E-mail: [email protected]
Other Consultants Outside ED
Dr. Eric Bettinger
Associate Professor, Economics
Case Western Reserve University
Weatherhead School of Management
10900 Euclid Avenue
Cleveland, OH 44106
Phone: (216) 386-2184
E-mail: [email protected]
Dr. Audrey Champagne
Professor Emerita
University of Albany
Educational Theory and Practice
Education 119
1400 Washington Avenue
Albany, NY 12222
Phone: (518) 442-5982
Dr. Stefanie DeLuca
Assistant Professor
Johns Hopkins University
School of Arts and Sciences
Department of Sociology
532 Mergenthaler Hall
3400 North Charles Street
Baltimore, MD 21218
Phone: (410) 516-7629
E-mail: [email protected]
Dr. Laura Hamilton
RAND Corporation
4570 Fifth Avenue, Suite 600
Pittsburgh, PA 15213
Phone: (412) 683-2300 ext. 4403
E‑mail: [email protected]
Dr. Jacqueline King
Director for Policy Analysis
Division of Programs and Analysis
American Council for Education
Center for Policy Analysis
One Dupont Circle, NW
Washington, DC, 20036
Phone: (202) 939-9551
Fax: 202-785-2990
E-mail: j[email protected]
Dr. Joanna Kulikowich
Professor of Education
The Pennsylvania State University
232 CEDAR Building
University Park, PA 16802-3108
Phone: (814) 863-2261
E‑mail: [email protected]
Dr. Daniel McCaffrey
RAND Corporation
4570 Fifth Avenue, Suite 600
Pittsburgh, PA 15213
Phone: (412) 683-2300 ext. 4919
E-mail: [email protected]
Dr. Jeylan Mortimer
University of Minnesota - Dept of Sociology
909 Social Sciences Building
267 19th Avenue South
Minneapolis, MN 55455
Room 1014a Social Sciences
Phone: (612) 624-4064
E-mail: [email protected]
Dr. Aaron Pallas
Teachers College
Columbia University
New York, NY 10027
Phone: (646) 228-7414
E-mail: [email protected]
Ms. Senta Raizen
Director
WestEd
National Center For Improving Science Education
1840 Wilson Blvd., Suite 201A
Arlington, VA 22201-3000
Phone: (703) 875-0496
Fax: (703) 875-0479
E-mail: [email protected]
9.Explanation of Any Payment or Gift to Respondents
Table 1 shows the incentive structure, by respondent type, requested for HSLS:09. In some cases, incentive experiments have been proposed to determine the effectiveness of the incentive on response rates. A description and rationale for each incentive or incentive experiment is provided below.
Table 1. Incentives by respondent type proposed for field test
Respondent |
Experiment? |
Incentive |
School |
Yes |
$500 technology allowance vs. $0 |
Student (in-school administration) |
No |
$10 cash |
School coordinator |
No |
$100 cash base honorarium; up to $150 cash for high student response |
Math or science teacher |
No |
$25 (base amount) plus $5 per class reported on (the number of class reports required will vary from teacher to teacher) |
School counselor |
No |
No incentive |
School administrator |
No |
No incentive |
Parents |
No |
No incentive |
NOTE: CATI = computer-assisted telephone interviewing.
Incentives for schools. Securing the cooperation of schools to participate in voluntary research has become increasingly difficult. Experience has been showing that many schools already feel burdened by mandated “high stakes” testing and, at the same time, are hampered by fiscal and staffing constraints. Moreover, roadblocks will occur not only at the school, but also at the district level, where research studies must sometimes comply with stringent requirements to submit formal and detailed applications similar to those one would submit to an IRB before individual schools can even be contacted. A successful incentive program can greatly reduce labor costs associated with school recruitment and refusal conversion efforts.
An experiment will be designed and executed for the field test to determine if offering a monetary technology allowance would be a level of incentive that would encourage schools to participate in HSLS:09. Laptop computers were considered as an incentive, but this idea was ruled out after considering drawbacks such as compatibility, usefulness of equipment, and security issues involved with transferring laptops containing confidential information to school staff. An incentive experiment was proposed at the school level for the field test to help offset some of the challenges associated with obtaining school cooperation. This experiment would compare the effect of a $500 technology allowance against no incentive. All schools within a given district would receive the same incentive. The technology allowance would be in the form of a check written to the school that can be used at the school’s discretion, although it is recommended that it be used toward technology for the school to align with the focus of the study.
Incentives for students. All students participating in the HSLS:09 field test will receive a $10 incentive. The use of a monetary student incentive was granted for the field test specifically because of the number of cases required to validate the psychometric properties of the math assessment items. In addition, the field test includes both ninth-grade and twelfth-grade students, and research has demonstrated the importance of incentivizing high school seniors to participate in voluntary research studies. To be equitable, all participating students will receive the $10 incentive in the field test.
F
or
the full scale study, NCES proposes to offer participating
ninth-grade students an education-related token incentive that is
less than $5 in cost. Based on the popularity of drawstring
backpacks among ninth-grade students, NCES proposes to offer
educational goodie bags to participating students. Each
participating student will receive a drawstring backpack (a sample
is shown in the photo on the right) filled with educational
materials such as pencils, a ruler, a notebook, etc. The total cost
of the bags plus their contents will be $5 or less.
A flyer will be included in the parental consent materials to notify the parent and students about the educational goodie bag that will be offered to participating students. However, students will not be told exactly what will be inside the bag. It is expected that the popularity of the bag itself, plus the added mystery surrounding the contents inside the bag, will motivate students to participate in HSLS.
There is a precedent for using a token incentive to participants of similar ages within this type of survey administration. ELS used a key light keychain as a token incentive for the sophomore cohort in 2002, and also implemented a drawing for two $20 Barnes and Noble gift cards in schools requiring active consent. This student incentive structure resulted in a response rate of 87.3%.
The strict data security requirements for the study prohibit the study name from being imprinted on any of the token materials. Instead, “Class of 2013” will be imprinted on any imprinted materials.
School coordinator honorarium. The school coordinator bears a heavy burden to ensure that data collection is successful in the school. The coordinator is expected to coordinate logistics with the data collection contractor, supply a list of eligible students for sampling to the data collection contractor, supply parent contacting information for sampled students, communicate with teachers about the study, distribute parental consent forms and reminder notices, coordinate the assignment of students to each session, assist the test administrator in ensuring the sampled students attend the testing session, assist the test administrator in arranging for follow-up sessions as needed, and distribute materials for the staff components of the study. All of these activities will occur under a tight timeline for HSLS:09 due to the fall data collection and the time at which the student list is ready at the school.
The school coordinator honorarium is based on the percentage of sampled students who participate in the study. The role of the school coordinator is critical for the success of the study. NCES planned for the same level of honorarium as was offered in PISA 2006 based on the similar timing and burden on the school coordinator. The school coordinator honorarium is planned at a base of $100 with up to an additional $50 for achieving high student response at the school. This is an increase from the honorarium offered to school coordinators in the 2004 round of ELS, which ranged from $50 as the base incentive to $100 for coordinators in schools that had high student response rates. NCES proposes increasing the level of these incentives to $100 and $150 respectively, to compensate for the additional work that is required to ensure that the study receives a complete list of ninth-graders as quickly as possible at the start of the school year as well as to compensate for the logistical burden of coordinating multiple sessions when required due to computer lab or laptop capacity in the school. This incentive is planned for the field test and main study.
Incentives for teachers. Math and science teachers will provide information on classroom attributes, teaching practices, and teaching experience. In addition, teachers will report on textbook use in each class they teach to HSLS:09 students. Past experience has demonstrated the need for a teacher-level incentive to achieve high response rates and many schools have required that teacher compensation be commensurate with their hourly wage. Thus, NCES has proposed a $25 base incentive for teachers plus an additional $5 for each additional classroom on which they report. This is proposed for both the field test and main study.
Incentive for counselors. No incentive is proposed for the counselors to complete their questionnaires. Counselors would typically provide the information requested in the questionnaire as well as the administrative records as part of their normal duties. Because of the nature of the study, NCES suspects that many school principals will designate a counselor to perform the school coordinator duties, in which case the counselor will receive the coordinator honorarium described above.
Incentive for school administrators. NCES has achieved high response rates for the school administrator questionnaire on ELS:2002, the ELS:2002 follow-up conducted in 2004, and in PISA:2006. Based on past experience, no incentive will be offered for the school administrator questionnaire on HSLS:09.
Incentives for parents. There is no precedent for offering an incentive to complete the parent questionnaire. Thus, no parent incentive is proposed for HSLS:09.
Reimbursement of reasonable school expenses. In some cases there may be requests from schools for reimbursement of expenses associated with the testing session (for example, keeping the school open for a special makeup testing session that occurs outside of normal school hours). Such cases will be reviewed by project staff on an individual basis and will be approved if the request is deemed reasonable.
10.Assurance of Confidentiality Provided to Respondents
A data security plan (DSP) for HSLS:09 was developed and deemed acceptable to Neil Russell and the computer security review board. The HSLS:09 plan will strengthen confidentiality protection and data security procedures developed for ELS:2002 and represents best-practice survey systems and procedures for protecting respondent confidentiality and securing survey data. An outline of this plan is provided in exhibit 1. The HSLS:09 DSP will
establish clear responsibility and accountability for data security and the protection of respondent confidentiality with corporate oversight to ensure adequate investment of resources;
detail a structured approach for considering and addressing risk at each step in the survey process and establish mechanisms for monitoring performance and adapting to new security concerns;
include technological and procedural solutions that mitigate risk and emphasize the necessary training to capitalize on these approaches; and
be supported by the implementation of data security controls recommended by the National Institute of Standards and Technology for protecting federal information systems.
Exhibit 1. HSLS:09 data security plan outline
HSLS:09 Data Security Plan Summary Maintaining the Data Security Plan Information Collection Request Our Promise to Secure Data and Protect Confidentiality Personally Identifying Information That We Collect and/or Manage Institutional Review Board Human Subject Protection Requirements Process for Addressing Survey Participant Concerns Computing System Summary General Description of the RTI Networks General Description of the Data Management, Data Collection, and Data Processing Systems Integrated Monitoring System Receipt Control System Instrument Development and Documentation System Data Collection System Document Archive and Data Library Employee-Level Controls Security Clearance Procedures Nondisclosure Affidavit Collection and Storage Security Awareness Training Staff Termination/Transfer Procedures Subcontractor Procedures |
Physical Environment Protections System Access Controls Survey Data Collection/Management Procedures Protecting Electronic Media Encryption Data Transmission Storage/Archival/Destruction Protecting Hard-Copy Media Internal Hard-Copy Communications External Communications to Respondents Handling of Mail Returns, Hard-Copy Student Lists, and Parental Consent Forms Handling and Transfer of Data Collection Materials Tracing Operations Software Security Controls Data File Development: Disclosure Avoidance Plan Data Security Monitoring Survey Protocol Monitoring System/Data Access Monitoring Protocol for Reporting Potential Breaches of Confidentiality Specific Procedures for Field Staff |
Under this plan, HSLS:09 will conform totally to federal privacy legislation, including
the Privacy Act of 1974 (5 U.S.C. 552a);
Section C of Education Sciences Reform Act of 2002 (P.L. 107-279);
the USA Patriot Act of 2001 (P.L. 107-56);
the Family Educational Rights and Privacy Act (FERPA) (20 U.S.C. 1232g; 34 CFR Part 99);
the Protection of Pupil Rights Amendment (PPRA) (20 U.S.C. § 1232h; 34 CFR Part 98);
the Freedom of Information Act (5 U.S.C. 552);
the Hawkins-Stafford Elementary and Secondary School Improvement Amendments of 1988 (P.L. 100-297);
Title IV of the Improving America’s Schools Act of 1994 (P.L. 103-382); and
the Office of Management and Budget Federal Statistical Confidentiality Order of 1997.
HSLS:09 also will conform to NCES Restricted Use Data Procedures Manual and NCES Standards and Policies. The plan for maintaining confidentiality includes obtaining signed confidentiality agreements and notarized nondisclosure affidavits from all personnel who will have access to individual identifiers. Each individual working in HSLS:09 will also complete the e-QIP clearance process. The plan also includes annual personnel training regarding the meaning of confidentiality and the procedures associated with maintaining confidentiality, particularly as it relates to handling requests for information and providing assurance to respondents about the protection of their responses. The training will also cover controlled and protected access to computer files under the control of a single database manager; built-in safeguards concerning status monitoring and receipt control systems; and a secured and operator-manned in-house computing facility.
Invitation letters will be sent to states, districts, and schools describing the voluntary nature of this survey. The material sent will include a brochure to describe the study and to convey the extent to which respondents and their responses will be kept confidential. (Materials are provided in appendix A.)
As NCES’s HSLS:09 contractor, all recruiting materials and procedures will be reviewed and approved by RTI’s Committee for the Protection of Human Subjects prior to sample selection. This committee serves as RTI’s Institutional Review Board (IRB) as required by 45 CFR 46. It is RTI policy that the all RTI research involving human subjects, regardless of funding source, undergoes IRB review in a manner consistent with the regulations in 45 CFR 46 to ensure that all such RTI studies comply with applicable regulations concerning informed consent, confidentiality, and protection of privacy.
11.Justification for Sensitive Questions
The data elements will be discussed in a separate submission to OMB.
12.Estimates of Annualized Burden Hours and Costs
Estimates of response burden for the HSLS:09 base-year field test and full-scale data collection activities are shown in tables 3 through 7. Because the proposed field test will be the first application of the proposed instrumentation, the estimates of response burden are based on initial estimates developed from experience with ELS:2002 and other educational longitudinal studies (e.g., NELS:88, HS&B). Please note that the time students will spend completing the cognitive assessment has not been included in the estimated burden. High school seniors will complete the assessment only and not the questionnaire; therefore, they are not represented in the burden estimate.
Table 3. Estimated burden on respondents for field test and full-scale studies
Respondents |
Sample |
Expected response rate |
Number of respondents |
Average burden/ response1 |
Range of response times |
Total burden (hours) |
Freshmen |
|
|
|
|
|
|
Field test (2008) |
1,595 |
87 |
1,387 |
30 minutes |
— |
694 |
Full-scale (2009) |
20,000 |
92 |
18,400 |
30 minutes |
— |
9,200 |
Total |
21,595 |
|
19,787 |
|
|
9,894 |
|
|
|
|
|
|
|
Seniors |
|
|
|
|
|
|
Field test (2008) |
1,650 |
85 |
1,402 |
5 minutes |
— |
117 |
Total |
1,650 |
|
1,402 |
|
— |
117 |
1 Please note that the time students will spend completing the cognitive assessments has not been included in the estimated burden.
Table 4. Estimated burden on parents for field test and full-scale studies
Parents |
Sample |
Expected response rate |
Number of respondents |
Average burden/response |
Range of response times |
Total burden (hours) |
Total |
21,250 |
|
19,550 |
|
|
6,842 |
|
|
|
|
|
|
|
Field test (2008) |
1,595 |
92 |
1,467 |
30 minutes |
30 |
734 |
Full-scale (2009) |
20,000 |
92 |
18,400 |
30 minutes |
30 |
9,200 |
Table 5. Estimated burden on math teachers for field test and full-scale studies
Math teachers (linked to students) |
Sample |
Expected response rate |
Number of respondents |
Average burden/response |
Range of response times |
Total burden (hours) |
Total |
10,295 |
|
9,471 |
|
|
6,314 |
|
|
|
|
|
|
|
Field test (2008) |
385 |
92 |
354 |
40 minutes |
30–50 minutes |
236 |
Full-scale (2009) |
5,600 |
92 |
5,152 |
40 minutes |
30–50 minutes |
3,435 |
Table 5a. Estimated burden on science teachers for field test and full-scale studies
Science teachers (linked to students) |
Sample |
Expected response rate |
Number of respondents |
Average burden/response |
Range of response times |
Total burden (hours) |
Total |
10,295 |
|
9,471 |
|
|
6,314 |
|
|
|
|
|
|
|
Field test (2008) |
330 |
92 |
304 |
40 minutes |
30–50 minutes |
203 |
Full-scale (2009) |
4,800 |
92 |
4,416 |
40 minutes |
30–50 minutes |
2,944 |
Table 6. Estimated burden on school administrators for field test and full-scale studies
School administrators |
Sample |
Expected response rate |
Number of respondents |
Average burden/response |
Range of response times |
Total burden (hours) |
Total |
850 |
|
832 |
|
|
420 |
|
|
|
|
|
|
|
Field test (2008) |
55 |
98 |
49 |
30 minutes |
— |
24.5 |
Full-scale (2009) |
800 |
98 |
784 |
30 minutes |
— |
392 |
Table 7. Estimated burden on school counselors for field test and full-scale studies
Counselors |
Sample |
Expected response rate |
Number of respondents |
Average burden/response |
Range of response times |
Total burden (hours) |
Total |
850 |
|
781 |
|
|
390.5 |
|
|
|
|
|
|
|
Field test (2008) |
55 |
92 |
46 |
30 minutes |
— |
23 |
Full-scale (2009) |
800 |
92 |
736 |
30 minutes |
— |
368 |
The cost to student participants is estimated at $6.55 per hour for the field test and $7.25 per hour for the main study. For freshmen, the cost is estimated at $4,546 for the field test and $66,700 for the main study. For seniors, who will participate only in the field test and complete only the cognitive assessment battery and a brief background questionnaire, the cost is estimated as $766.
For parents, assuming a $20 hourly wage, the cost to parent respondents is estimated to be $14,680 for the 2008 field test and $184,000 for the 2009 base-year main study.
For teachers (math and science teachers providing textbook information for each course taught), teacher burden is highly variable because teachers may have different numbers of classes to provide information for. An incremental incentive is proposed for this model based on success with a similar design in ELS:2002. ELS:2002 employed a linked design in which teachers reported on students, burden in the student ratings portion of the teacher questionnaire ranged from as few as 1 student (4 minutes student-rating burden) to as many as 25 students (100 minutes student-rating burden) in small schools where there was only a single teacher for a particular subject in the relevant (10th) grade. In HSLS:09, teachers will complete a 30-minute base questionnaire and the burden in the textbook portion will be 5 additional minutes per class, after the first, for an average of 40 minutes per teacher.
Slightly larger numbers of math teachers than science teachers across high schools are expected. Burden for math teachers is shown in table 5, and burden for science teachers is shown in table 5a. Sample sizes for the teacher sample are harder to predict with full accuracy than other sample sizes in HSLS, since the number is not preset for this component and some of the information needed to model probable sample sizes is not available from other national datasets. (Ideally, one would be able to tap comprehensive national statistics for how many science and mathematics teachers, in each school in a simulated stratified probability-proportionate-to-size [PPS] sample, were engaged in teaching ninth-graders.)
Costs to teacher respondents may be estimated as follows. Assuming an hourly wage of $20 for school personnel, field test respondent costs amount to $4,720 for math teachers and $4,060 for science teachers. Main study respondent costs for this component to $68,700 and $58,880 for math and science teachers, respectively. Under the ninth-grade math and science teacher option, the cost for this component increases slightly although exact cost estimates are not available yet. This option has not been accepted or approved yet, but is mentioned only to give complete and thorough context.
For school administrators (the greater part of the questionnaire is typically completed by clerical staff in the school office with the last section completed by the school principal), again assuming a $20 hourly cost, the cost to respondents is $490 in the field test and $7,840 in the main study.
For the counselor questionnaire, the respondent dollar cost, assuming an average hourly rate of $20 for school employees, is estimated to be $460 in the field test and $7,360 in the main study.
Included in the parent, teacher, school administrator, and counselor notification letters will be the following burden statement:
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number of this information collection is [1850-New], and it is completely voluntary. The time required to complete this information collection is estimated to average 30 minutes for the parent, teacher, and school administrator questionnaires, including the time to review instructions and complete and review the information collection. The student questionnaire will be no more than 35 minutes in length, and the math test will take about 40 minutes. If you have any comments concerning the accuracy of the time estimate or suggestions for improving the interview, please write to: U.S. Department of Education, Washington, DC 20202-4651. If you have comments or concerns regarding the status of your individual interview, write directly to: Dr. Laura LoGerfo, National Center for Education Statistics, 1990 K Street NW, Washington, DC 20006.
13.Estimates of Other Total Annual Cost Burden
There are no capital, startup, or operating costs to respondents for participation in the project. No equipment, printing, or postage charges will be incurred.
14.Annualized Cost to the Federal Government
Estimated costs to the federal government for HSLS:09 are shown in table 8. The estimated costs to the government for data collection for the field test and full-scale studies are presented separately. Included in the contract estimates are all staff time, reproduction, postage, and telephone costs associated with the management, data collection, analysis, and reporting for which clearance is requested.
Costs to NCES |
Amount |
Total HSLS:09 base-year costs |
$ 14,485,784 |
Salaries and expenses |
719,900 |
Contract costs |
15,205,684 |
|
|
Field test (2008) |
2,820,025 |
Salaries and expenses |
215,648 |
Contract costs |
3,035,673 |
|
|
Full-scale survey (2009) |
11,665,759 |
Salaries and expenses |
504,252 |
Contract costs |
12,170,011 |
NOTE: All costs quoted are exclusive of incentive fee. Field test costs represent Tasks 2 and 5 of the HSLS:09 contract; base-year main study costs include tasks 1, 3, 4, and 6.
15.Explanation for Program Changes or Adjustments
This is a new collection. This submission requests data collection approval for the field test and base year of HSLS:09. Thus, there is no precedent for the study in terms of a previously approved collection for which approval has expired.
16.Plans for Tabulation and Publication and Project Time Schedule
The HSLS:09 field test will be used to test and improve the instrumentation and associated procedures. Publications and other significant provisions of information relevant to the data collection effort will be a part of the reports resulting from the field test and main study, and both public use (Data Analysis System [DAS]) and restricted use (electronic codebook microdata) files will be important products resulting from the full-scale survey. The HSLS:09 data will be used by public and private organizations to produce analyses and reports covering a wide range of topics.
Data files will be made available to a variety of organizations and researchers, including offices and programs within the U.S. Department of Education, the Congressional Budget Office, the Department of Health and Human Services, the Department of Labor, the Department of Defense, the National Science Foundation, the American Council on Education, and a number of other education policy and research agencies and organizations. The HSLS:09 contract requires the following reports, publications, or other public information releases:
detailed methodological reports (one each for the field test and full-scale survey) describing all aspects of the data collection effort;
complete full-scale study data files and documentation for research data users;
a DAS for public access to HSLS:09 results;
an ECB for restricted access to HSLS:09 microdata; and
a “first look” summary of significant descriptive findings for dissemination to a broad audience (the analysis deliverable will include technical appendices).
Final deliverables are scheduled for completion by mid-2010.
The operational schedule for the HSLS:09 field test and full-scale study is presented in table 9.
17.Reason(s) Display of OMB Expiration Date Is Inappropriate
The expiration date for OMB approval of the information collection will be displayed on data collection instruments and materials. No special exception to this requirement is requested.
18.Exceptions to Certification for Paperwork Reduction Act Submissions
There are no exceptions to the certification statement identified in the Certification for Paperwork Reduction Act Submissions of OMB Form 83-I.
Activity |
Start |
End |
Field test |
|
|
School sampling |
2/2008 |
2/2008 |
Sample recruitment |
2/2008 |
11/2008 |
List receipt, student sampling |
8/2008 |
11/2008 |
Student/staff data collection |
9/2008 |
12/2008 |
Parent data collection |
10/2008 |
12/2008 |
Nonresponse follow-up |
10/2008 |
12/2008 |
|
|
|
Base year |
|
|
School sampling |
2/2008 |
2/2008 |
Sample recruitment |
2/2008 |
11/2009 |
List receipt, student sampling |
8/2009 |
11/2009 |
Student/staff data collection |
9/2009 |
11/2009 |
Parent data collection |
10/2009 |
2/2010 |
Nonresponse follow-up |
10/2009 |
3/2010 |
1 RTI International is a trade name of the Research Triangle Institute.
2 While sometimes different response formats are needed for questions that differ in subject and intent, in the main, it is preferable to have as much consistency of response format as possible, within a given questionnaire, to minimize the cognitive demand on the respondent. For this reason, questions generally use a Likert scale and on agree/disagree formats, questions follow NELS:88 and ELS:2002 by using a 4-point scale: strongly agree, agree, disagree, strongly disagree. A specific rationale for avoiding a neutral or “middle position” for agree-disagree questions is provided by Converse and Presser’s Sage monograph (Survey Questions: Handcrafting the Standardized Questionnaire) on how to write survey questions: “Do not explicitly provide the middle category, and thereby lose information about the direction in which some people lean.” However, while this is taken as the default position, an electronic administration also allows for assigning a subset of alternatively-scaled items to a random subset of survey completers so that comparisons of response formats can be undertaken in the field test analyses. This technological advantage is being exploited in the field test instrumentation to test and compare 4-point versus 5-point response options.
| File Type | application/msword |
| File Title | Chapter 2 |
| Author | spowell |
| Last Modified By | katrina.ingalls |
| File Modified | 2008-08-01 |
| File Created | 2008-08-01 |
© 2024 OMB.report | Privacy Policy