ELS 2002 3rd-Follow-up 2012 Field Test Parts B C D E

June 2011

Education Longitudinal Study: 2002
(ELS:2002)

Third Follow-up 2011 Field Test

OMB Supporting Statement

Parts B, C, D, and E

OMB# 1850-0652 v.7

National Center for Education Statistics

Institute of Education Sciences

U.S. Department of Education

TABLE OF CONTENTS

Part B: Statistical Sampling Design

Part C: Survey Item Justification

Part D: Linkages to Extant Data Sources

Part E: Field Test Response Propensity Modeling Experiment and Case Assignment

December 2010

Education Longitudinal Study: 2002
(ELS:2002)

Third Follow-up 2011 Field Test

OMB Supporting Statement

Part B

OMB# 1850-0652 v.7

National Center for Education Statistics

Institute of Education Sciences

U.S. Department of Education

TABLE OF CONTENTS

Section Page

B. Collection of Information Employing Statistical Methods B-1

B.1 Respondent Universe B-1

B.2 Statistical Procedure for Collecting Information B-2

B.2.a Third Follow-Up Field Test Sample Design B-2

B.2.b Third Follow-Up Main Study Sample Design B-2

B.3 Methods for Maximizing Response Rates B-2

B.4 Tests of Procedures and Methods B-6

B.5 Reviewing Statisticians and Individuals Responsible for Designing and Conducting the Study B-6

Exhibits

Number Page

Exhibit B-1. Third Follow-up Full-Scale Data Collection Schedule and Projected Response by Survey Mode 11

Exhibit B-2. RTI Consultants on Statistical Aspects of ELS:2002 B-7

Exhibit B-3. Other Contractor Staff Responsible for Conduct of ELS:2002 B-7

B.Collection of Information Employing Statistical Methods

The respondent universe for the ELS:2002 third follow-up field test and full-scale study data collection is described in the first section below. The second section describes the sampling and statistical methodology proposed for the field test and main study. The other sections describe methods for maximizing response rates, the special tests of procedures and methods, and the statisticians and other persons responsible for designing and conducting the study.

B.1 Respondent Universe

The target populations of the third follow-up field test study are the 2001 high school sophomore and the 2003 high school senior class. The sophomore cohort comprises students who were enrolled in the 10th grade in spring term 2001, and the 12th-grade cohort comprises seniors in the spring term of 2003. The sophomore cohort population includes high school dropouts.

The target populations of the third follow-up full-scale study are the 2002 sophomore cohort and the 2004 senior cohort. The sophomore cohort consists of those students who were enrolled in the 10th grade in the spring of 2002, and the 12th-grade cohort consists of those students who were enrolled in the 12th grade in the spring of 2004. The sophomore cohort includes students who dropped out of school between 10th and 12th grade. Response rates through the second follow-up were high enough to expect large returns in the third follow up of the study’s primary groups of interest: dropouts from high school, non-college bound high school graduates, and college bound high school graduates (i.e., those who enrolled in college at time of last contact). The overall response rate was 89%. Response rates for respondents who had ever reported a dropout episode (~1200 cases) were about 83% for the second follow-up. Non-college bound students also make up a large part of the sample; about 30% of second follow-up respondents reported never enrolling in postsecondary education as of 2006.

For both field test and full scale, the sampling frame for the third follow-up consists of students who were enrolled in the 10th grade in the base-year study, or students who were enrolled in the 12th grade in the first follow-up study, and participated, at minimum, in either the base year or first follow-up rounds. The sampling frame will exclude students who are deceased or were consistently (across rounds) physically or mentally incapable of participation. Post-tracing, the questionnaire will only be fielded to sample members who remain “within scope” and will exclude those who are out of scope. Those who will be considered out of scope for the third follow-up round include all those who are found to be: out of country, institutionalized, incarcerated, or “newly” incapable (having suffered a major cognitive or physical impairment). However, such sample members may be in-scope for a fourth follow-up, should a subsequent round be funded. (For example, an expatriate sample member might repatriate, and thus become in-scope in the future.) Sample members who have asked that their data be withdrawn from the study are eligible members of the sample frame, but will be treated as permanent nonrespondents and not fielded. Such eligible but nonfielded cases will be accounted for in the weighting and count against the response rate.

B.2 Statistical Procedure for Collecting Information

B.2.a Third Follow-Up Field Test Sample Design

The ELS:2002 third follow-up field test sample will comprise respondents in the base year and/or first follow-up field test who may or may not also have been second follow-up field test respondents. The ELS:2002 second follow-up field test sample members were initially selected for the sample either in the base year when they were 10th graders in 2001, or they were added to the sample as freshened students when they were in the 12th grade in 2003.

The size of the field test sample (the subset of cases actively to be pursued for data collection) is 1,060. For purposes of cost containment, while collecting sufficient observations to evaluate questionnaire performance, the target yield for the first follow-up sample is approximately 500 individuals.

B.2.b Third Follow-Up Main Study Sample Design

The ELS:2002 third follow-up full-scale sample will consist of approximately 16,200 sample members who were sophomores in 2002 or seniors in 2004 or both. The procedures that will be employed will target an overall response rate of 90 percent.

B.3 Methods for Maximizing Response Rates

Our plan to maximize response rates focuses on two related goals of the ELS:2002 third follow-up field test and full-scale data collection: (1) successful locating of sample members, and (2) a successful data collection.

The first goal is successfully locating and contacting sample members by successfully implementing a tracing and sample maintenance plan. A successful locating effort is dependent on a multitude of factors including the characteristics of the population, the age of the locating information for the population, and the completeness and accuracy of that information. The locator database for the cohort includes critical tracing information for most of the sample members, including their previous residences and telephone numbers. Moreover, Social Security numbers are available for the majority of the sample members (89 percent of the field test sample members, 91 percent of the full scale sample members).

The tracing and sample maintenance plan includes, initially, the use of batch tracing services without direct respondent contact. Subsequently, direct mailings to sample members or their parents will take place. Batch tracing is a relatively low-cost method of updating addresses and telephone numbers for sampled individuals. The two primary batch tracing services include National Change of Address (NCOA) and Phone Append. NCOA will provide updated addresses for sample members, especially those who have recently moved. Phone Append will confirm or update the telephone number matched to each sample member at their most current known address. Cases whose contact information is not confirmed or updated by these steps will be sent for Accurint batch tracing. Using the updated address information obtained from these steps, the batch tracing activity will be followed with a direct mailing to sample members and their parents.

The inclusion of parents is an important support to the tracing and contacting goals of the study. Parents have been deeply involved in the study since its onset: parent permission was sought for student participation in the base year and first follow-up, and a parent survey was conducted in the base year. Parents have also previously been contacted for tracing and locating purposes. The age of the sample population makes it very likely that many of the sample members have recently gone through major life transitions. For students who entered postsecondary education (74 percent at the second follow-up), they are now more than 5 years past their initial entry point into postsecondary education and most students who persisted in their education are likely to have completed their certificate, associate’s degree, or bachelor’s degree programs. Sample members are unlikely to still be attending the undergraduate institution they were attending at the time of the second follow-up. Many of the full-scale students are now 23 or 24 years of age (and will be 25 or 26 years of age at the time of data collection) and have entered the workforce and started their careers. Some students have married and changed names, making the task of updating their locating information somewhat more difficult. Another consideration is that 38 percent of adults aged 18 to 24 and 46 percent of adults aged 25 to 29 live in wireless telephone-only households,¹ creating a greater locating challenge. The majority of the ELS sample will fall in the former age group as we begin sample maintenance and the latter group as we begin data collection. For these reasons, we expect sample members’ parents to be a useful source for updating contact information for their children. Because parents’ residences will likely be more stable than their children’s, they will be very important to the tracing effort for a significant number of sample members. For this reason, the batch tracing activities and the sample maintenance mailings described in this plan will include parents. Also, study contacting materials will be produced in English, Spanish, and, on an at-need basis, up to four Asian languages, to also facilitate contacts with parents whose first language is not English. Panel maintenance activities to facilitate sample locating were described fully in an earlier submission to OMB, that obtained permission to contact sample members or their parents and are currently ongoing.

We will also conduct intensive in-house tracing at RTI during data collection for cases we cannot locate. The goal of intensive tracing is to obtain a telephone number at which the sample member can be reached. Tracing procedures may include (1) Directory Assistance for telephone listings at various addresses, (2) criss-cross directories to identify (and contact) the neighbors of sample members, (3) calling persons with the same unusual surname in small towns or rural areas to see if they are related to or know the sample member, and (4) contacting the current or last known residential sources such as the neighbors, landlords, and current residents of the last known address. Other more intensive tracing activities could include (1) database checks for sample members, parents, and other contact persons, (2) credit database and insurance database searches, (3) drivers’ license searches through the appropriate state departments of motor vehicles, (4) calls to colleges, military establishments, and correctional facilities to follow up on leads generated from other sources, (5) calls to alumni offices and associations, and (6) calls to state trade and professional associations based on information about field of study in school and other leads. The same vendors that were mentioned in the approved panel maintenance submission to OMB will be utilized.

Communication with sample members prior to and during data collection will promote cooperation by offering multiple methods for them to contact us (see Appendix 6 for contacting materials). The lead letters will provide the study website where sample members can find more information about the study and participating (by completing the questionnaire) via the Web. The letters will also provide telephone numbers respondents can use to contact RTI staff. Follow-up telephone calls will also be used to determine whether sample members have in fact received the materials we will have mailed to them or visited the study website. Contact materials and the web site have been designed so as to be attractive and informative in a manner that will enhance participation in ELS:2002.

For those with Internet access, the first opportunity will be to complete a self-administered Web questionnaire. Instructions for completing the questionnaire via the website will also need to be as clear and simple as possible to facilitate maximum web participation. Because it is not possible to ensure that all potential respondents fully understand the instructions or that computer glitches will not occur, we will also make it clear to potential web respondents that help desk staff will be available to them if and when they need them. Simply offering sample members a self-administered option is likely to increase response by allowing respondents greater discretion over how and when they can participate.

Despite the many advantages of offering a web survey mode, previous studies indicate that response rates are somewhat higher in interviewer-administered surveys than in self-administered surveys. Further methods of contact will include phone interviews (and in full scale in-person interviews) to increase contact with sample members over mail and Web contacts. Given the need to achieve high response rates, the interviewer training will focus considerable attention on enlisting cooperation. A large portion of the interviewer training for the third follow-up will concentrate on the most effective techniques for increasing participation. The two most important techniques on which interviewers will be trained are maintaining interaction with sample members and tailoring their approach to address the specific situation or concerns of potential respondents. An important part of these efforts are not only to highlight the importance of ELS:2002, but also to emphasize the importance of each respondent’s participation in the third follow-up survey. Exhibit B-1 summarizes the third follow-up full-scale data collection schedule and projected responses by survey mode.

Exhibit B-1. Third Follow-up Full-Scale Data Collection Schedule and Projected Response by Survey Mode

The use of cell phone calling and text messaging is a relatively new means for contacting sample members. Little research has been conducted on the effects of text messaging on participation rates. Research conducted by Brick et al. suggests that text messaging as a method of prenotifying sample members has nearly equal response rates as control group counterparts (Brick, Brick, Dipko, Presser, Tucker, and Yuan 2007). According to Lambries et al., those households using primarily cell phones required more attempts to contact than those using both landline and cell phones and those using landline only (Lambries, Link and Oldendick 2006). Households that used cell phones primarily showed differences of 1.1 more attempts than landline-only household and 0.8 more attempts than both landline and cell phone households.

However, text messaging has some advantages as the first means of contacting sample members. Text messaging may help identify working numbers and, in turn, increase the efficiency of the calling process (Steeh, Buskirk and Callegaro 2007). The research by Steeh et al. concludes that text messages have two advantages as the first means of contact: outcome rates are substantially improved and information about the working status of the number is obtained. Further research in this field is needed to better understand the effects of cell phone calling and text messaging on participation rates. The previously approved panel maintenance mailings include a question asking sample members if they would like to receive a text message when data collection is about to begin.

A final element of data collection strategy will be offering respondents incentive payments for their participation in the third follow-up as was done, for example, in the second follow-up. A propensity modeling experiment including differential incentive amounts are documented in Part A of this clearance package.

The key to achieving a high response rate in the third follow-up data collection will be combining all survey design elements into a comprehensive and effective strategy.

B.4 Tests of Procedures and Methods

Many of the procedures and methods developed for the ELS:2002 field test study have been developed and employed in prior NCES studies such as NELS:88/2000 or BPS and similar postsecondary studies. Given the mobility of the youthful population for the study, these methods include locating protocols as well as data collection systems and methodologies.

B.5 Reviewing Statisticians and Individuals Responsible for Designing and Conducting the Study

A number of individuals have consulted with NCES and RTI on the design and analysis plans for the ELS:2002. Members of the TRP have been described in Part A of this submission. In addition, Dr. Jeffrey A. Owings, Associate Commissioner for the Elementary/Secondary and Library Studies Division at NCES, has reviewed and approved the statistical aspects of the study. Other statistical reviewers at NCES include the project officer, John Wirt; the NCES Chief Statistician, Marilyn Seastrom; the Disclosure Review Board chair, Neil Russell; and Elise Christopher. Exhibit B-1 provides the names of RTI consultants on statistical aspects of ELS:2002, while Exhibit B-2 lists other principal RTI professional staff assigned to the study.

Exhibit B-2. RTI Consultants on Statistical Aspects of ELS:2002

Name	Affiliation
James Chromy	RTI
Steven J. Ingels	RTI
Daniel J. Pratt	RTI
John Riccobono	RTI
Peter H. Siegel	RTI
David Wilson	RTI

Exhibit B-3. Other Contractor Staff Responsible for Conduct of ELS:2002

Name	Affiliation
Ben Dalton	RTI
Donna Jewell	RTI
Erich Lauff	RTI
Tiffany Mattox	RTI
Jim Rogers	RTI

December 2010

Education Longitudinal Study: 2002
(ELS:2002)

Third Follow-up 2011 Field Test

OMB Supporting Statement

Part C

OMB# 1850-0652 v.7

National Center for Education Statistics

Institute of Education Sciences

U.S. Department of Education

Table of Contents

Contents

C.1 Description of Respondents 3

C2. SOURCES OF QUESTIONNAIRE MATERIAL 3

C.3 Questionnaire Gating: Current Activities 5

C.4 Research Area: Education 6

C.4.1 Specific Topic: High School Completion 6

C.4.2 Specific Topic: Postsecondary Enrollment 7

C.4.3 Specific Topic: The College Experience 8

C.4.4 Specific Topic: Educational Aspirations and Expectations 8

C.4.5 Specific Topic: Educational Debt and Finance 8

C.5 Research Area: Occupation 9

C.5.1 Specific Topic: Military Occupations 11

C.5.2 Specific Topic: Employment 11

C.5.3 Specific Topic: Career (Conceptualized Through Social Cognitive Career Theory) 12

C.5.4 Specific Topic: Job Training, Certification, and Licensure 12

C.5.5 Specific Topic: Employment History 13

C.5.6 Specific Topic: Obstacles to Career Goals 13

C.6 Family, Finances and Life Events 13

C.6.1 Specific Topic: Living and Family Arrangements and Configurations 13

C.6.2 Specific Topic: Income and Assets 14

C.6.3 Specific Topic: Civic Participation and Cultural Enrichment 14

C.6.4 Specific Topic: Life Events and Values 16

C.8 ITEM MODIFICATION INFORMATION 17

C.9 NEW QUESTION LIST 25

C.1 Description of Respondents

As described in Part A, the third follow-up study will provide data to map and understand the outcomes of the high school cohorts’ transition to adult roles and statuses at about age 26. Main research areas include the domains of 1) education, 2) occupation, and 3) other life outcomes including family formation, financial standing, and civic participation.

ELS:2002 is unique in that it provides information on these domains for respondents with many different educational experiences: dropouts from high school, non-college bound high school graduates, and college bound high school graduates (i.e., those who enrolled in college at time of last contact).

Response rates through the second follow-up were high enough to expect large returns of these groups for the third follow up. The overall response rate was 89%. Response rates for respondents who had ever reported a high school dropout episode (~1200 cases) were about 83% for the second follow-up. Non-college bound students also make up a large part of the sample; about 30% of second follow-up respondents reported never enrolling in postsecondary education as of 2006.

Not all sections of the questionnaire will apply to each respondent. Given the skip patterns (see Appendix 1), individual respondents will receive different subsets of all questions, with the time to complete the instrument averaging under 35 minutes.

C2. SOURCES OF QUESTIONNAIRE MATERIAL

The majority of ELS third follow-up 2011 field test questionnaire items originate from NCES surveys: ELS previous waves, NELS, HS&B, BPS, and B&B (see annotations of “Source” under each item in Appendix 1).

Other large non-NCES national surveys served as sources for a small number of questionnaire items. One item on postsecondary engagement (R1) came from the National Survey of Student Engagement (NSSE), a study of how undergraduate students spend their time and what they gain from attending college. NSSE is conducted by the Center for Postsecondary Research (CPR) in the Indiana University School of Education. Items on occupational earnings (EB3-EB5) came from the National Longitudinal Study of Youth (NLSY), a study of young adults and their labor market outcomes which is conducted by the Bureau of Labor Statistics. Items on military occupations (M1-M10, EC4) came from the National Longitudinal Study of Adolescent Health (AddHealth) Waves III and IV (covering 18-26 and 24-32 years old, respectively), which was a school-based longitudinal study of how social environments and behaviors in adolescence are linked to health and achievement outcomes in young adulthood. AddHealth was conducted with a nationally-representative sample of adolescents in grades 7-12 in the United States in 1994-95 and had a home-based questionnaire component in 2008. One item on barriers to career advancement (EF1) came from the Youth Development Survey conducted by the National Institute of Child Health and Human Development, which examines the consequences of work and other formative experiences in adolescence for the transition to adulthood, as well as the effects of experiences during this transition, for mental health, for economic attainment, and multiple facets of behavioral adjustment. One item on donating behavior (CP7) came from the University Learning Outcomes Assessment (UniLOA; Frederick and Barratt 2009, www.uniloa.org), which was developed by the Center for Measuring College Behaviors and Academics in the Bayh College of Education at Indiana State University. The UniLOA measures holistic student growth, learning and development in seven domains; the domain of citizenship includes the donation question. To date, the UniLOA has been administered to over 10,000 postsecondary students across the nation, representing both large and small private and public institutions.

A source other than from a large national study was the REFLEX questionnaire (EB2, EH7-EH8), which was developed by the European Union’s Sixth Framework Programme, involving partners from fifteen countries (Austria, Finland, France, Germany, Italy, the Netherlands, Norway, Spain, UK, Belgium, Czech Republic, Portugal, Switzerland, Japan and Estonia). The REFLEX project focuses on three broad and interrelated questions: (1) which competencies are required by higher education graduates in order to function adequately in the knowledge society; (2) what role is played by higher education institutions in helping graduates to develop these competencies; and (3) what tensions arise as graduates, higher education institutions, employers and other key players each strive to meet their own objectives, and how can these tensions be resolved.

A table identifying all survey items and providing additional information about them is provided in Exhibit C-1.

C.2.1 Expert Review of Questionnaire Material

Planning for the ELS:2002 third follow-up has updated the study by exploring new topics of interest to researchers as outcome measures, to be linked to the predictor measures recorded in previous waves. In developing the questionnaire for this round (Appendix 1), some important systematic inputs were solicited from experts. Specifically, reviews were commissioned reflecting the perspective of longitudinal research in labor economics and life course theory, as well as social learning theory with a career focus, from three distinguished researchers: Dr. Randall Olsen, Dr. Michael Shanahan, and Dr. Robert Lent. New items on job training and certification were developed by NCES’ Postsecondary Adult and Career Education division, and new questions based on social-cognitive career theory were developed by Dr. Bob Lent.

Dr. Randall J. Olsen is a Professor in the Department of Economics and the Center for Human Resource Research at The Ohio State University (Appendix 3). Olsen’s research focuses on econometrics, labor economics, applied microeconomics, and economic demography. He is the principal investigator of the National Longitudinal Survey of Youth (NLSY), a study of young adults and their labor market outcomes, which is conducted by the Bureau of Labor Statistics. As well as being widely published in leading economic journals, Olsen has advised various industry and government interests in economics and demography over the past four decades, including testifying before the US Senate Finance Committee on poverty-reduction policies.

Dr. Michael J. Shanahan is a Professor in the Department of Sociology at the University of North Carolina at Chapel Hill (Appendix 4). Shanahan is the author of over 60 peer-reviewed articles, chapters and books on the life course, including the Handbook of the life course (Mortimer & Shanahan, 2003). His research focuses on life course theory and methods, genetics and the life course (with an emphasis on status, stress, social capital, and health), and the transition to adulthood.

Dr. Robert W. Lent is a Professor and Co-Director of the Counseling Psychology Program in the Department of Counseling and Personnel Services at the University of Maryland. Lent has written numerous scholarly and applied research publications on self-efficacy and academic achievement, including Career development and counseling: Putting theory and research to work (Brown & Lent, 2005), and is an editor of the Journal of Career Assessment.

C.3 Questionnaire Gating: Current Activities (CA1 – CA3)

The first questionnaire area is a description of respondent status. This is a high-priority section of the questionnaire because it provides the foundation and path logic for much of the remainder of the survey instrument. These items will route each respondent through the questionnaire according to their indicated activities.

C.4 Research Area: Education

Education is the foundation of this study of a high school grade cohort followed over time. Direct and indirect educational outcomes by age 26 comprise a major focus of this questionnaire. By the third follow-up, most respondents will have made the transition from full-time education to the world of work and other adult roles. In other words the cohort will be, primarily, young adults, and not students. Nevertheless, the survey will update education information not collected since six years before, and thus encompassing the postsecondary educational careers of most sample members. Many respondents will likely receive questions in only a few of the education topic areas. Those who completed high school previously by 2006 will skip that section. Those who indicate no new or current postsecondary enrollment in the “Current Activities” section will skip the “Postsecondary Enrollment” and “College Experience” sections. (Of course while far fewer sample members will now be enrolled in college, adult education will be of expanded relevance, and job training, certification and licensure become major educational topics for the study—see C.5.4).

Five specific facets of education are examined in the ELS third follow-up questionnaire: (1) high school completion; (2) postsecondary enrollment; (3) the college experience; (4) educational aspirations and expectations; and (5) educational debt and finance. Each of these five facets of education is commented upon in further detail below.

C.4.1 Specific Topic: High School Completion (HS1-HS7)

The first area of questionnaire education content is the research area “high school completion.” This section is high priority for the relatively small (but highly policy-relevant) number of students who had not completed their secondary schooling (or equivalency) by 2006, or whose high school completion status was not known as of the second follow-up. One of the key milestones in a young person’s life is completion of high school. In its earlier rounds, ELS:2002 captured dropouts and students who fell behind the modal grade progression of their grade cohort. The basic pattern—high school dropouts’ continuing high school or GED completion—is likely to continue through the third follow-up. In this wave of the survey, we will update high school completion information for those who had not completed high school by 2006 or who were nonrespondents in 2006.

For those students who completed their high school degrees by obtaining a GED, we will obtain their reasons for completing their high school programs by this alternative path. Additional information, for linkage, may be obtained from GED Testing Service, a tack that was taken in the prior round when GED information could include test scores or pass/fail. For the questionnaires, standard questions are available from past rounds of ELS:2002 (which for comparability across multiple cohorts were taken from NELS:88) to address completion status and reasons for secondary education completion through an alternate route.

C.4.2 Specific Topic: Postsecondary Enrollment (EH1 – EH18)

The second follow-up of ELS in 2006 provided detailed information on issues related to early access to postsecondary education relative to high school experiences and family background. The timing of the third follow-up is ideal with respect to capturing degree attainment for students who enrolled in postsecondary education—including those who delayed enrollment by a year or two. Because it takes an average of almost 6 years to complete a bachelor’s degree, the third follow-up survey (conducted 8 years after most ELS participants graduated from high school) will be able to capture information on most of the college graduates as well as postsecondary education dropouts and stopouts.

The main function of the postsecondary enrollment section is to gather information on the names and locations of any postsecondary institutions that were not reported on the last interview in 2006. This information will then be used to contact the institutions and get copies of student transcripts. Postsecondary transcripts will provide data on a number of important topic areas: ultimate educational attainment, the grade-measured quality of academic performance, and educational persistence, intensity of enrollment, and transfer. This will shorten the interview for respondents and make time for more outcome measure items, such as career attainment and financial standing. This strategy was successfully applied in the postsecondary transcript collection of NELS:88/2000 to gather more data with decreased burden. On the basis of spring 2000 questionnaire information, transcript data were collected from initial postsecondary enrollment through any enrollment up to summer 2000. About 92 percent of requested transcripts were received. ELS:2002/12 will take advantage of recent NCES postsecondary transcript efforts to use updated course coding lists, fully specified new derived variables, and revamped coding engines.

The postsecondary education enrollment section will also include items related to the less-than-4-year institutions that have been a growing factor in postsecondary education and that many of the cohort members have attended. Information on prior postsecondary status will be preloaded from the 2006 interview.

C.4.3 Specific Topic: The College Experience (R1 – R4)

It is important to have some information on the institutional context of education at the postsecondary level, even if that data will seem sparse compared to the richness of the high school context data collected earlier in the study. An attempt has been made to maintain comparability between the ELS:2002 college experience data and those of other time series (whether repeated cross-sections, or longitudinal) studies of postsecondary education. The four items in this section are drawn from the National Survey of Student Engagement (NSSE), NELS F4, and/or B&B:93/03. These items gather information on high-impact college practices (such as study abroad, honors programs, research projects, etc.), and reasons for college dropout and lack of persistence. They also inquire into the current beliefs of the respondent about the value of their postsecondary education. This is an important area of research because the extent of students’ engagement in college education and its perceived contributions to the student’s overall development of knowledge and skills may relate to the student’s subsequent career development and other life course outcomes. Given that these questions will be retrospective in focus (few sample members will still be enrolled in college), we have been guided by the need to select items that are salient and fairly precisely knowable, and that are subject neither to post hoc rationalization or social desirability bias, as well as recall bias stemming from decay of memory. Dr. Robert M. Gonyea, the research coordinator for NSSE and Associate Director of the Indiana University Center for Postsecondary Research, was added to the Technical Review Panel for his expertise on content issues and knowledge of measurement issues within the domain of the college experience. Dr. Gonyea helped vet the college experience items on both methodological and substantive grounds.

C.4.4 Specific Topic: Educational Aspirations and Expectations (EA1)

It is of interest to know the degree to which the cohort has further educational attainment expectations and aspirations that remain unfulfilled at age 26. Because the question about expected education at age 30 has been asked in prior waves, ELS can do something that other postsecondary studies cannot, which is track shifts in educational expectations since secondary school (e.g., reasons for not completing their educational programs). Important aspirational shifts since secondary school may not happen in the same way following the start of postsecondary education. These data are of special importance to studies of secondary school tracking, and entrance to or exit from science and math fields.

C.4.5 Specific Topic: Educational Debt and Finance (ED1 – ED7)

ELS:2002 data will enable research on the relationships between educational goals and attainment, course of study, financial aid received, career path and plans, family formation, wealth, and debt. Debt is a new priority area for the third follow-up because the second follow-up asked questions about willingness to incur indebtedness through financial aid.

Financial aid data are used to examine access and choice, persistence, and attainment. They also relate to the constraints of indebtedness. Proposed questions to elicit financial aid information were developed in consultation with the NCES PACE (Postsecondary, Adult and Career Education) division. For each institution attended, respondents will be asked if they received grants or scholarships. If the answer is yes, they will be asked the amount.

C.5 Research Area: Occupation

Capturing employment, both for ELS participants who did not enroll in postsecondary education (“non-college bound”) and for those who did enroll, is important to better understand the rate of economic (and non-economic) return to individuals and society for various levels of education. In addition to determining the employment outcomes of the non-college bound population, examining the early labor market experiences of ELS participants who obtained postsecondary educationfrom short-term vocational credentials to advanced degreeswill help researchers and policymakers discern the benefits of various levels of postsecondary education. While the economic returns of a bachelor’s degree relative to a high school diploma, both in terms of occupations and earnings, have been well documented, the benefits of sub-baccalaureate credentials and on-the-job training for new entrants into the labor force, have been less clearly demonstrated and many non-economic returns to such human investments could usefully be documented as well.

Studies conducted by NCES’s PACE division such as B&B similarly map economic and non-economic returns to education. But secondary school surveys, such as NELS and ELS, contain additional high school control variables detailing each student’s experience as reported by the student, parent, and teacher, as well as math and reading assessments. The questions in this section could shed light on the employment outcomes and experiences, for example, of the large number of high school and community college students who dropped out struggling with math requirements as well as the large number of minority and low-income students whose high school math ability exceeded their course placements.

Six occupational sub-topics are noted below: (1) job training, certification, and licensure; (2) military occupations; (3) employment; (4) career; (5) employment history; and (6) obstacles to career goals.

C.5.1 Specific Topic: Job Training, Certification, and Licensure (PL1 – PL8; JT1 – JT3)

Another key aspect of employment is reflected by job-related training, and certificates and licenses. Recent history has seen a growing consensus about the skill requirements of the “21st Century workforce.” Along with the formal educational training of workers, the new flexible workforce will need workers who are continuously learning new skills and competencies, some of which may be validated with formal state or professional licensure and certification. By the year 2012, many of the members of the ELS cohorts will have been in the workforce and will have been exposed to on-the-job training and further job-related education through their employers. How and why this training is taking place is of great concern to employers, employees, and policymakers who are creating programs to facilitate this type of training.

To ensure accurate recall periods and to more closely target specific opportunities for training, we will ask about job-related training received in the current (or most recent) job. We will also limit the reference period to the last 12 months, to avoid recall difficulties.

Since this area is also being explored by NCES’s PACE division, we have adopted a subset of the PACE items on certification and licensure, and will look to the evidence of their field-testing as well as our own in evaluating content in this area. In short, we will actively continue to collaborate with PACE.

C.5.2 Specific Topic: Military Occupations (M1 – M10)

Military job training and occupational role potentially require some special questions. Although active duty was gathered as a past or present status, by design, in NELS:88, military service is not explored as a job or career experience; only the civilian labor market is so treated. Military experience is useful to know given the role of the armed services in employment and training, for a cohort coming of age in war time. For this group, on-the-job training is of the essence. This will be a smaller group of respondents, and those who do not indicate current or past military employment will skip this section. For those respondents formerly or currently in the military, the survey will collect information on location of service, branch of the military, component, entry time (and exit time if applicable) and military pay-grade.

There is also evidence that the transition to adulthood is marked by somewhat differing processes and outcomes than in the civilian labor sector, which further argues for inclusion of military participation content in the questionnaire. Kety, Kleykamp and Segal (2010) observe that the military is both career-oriented and family-oriented, resulting in a more stable and orderly transition to adulthood than experienced by civilian peers.

C.5.3 Specific Topic: Employment (E1 – E13; EB1 – EB5)

Economic stability and self-sufficiency are a major marker of transition to adulthood, though increasingly difficult to achieve, especially for the less educated (Danziger and Ratner 2010). One of the primary goals of ELS:2002 has been the collection of information about young people’s entry into the labor force, and especially the examination of information on the longer-term individual and institutional effects of secondary and postsecondary education, dropout and stopout behaviors, and aspirations. Three constructs mark distinctive features of employment: 1) current job (or most recent), 2) the notion of career, both as a status and as an animating plan, and 3) employment history (discussed in C.5.5).

Capturing employment, both for ELS participants who did not enroll in postsecondary education (“non–college bound”) and for those who did enroll, is important to better understand the rate of economic and noneconomic return to individuals and society for various levels of education. In addition to determining the employment outcomes of the non–college-bound population, examining the early labor market experiences of ELS participants who obtained postsecondary educationfrom short-term vocational credentials to advanced degreeswill help researchers and policymakers discern the benefits of various levels of postsecondary education. An important value added by this ELS questionnaire is the information gathered here on the benefits of sub-baccalaureate credentials, the many economic and noneconomic benefits of which have been less well documented by other postsecondary studies. Job benefits (both monetary and nonmonetary) are a further dimension of employment, including medical insurance, retirement plans, intellectual challenges, and earnings (items EB1 – EB5).

C.5.4 Specific Topic: Career (Conceptualized Through Social Cognitive Career Theory) (EC1 – EC4)

The ELS:2002/12 questionnaire also seeks to understand the role of career, and has approached this construct through the perspective of social cognitive career theory. Social cognitive theory posits that individual learning and knowing are influenced by observation of and interaction with others. With the assistance of Professor Robert Lent of the University of Maryland, a number of items reflecting social cognitive career theory have been written for ELS:2002/12, a new content area for the study. Within the occupational domain, items measure several key constructs: self-efficacy, outcome expectations, interests, supports, domain satisfaction, and persistence intentions. These variables are intended to complement and build on predictors and intermediate outcomes already included in ELS:2002, including earlier attempts to gather information on self-efficacy and self-directed learning. The items were reviewed positively by the Technical Review Panel and were recently cognitively tested and prepared for the field test. The field test will enable scale dimensionality and reliabilities to be assessed.

C.5.5 Specific Topic: Employment History (EH1 – EH9)

The need for employment data is explained in section C.5.2. A subpoint of its implementation is the status of employment event histories, given study design constraints. Employment history must be captured in some viable way. Because ELS:2002/12 and its 2011 field test must cover a 6-year interview gap in about 35 minutes, detailed employment event histories are unfortunately not possible. However, summary items will gather vital information, both for employment and unemployment spells.

C.5.6 Specific Topic: Obstacles to Career Goals (EF1 – EF3)

One important perspective on employment and career goals is the presence or absence of perceived or actual barriers (for example, perceived discrimination, lack of ability, lack of money, or family obligations etc.). Previous waves of ELS inquired about perceived or actual barriers to educational attainment, and so this section provides a relation to those items and a way for researchers to investigate if educational barriers continue into the employment domain. This short section will inquire about obstacles to career in the 6 years since last interview.

C.6 Family, Finances and Life Events

The final section of the questionnaire gathers contextual information on the outcomes of education and early work force experiences. Several specific topics are explored below: (1) living and family arrangements and configurations; (2) income and assets; (3) civic participation; and (4) life events and values.

C.6.1 Specific Topic: Living and Family Arrangements and Configurations (LF1 – LF16)

Living arrangements, family structure, and family formation have been the subject of many past questions in the later stages of the secondary longitudinal studies. Some statuses, such as that of single mothers, or number of dependents, may be especially of interest in interpreting outcomes at age 26. These data will provide evidence of any family situations that may have affected supports or barriers to education and employment. Past research with NELS:88 (Hardie 2010) (dissertation) demonstrates that the order and timing of family formation and dissolution events can disrupt young people’s paths to attainment in early adulthood while Glick et al. (2006) (Social Forces, 84(3): 1391-1415) explore how educational processes, including school engagement, influence the timing of family formation. One of the most important social changes of recent years has been the increasing delay in home-leaving, marriage, and the onset of childbearing, with also an increase in non-marital parenthood (Furstenberg 2010). The changing timing and sequencing of adult transitions has much to do with family formation and arrangements, and is a desirable area of inquiry for ELS:2002/12.

C.6.2 Specific Topic: Income and Assets (A1 – A12)

Income and assets data provide information on the return on investments in education, and resources that can be leveraged to enhance positive life changes. Considering the substantial earnings advantages of education, economic returns are one of the most important outcomes of education to record for ELS:2002. It is necessary to collect income information and information about basic assets from sample members on all pathways. Hourly rate of pay is extremely important in labor market analysis. Given the age span of the secondary longitudinal studies cohorts, they have proved especially informative on the rates of return to investments in subbaccalaureate education. Marcotte, Bailey, Borkoski and Kienzl (2005) examine returns on community college education using NELS:88 data. Some 12 years later, ELS:2002 will provide the opportunity to update such analyses, taking into account changes in both the labor market and community colleges during this time period.

C.6.3 Specific Topic: Civic Participation (CP1 – CP7)

Civic participation is (1) a major lifecourse marker of adult status, (2) important for personal growth and identity formation during the transition to adulthood, and (3) is important to the health and performance of democracy (Flanagan and Levine, 2010). Civic engagement questions (CP1 – CP6) have been asked in the predecessor studies, NLS:72, HS&B, and NELS:88, as well as the prior round of ELS:2002. Similar items are also asked on the American National Election Survey (ANES) and the National Longitudinal Study of Youth (NLSY97), but the wording from previous rounds of NCES studies are used to enable comparisons across the cohorts. These are relatively quick and simple items to collect, and in turn, provide measures of intra- and inter-cohort change on a key dimension marking the transition to adult status. A major current question is whether the timing of this transition has changed, in the context of a longer path to adulthood (Flanagan and Levine, 2010).

This transition tends to be smoother and more complete for some groups of adolescents than for others. The sample design of the secondary longitudinal studies provides the ideal analysis base for exploring civic participation issues, in that they can capture the considerable differences between the institutional opportunities for civic engagement among college and non-college youth during the young adult years—the ELS:2002 panel comprises both college goers and the non-college stream. As Furstenburg remarks, “As the transition to adulthood has lengthened, colleges have become the central institution for civic incorporation of younger generations. But no comparable institution exists for young adults who do not attend college.”

Items asked of 26-year-olds in NELS:88/2000 and proposed for use in the ELS third follow-up exemplify the potential of this area; for example, whether the individual performed volunteer work, the type of organizations involved in rendering community service, voter registration, and voting in the most recent presidential election. There are numerous research articles listed in the NELS:88 Bibliography that have employed the civic participation data. Some examples include: “Bowling Young: How Youth Voluntary Associations Influence Adult Political Participation” by McFarland and Thomas (2006) (American Sociological Review, 71(3): 401-425); “Preparing For Public Life: School Sector And The Educational Context Of Lasting Citizen Formation” by Dill (2009) (Social Forces, 87); and “High School Community Service As A Predictor Of Adult Voting And Volunteering” by Hart, Donnelly, Youniss and Atkins (2007), (American Educational Research Journal 44[1]: 197-219).

Note, however, that while civic participation items have principally been taken from prior education longitudinal studies or the prior round of ELS:2002, these sources provide no measure for monetary contributions or donations. A new item has therefore been added to capture financial contributions to causes, a behavior which has been reliably linked to the construct of citizenship (Barrett and Frederick, 2009) thus relates to voting and volunteerism.

C.6.4 Specific Topic: Life Events and Values (LV1 – LV3)

Significant life events (such as the death of a loved one, or being the victim of a serious crime) may have serious effects on a respondent’s life course. Hoffman (2003) establishes a link between stressful life events and delinquent behavior as well as poorer schooling outcomes. Knowing whether such events have occurred gives researchers another tool in examining barriers to positive education or employment outcomes. Sample member’s values (e.g., having strong friendships, finding steady work) have been collected in prior rounds of ELS, as well as in NELS:88, and collecting them again in the third follow-up allows for both inter- and intra-cohort comparisons.

C.8 Specific ITEM INFORMATION

Exhibit C-1. Item-level Descriptions and Justifications

The following table includes items used from prior rounds of ELS, from NELS or other survey sources (P = Prior). Revised items are compared to the old wording for reference (R = Revised). Several new items were written for ELS:2002/2012 by Professor Robert Lent of the University of Maryland (Appendix 5) to reflect recent developments in social-cognitive career theory. Other new items were added to address particular data requirements identified by the Technical Review Panel (N = New).

Item	Source	Status	Old wording (if Revised Item)	Item Wording	Justification
CA1	NELS F4	R	First, I would like to ask you some questions about your current activities. Are you currently… Response options: Yes, No (for each) Working for pay at a full-time job Working for pay at a part-time job or jobs Taking vocational or technical courses at any school or college Taking academic courses at a two- or four-year college, including graduate or professional schools Serving in another work experience, such as an apprenticeship, training program, or internship Keeping house full-time (homemaker) Holding a job but temporarily on leave or waiting to report to work	First, we would like to ask you some questions about your current activities. Are you currently… Response options: Yes, No (for each) Working for pay at one or more full-time jobs (35 hours/week or more) Working for pay at one or more part-time jobs (less than 35 hours/week) Taking vocational or technical courses at any school or college Taking academic courses at a two- or four-year college, including graduate or professional schools Serving in another work experience, such as an apprenticeship, training program, or internship Serving as full-time manager of your own household Caring for dependent children or adults Serving in the armed forces - either active duty, reserves, or National Guard?	Added parenthetical to first two items to clarify meaning of full-time v. part-time. Used a more modern wording for “homemaker” item. Replaced last item in source question (temporarily on leave) with two new items (caring for dependents and military) deemed of greater importance. These items performed very well. Data exists for 99% of eligible respondents for the NELS F4 items.
CA2	NELS F4	R	Were your work and school activities during the last week in January, 2000, the same as they are now?	Were your work and school activities during the last week in June, 2011, the same as they are now?	Revised month/year point-of-reference to reflect timing of ELS F3 FT data collection. This item performed very well. Data exists for 99% of eligible respondents for this item.
CA3	NELS F4	R	During the last week in January were you… Response options: Yes, No (for each) Working for pay at a full-time job Working for pay at a part-time job or jobs Taking vocational or technical courses at any school or college Taking academic courses at a two- or four-year college, including graduate or professional schools Serving in another work experience, such as an apprenticeship, training program, or internship Keeping house full-time (homemaker) Holding a job but temporarily on leave or waiting to report to work	During the last week in June 2011 were you… Response options: Yes, No (for each) Working for pay at one or more full-time jobs (35 hours/week or more) Working for pay at one or more part-time jobs (less than 35 hours/week) Taking vocational or technical courses at any school or college Taking academic courses at a two- or four-year college, including graduate or professional schools Serving in another work experience, such as an apprenticeship, training program, or internship Serving as full-time manager of your own household Caring for dependent children or adults Serving in the armed forces - either active duty, reserves, or National Guard?	Revised month/year point-of-reference to reflect timing of ELS F3 FT data collection. Added parenthetical to first two items to clarify meaning of full-time v. part-time. Used a more modern wording for “homemaker” item. Replaced last item in source question (temporarily on leave) with two new items (caring for dependents and military) deemed of greater importance. These items performed very well. Data exists for 99% of eligible respondents for the NELS F4 items.
HS1	ELS F2	P		Have you received a high school diploma, certificate of attendance, or a GED or other equivalency certificate? Response options: Yes, No	This item performs very well. 99% of the F2 respondents to whom this question applied provided a response.
HS2	n/a	N		Are you currently working towards a GED or equivalent? Response options: Yes, No	Important information to obtain, easy question to answer.
HS3	ELS F2	P		What type of high school diploma or certificate did you complete? Did you receive a… Response options: diploma; certificate of attendance, or; GED or other equivalency certificate?	This item performs very well. 99% of the F2 respondents to whom this question applied provided a response.
HS4	ELS F2	P		In what month and year did you receive your [diploma/certificate of attendance/GED or other equivalency/high school credential]? Response options: Month, Year	This item performs very well. 99% of the F2 respondents to whom this question applied provided a response.
HS5	ELS F2	P		How did you earn the GED or equivalency, or in other words, what program or school were you enrolled in, if any? Response options: No program, you just took the exam; part of a job training program; enrolled through adult education; part of a child care program or early childhood program; some other program	This item performs well. 97% of the F2 respondents to whom this question applied provided a response.
HS6	ELS F2	P		From what state did you receive your GED or equivalency?	This item performs well. 98% of the F2 respondents to whom this question applied provided a response.
HS7	ELS F2	P		Why did you decide to complete your GED or equivalency? Was it… Response options: Yes, No To improve, advance, or keep up to date on your current job? To train for a new job/career? To improve basic reading, writing or math skills? To meet requirements for additional study? Required or encouraged by your employer? For personal, family or social reasons?	This item performs well. Between 90% and 95% of the respondents to whom these items applied provided a response.
EH1	NELS F4	R	[When we spoke with you in 1994, you indicated that you had attended [XXXX, XXXX, etc.] after high school. Since that time, have you attended any other college, university, or vocational, technical or trade school for academic credit? / Since leaving high school, have you attended any college, university, or vocational, technical or trade school for academic credit?]	[When we spoke with you in 2005, you indicated that you had attended [XXXX, XXXX, etc.] after high school. Since that time, have you attended any other college, university, or vocational, technical or trade school for academic credit? / Since leaving high school, have you attended any college, university, or vocational, technical or trade school for academic credit?]	Revised year referenced in question wording to reflect timing of ELS F2 data collection. In NELS, 45% of respondents in F4 reported attending another postsecondary institution since the last time of contact.
EH2	NELS F4	P		[Schools we know about so far are: XXXX, XXXX, XXXX, etc.] What [other] college, university, or vocational, technical or trade school have you attended since leaving high school? Response options: text boxes for school name and city (plus dropdown list for state), along with an IPEDS coder	Item is needed to ensure the complete set of attended institutions is collected for each respondent, thereby facilitating a thorough transcript data collection.
EH3	ELS F2	R	What month and year did you first start attending [XXXX]?	In what month and year did you first attend [XXXX]?	Revised for improved readability. The measure performed very well in ELS F2; such a date was provided for 99% of the attended institutions reported by ELS F2 respondents.
EH4	NELS F4	P		Schools we know about so far are: [XXXX]. Did you attend elsewhere? Response options: Yes, No	Collecting the entire set of attended institutions for each sample member is necessary to ensure a thorough transcript data collection; this item is necessary to determine whether the entire set of attended institutions has been collected (i.e. this question is necessary to determine whether the respondent should be routed back to EH2).
EH5	NELS F4	R	Which of the schools did you attend most recently?	Which institution did you last attend?	Revised for improved readability. This measure performed very well. Data exists for 99% of eligible respondents.
EH6	NELS F4	P		In what month and year did you last attend [last/only PS school attended]? Month, Year	This measure performed very well. Data exists for 99% of eligible respondents.
EH7	NELS F4	P		Have you earned a degree or certificate from [last/only PS school attended]? Response options: Yes, No	This measure performed very well. Data exists for 99% of eligible respondents.
EH8	NELS F4	P		What type of degree or certificate did you receive from [last/only PS school attended]? (If you received more than one degree or certificate from [last/only PS school attended], please indicate the highest degree or certificate received.) Response options: Certificate; Associate’s Degree; Bachelor’s Degree; Master’s Degree; Ph.D. or equivalent (E.G., ED.D., D.P.H.); Professional doctorate (M.D., J.D., L.L.B., D.D.S. ETC.)	This measure performed very well. Data exists for 99% of eligible respondents.
EH9	NELS F4	P		In what month and year did you receive your [credential] from [last/only PS school attended]? Month, Year	This measure performed very well. Data exists for 99% of eligible respondents.
EH10	NELS F4	P		What was your primary major or program of study for your [credential] from [last/only PS school attended]? Response options: textbox for major, plus CIP coder	This measure performed very well. Data exists for 99% of eligible respondents. CIP coder (which enables real-time coding of areas of study) should aid ease of response as well as accuracy.
EH11	NELS F4	P		Did you have a secondary major or program of study for your [credential] from [last/only PS school attended]? Response options: Yes, No	This measure performed very well. Data exists for 99% of eligible respondents.
EH12	NELS F4	P		What was your secondary major or program of study for your [credential] from [last/only PS school attended]? Response options: textbox for major, plus CIP coder	This measure performed very well. Data exists for 99% of eligible respondents. CIP coder should aid ease of response as well as accuracy.
EH13	NELS F4	P		[You have already told us about your [credential] from [XXXX]]. What other degrees or certificates, if any, do you have? Response options: Do not have any other degrees or certificates; Certificate; Associate’s Degree; Bachelor’s Degree; Master’s Degree; Ph.D. or equivalent (E.G., ED.D., D.P.H.); Professional doctorate (M.D., J.D., L.L.B., D.D.S. ETC.)	This measure performed very well. Data exists for 99% of eligible respondents.
EH14	NELS F4	P		From what institution did you earn your [credential]? Response options: [all PS schools indicated in F2/F3]	This measure performed very well. Data exists for 99% of eligible respondents.
EH15	NELS F4	P		In what month and year did you receive your [credential] from [XXXX]? Month, Year	This measure performed very well. Data exists for 99% of eligible respondents.
EH16	NELS F4	P		What was your primary major or program of study for your [credential] from [XXXX]? Response options: textbox for major, plus CIP coder	This measure performed very well. Data exists for 99% of eligible respondents. CIP coder should aid ease of response accuracy as well.
EH17	NELS F4	P		Did you have a secondary major or program of study for your [credential] from [XXXX]? Response options: Yes, No	This measure performed very well. Data exists for 99% of eligible respondents.
EH18	NELS F4	P		What was your secondary major or program of study for your [credential] from [XXXX]? Response options: textbox for major, plus CIP coder	This measure performed very well. Data exists for 99% of eligible respondents. Data was previously coded from verbatim responses to interviewer. CIP coder should aid ease of response as well as accuracy.
R1	NSSE	R	Which of the following have you done or do you plan to do before you graduate from your institution? Response options: Done, Plan to do, Do not plan to do, Have not decided	[Have you participated/Did you participate] in any of the following as a part of your [college/undergraduate] education? Response options: Yes, No	NSSE question is asked of respondents currently in college; revised for ELS to account for the more likely scenario that they are no longer in college.
R2	NELS F4	R	You told me earlier that you are no longer enrolled in any school and that you did not obtain a degree or certificate. Why did you leave school? Done taking the desired classes Financial reasons Change in family status (e.g., marriage, baby, death in family) Personal problems/injury/illness/conflicts with demands at home Academic problems Not satisfied with program/school/campus/faculty Classes not available/class scheduling not convenient Job/military considerations Moved from the area Decided to take time off from studies Enrollment doesn’t suit lifestyle/boredom with school School/program closed/lost accreditation	[Earlier you indicated that you are no longer enrolled in any school and that you did not obtain a degree or certificate. / Earlier you indicated that you had once attended a 4-year school, are no longer enrolled in any school, and that you did not obtain a 4-year degree.] Which of the following are reasons you left school [without completing a 4-year degree]? Done taking the desired classes Financial reasons Change in family status (for example, marriage, baby, death in family) Personal problems, injury, or illness Conflicts with demands at home Academic problems Not satisfied with the program, school, campus, or faculty Classes were not available or class scheduling was not convenient Job or military considerations Moved from the area Decided to take time off from studies Enrollment did not suit your lifestyle or you were bored with school The school or program closed or lost accreditation	Added conditional wording to account for the fact that in ELS we plan to ask this question of respondents who have attended at a 4-year institution, but who only have a 2-year degree and are no longer attending college. Split 4th item in the original question to distinguish combined items. In NELS F4, of the 20% of respondents who indicated PSE enrollment but no PSE completion, only .8% did not answer this question. This is thus a very useful question for researchers of postsecondary persistence behavior.
R3	B&B:03/04	R	Which of the following aspects of your [credential] are you satisfied with at [name of school at which it was earned]? Response options: Yes, No	How satisfied are you with the following aspects of your [credential] at [name of school at which it was earned]? Response options: Very satisfied, Somewhat satisfied, Somewhat dissatisfied, Very dissatisfied	Revised response options from yes/no to a Likert-type scale to obtain further specificity in terms of respondent’s satisfaction. Revised stem wording to reflect revised response options.
R4	B&B:03/04	R	Which of the following aspects of your undergraduate education would you consider to be very important to your life now? Response options: Yes, No	How important would you say your [college/undergraduate] education was in preparing you for the following aspects of your life? Response options: Very important, Somewhat important, Not at all important	Revised response options from yes/no to a Likert-type scale to obtain further specificity in terms of importance to respondent. Revised stem wording to reflect revised response options.
EA1	ELS F2	P		What is the highest level of education you ever expect to complete? Response options: Less than high school graduation GED or other high school equivalency only High school diploma only Complete a 1- or 2-year program in a community college or vocational school Bachelor’s degree Master’s degree or equivalent Ph.D., professional doctorate, or equivalent Don’t know	Percent of eligible respondents answering this question: ELS F2: 99% ELS F1: 99% ELS BY: 99% NELS F4: 99% NELS F3: 99% NELS F2: 93% This item has consistently performed well over time.
ED1	B&B:08/09	R	Other than money you may have borrowed from family or friends, did you take out any type of education loans to help pay for your undergraduate education?	Other than money you may have borrowed from family or friends, did you take out any type of education loans to help pay for your education since high school?	Replaced “undergraduate education” with “education since high school” in hopes that that terminology would be more easily understood, and to also capture loans that may have be taken out for graduate education.
ED2	B&B:08/09	R	How much of the amount that you borrowed in total undergraduate loans do you still owe? (If you are unsure of the amount, provide your best estimate.)	How much of this amount that you borrowed do you still owe? (If you are unsure of the amount, provide your best estimate. If you have already repaid these loans in full, please enter ‘0’.)	Revised question wording such that it is not limited to undergraduate education. Provided further guidance in parenthetical to aid self-administered respondents.
ED3	BPS:09	R	How much do you pay each month for your undergraduate education loans?	How much do you pay each month for these loans? (If none, please enter 0.)	Revised question wording such that it is not limited to undergraduate education. Added parenthetical to provide guidance for self-administered respondents.
ED4	n/a	N		Has any of your student loan debt been paid off by you, your family, or been forgiven by a loan forgiveness program? Response options: Paid/forgave none of the debt, Paid/forgave some of the debt, Paid/forgave all of the debt You Your family Forgiven by a loan forgiveness program Before you continue to the next question, we would like to know if you had any difficulty understanding or answering this question so that we may improve it for future surveys. Did you have any difficulty such as not understanding question wording, being uncertain of the meaning of certain terms or response choices, or not having the information needed to answer the question? Response options: Yes; No If yes, please describe any difficulty you had. Please be as specific as possible. Response option: [textbox in which respondent can describe any difficulty]	Provides researchers with information on how student loan debts are being repaid. For the field test, we are including a same-screen follow-up asking respondent if there was any difficulty in answering question; responses to this follow-up should aid in assessing performance of the new question.
ED5	BPS:09	R	In which of the following ways has your undergraduate student loan debt influenced your employment plans and decisions?	Did your student loan debt influence your employment plans and decisions in any of the following ways?	Revised question wording such that it is not limited to undergraduate education. Changed from a check-all-that-apply question to a yes/no likert-type question, and revised question wording accordingly.
ED6	n/a	N		While attending [institution name], did you receive any grants or scholarships? [ED6/ED7 to be asked for each school attended.] Response options: Yes; No	Provides researchers with information on funds that students may have received to help pay for postsecondary education which did not incur debt for the sample members (e.g., grants and scholarships). These questions were developed in consultation with NCES’ Postsecondary, Adult and Career Education Division, and their performance will be assessed utilizing the field test results.
ED7	n/a	N		Approximately how much did you receive in grants and/or scholarships while attending [institution name]? (Please round to the nearest $1000.) [ED6/ED7 to be asked for each school attended.] Response options: text box with dollar amounts Before you continue to the next question, we would like to know if you had any difficulty understanding or answering this question so that we may improve it for future surveys. Did you have any difficulty such as not understanding question wording, being uncertain of the meaning of certain terms, or not having the information needed to answer the question? Response options: Yes; No If yes, please describe any difficulty you had. Please be as specific as possible. Response option: [textbox in which respondent can describe any difficulty]	As with ED6. For the field test, we are including a same-screen follow-up asking respondent if there was any difficulty in answering question; responses to this follow-up should aid in assessing performance of the new question.
PL1	n/a	N		Now we would like to ask you about professional certification and licensure. Do you have a current professional certification or a state or industry license? (A professional certification or license verifies that you are qualified to perform a specific job. It includes things like licensed realtor, certified medical assistant, certified construction manager, or Cisco Certified Network Associate.) Response options: Yes, No Before you continue to the next question, we would like to know if you had any difficulty understanding or answering this question so that we may improve it for future surveys. Did you have any difficulty such as not understanding question wording, being uncertain of the meaning of certain terms, or not having the information needed to answer the question? Response options: Yes; No If yes, please describe any difficulty you had. Please be as specific as possible. Response option: [textbox in which respondent can describe any difficulty]	Certifications and licenses are of great concern to employers, employees, and policymakers. NCES’s Postsecondary, Adult, and Career Education Division (PACE) is developing new questions on this topic which we propose using in the ELS F3 field test. For the field test, we are including a same-screen follow-up asking respondent if there was any difficulty in answering question; responses to this follow-up should aid in assessing performance of the new question.
PL2	n/a	N		Do you have more than one certification or license? Response options: Yes, No	Certifications and licenses are of great concern to employers, employees, and policymakers. NCES’s Postsecondary, Adult, and Career Education Division (PACE) is developing new questions on this topic which we propose using in the ELS F3 field test.
PL3	n/a	N		How many do you have?	As with PL2.
PL4	n/a	N		[Let’s talk about your most recent certification or license.] Is it a certification, license, or both? (“Both” typically occurs when someone gets a license upon completion of a certification program. If “both,” ask rest of questions about certification.) Response options: Certification, License, Both	As with PL2.
PL5	n/a	N		Did you have to do any of the following to get this [certification/license]? Response options: Yes, No Demonstrate skills while on the job? Pass a test or exam? Submit a portfolio of your work?	As with PL2.
PL6	n/a	N		To maintain this [certification/license], do you have to… Response options: Yes, No Take continuing education classes or earn CEUs? Take periodic tests?	As with PL2.
PL7	n/a	N		Can this [certification/license] be… Response options: Yes, No Revoked or suspended for any reason? Used if you wanted to get a job with any employer in that field? (Answer “yes” for credentials that are recognized state-wide or regionally)	As with PL2.
PL8	n/a	N		What benefits did you receive or do you plan to receive from earning this [certification/license]? Response options: Received; Not received but expect to receive; Not received and not expected to receive Higher pay or bonus Promotion upon completion of the training Future advancement opportunities Improved job performance Remain current with new regulations, laws, or technologies Change job or career field, enter the workforce, or start own business	As with PL2.
M1	AddHealth	P		Have you ever been in the military? Response options: Yes, No	In the most recent wave of AddHealth (Wave IV, 2007-08), 7% of respondents reported having served in the military. This is important to know because the US military is a large provider of on-the-job training, and this population may increase as more young adults serve during wartime. This measure performed very well. 100% of respondents answered this question in Wave IV.
M2	AddHealth	R	Was your military service in the US, outside the US, or both?	[Was your military service/Has your military service been] in the US, outside the US, or both?	Added conditional wording to account for those currently in the military. This measure performed very well. 100% of respondents who indicated previous military experience answered this question in Wave IV.
M3	AddHealth	P		In which branches of the military have you served? You may select more than one answer. Response options: Yes, No Army Air Force Marine Corps Navy Coast Guard	This measure performed very well. 100% of respondents who indicated previous military experience answered this question in Wave IV.
M4	AddHealth	P		In which branch are you currently serving? Response options: Army; Air Force; Marine Corps; Navy; Coast Guard	As with M3.
M5	AddHealth	P		In which component are you currently serving? Response options: Active duty; Reserves; National Guard	As with M3.
M6	AddHealth	P		In what month and year did your first military service begin? Month, Year	This measure performed very well. 99% of respondents who indicated previous military experience answered this question in Wave IV.
M7	AddHealth	P		In what month and year did your most recent military service end? Month, Year	This measure performed very well. 99% of respondents who indicated previous military experience answered this question in Wave IV.
M8	AddHealth	R	What is the highest military rank you have achieved? Response options: E-1; E-2; E-3; E-4; E-5; E-6; E-7; E-8; O-1; O-1E; O-2; O-2E; O-3; O-3E; W-1; W-2	What is the highest military pay grade you have achieved? Response options: E-1...E-10; O-1...O-10; O1E...O3E; W-1...W-5	Replaced “rank” with “pay grade” since “pay grade” more accurately characterizes response options. Updated response options based on military.com website.
M9	AddHealth	P		What is the total amount of time you (have) served on active duty? (If none please enter ‘0’.) Years, Months	This measure performed very well. 100% of respondents who indicated previous active duty military experience answered this question in Wave IV.
M10	AddHealth	P		What is the total amount of time you (have) served in a combat zone? (If none please enter 0.) Years, Months	This measure performed very well. 99% of respondents who indicated previous combat answered this question in Wave IV.
E1	NELS F4	P		Since high school, have you ever held a job for pay? Response options: Yes, No	This measure performed very well. 100% of eligible respondents answered this question. 96% reported having worked for pay at some time since high school.
E2	NELS F4	P		You mentioned before that you are not currently working for pay at [a full-time / either a full-time or a part-time ] job. Do you want a [full-time / full- or part-time] job for pay at this time? Response options: Yes, No	This measure performed very well. 99% of eligible respondents (those who indicated no current employment) answered this question.
E3	n/a	N		[Including your military service, how / How] many full-time jobs and how many part-time jobs for pay have you held for pay since January 2005? Number of full-time jobs (35 hours/week or more) Number of part-time jobs (less than 35 hours/week)	Asking how many jobs the respondent has held will help put a lot of the economic data in context. If analysts want to understand how effective education has been in improving economic circumstances, they may want to measure job changing behavior as this is one of the key pathways through which earnings increase.
E4	NELS F4	P		[Including service in the armed forces, in / In] what month and year were you last working for pay? Month, Year	This measure performed very well. 99% of eligible respondents (those who indicated not currently working for pay) answered this question.
E5	NELS F4	P		Do you currently have more than one [full-time/part-time/military] job? Response options: Yes, No	This measure performed very well. 100% of eligible respondents answered this question.
E6	NELS F4	R	Altogether, how many jobs do you have?	Altogether, how many full-time and part-time jobs do you currently have? Current number of full-time jobs (35 hours/week or more) Current number of part-time jobs (less than 35 hours/week)	Revised question wording to reflect the fact that we are asking for separate FT and PT counts, as opposed to asking for a single response.
E7	NELS F4	P		[I would like you to answer the following questions for your primary/most important/military job. For your primary job, what is your job title? / For your most recent job, what was your job title?] What [do/did] you do as an [XXXX]? Response options: A textbox for job title and another for job duties, along with O*NET coder	This measure performed very well. 99% of eligible respondents answered this question. The use of real-time coding (the O*NET coder) considerably aids ease of responding and data quality.
E8	ELS F2	P		In what month and year did you begin your [job as a XXXX/primary job/current job/military job/most recent job]? Month, Year	This measure performed very well. 99% of eligible respondents answered this question.
E9	NELS F4	P		For your [job as a XXXX/primary job/current job/military job/most recent job] [are/were] you working for yourself or someone else? Response options: Self-employed, Someone else	This measure performed very well. 99% of eligible respondents answered this question.
E10	NELS F4	R	What type of company [employs/employed] you? [Is/Was] it a… Response options: Private, for-profit, company; Non-profit or not-for-profit company; Local government; State government; Federal government, including civilian employees of the military; Military, including National Guard	What type of organization or business [employs/employed] you? [Is/Was] it a… Response options: Private, for-profit, company; Non-profit or not-for-profit company; Local government; State government; Federal government, including civilian employees of the military; Military, including National Guard	Replaced “company” with “organization or business” to better characterize response options. In NELS F4, this measure performed very well. 99% of eligible respondents answered this question.
E11	NELS F4	P		How many hours per week, in a typical week, do you currently work for pay in your [job as a XXXX/primary job/current job/military job/most recent job]?	This measure performed very well. 99% of eligible respondents answered this question.
E12	NELS F4	P		[Now I would like you to consider all of your current jobs for pay.] How many hours per week do you work for pay in a typical week at these jobs?	This measure performed very well. 99% of eligible respondents answered this question.
E13	NELS F4	P		Which one of the following four statements best describes your [job as a XXXX/primary job/current job/military job/most recent job]? Response options: Someone else decided what you did and how you did it; Someone else decided what you did, but you decided how to do it; You had some freedom in deciding what you did and how you did it; You were basically your own boss	This measure performed very well. 99% of eligible respondents answered this question.
EB1	BPS:09	R	Which of the following benefits does your current employer offer? Medical insurance and/or other health insurance such as dental or vision Life insurance Retirement or other financial benefits, such as a 401(k)/403(b)	[Which of the following benefits [does/did] your [primary/current/most recent] employer offer? / As a self employed [XXXX] / In military position which of the following do you have?] Response options: Yes, No Medical insurance or other health insurance such as dental or vision Life insurance Retirement or other financial benefits, such as a 401(k)/403(b)	Added conditional wording such that the current/primary/most recent/military job is referenced, as opposed to just current employer, in case not currently employed.
EB2	REFLEX	R	Please indicate how important the following job characteristics are to you personally, and to what extent they actually apply to your current work situation. Job security Opportunity to learn new things High earnings New challenges Good career prospects Enough time for leisure activities Social status Chance of doing something useful for society Good chance to combine work with family tasks	Please indicate to what extent the following job characteristics [apply/applied] to your [job as a XXXX/primary job/current job/military job/most recent job]. Job security Opportunity to learn new things High earnings New challenges Enough time for leisure activities Chance of doing something useful for society Good chance to combine work with family tasks	Dropped two items (‘good career prospects’ and ‘social status’) which overlap with other questions. Source question wording is drawn from a paper questionnaire (which does not have conditional wording), while the ELS instrument can conditionally reference the respondents actual current job. The REFLEX questionnaire is essentially asking for two responses for each item – one indicating how important each characteristic is to the respondent, and one indicating to what extent each characteristic actually applies to their current situation. In ELS the hope is to only ask about the latter, and so the question wording has been revised accordingly.
EB3	NLSY	P		Now we would like to ask you a few questions concerning your earnings at your [job as a XXXX/primary job/current job/most recent job]. For your [job as a XXXX/primary job/current job/most recent job], what is the easiest way for you to report your total earnings before taxes or other deductions? We use this information to compare the amount that people earn in different types of jobs. Response options: per hour; per day; per week; every 2 weeks; twice a month; per month; per year	The first of three questions for eliciting the respondent’s earnings in the current/most recent job in a proven way that is compatible with web administration. The similar question from NELS F4 was used in a telephone interview. First, the respondent is asked for the “easiest way” to report his/ her earnings. 99% of eligible respondents answered this question in NLSY.
EB4	NLSY	P		Even though you told me it is easier to report your earnings [EB3], [are/were] you paid at an hourly rate on your [job as a XXXX/primary/current job/most recent] job? Response options: Yes, No	If the answer to the first question is other than an hourly rate, then the respondent is then asked if they were nonetheless actually paid on an hourly basis or not. 99% of eligible respondents answered this question in NLSY.
EB5	NLSY	P		For your [job as a XXXX/primary/current/most recent] job about how much [per hour (if EB3=per hour or EB4=yes) / per EB3] [do/did] you earn before taxes and other deductions?	If the respondent answers in either of the first two questions that they were paid on an hourly basis, then they are asked what that rate is. Otherwise, they are asked the rate that is easiest for them to report. This approach maximizes the frequency with which we receive an hourly rate of pay from the respondent, which attenuates measurement error problems that arise when the time unit of rate of pay is other than hourly and reported hours must be used to calculate the hourly rate of pay.  This is important as hourly rate of pay is a workhorse variable of labor market analysis. 99% of eligible respondents answered this question in NLSY.
EC1	n/a	N		Please indicate the extent to which you agree or disagree with each of the following statements with respect to your [job as a XXXX/primary job/current job/military job]: Response options: 1=Strongly disagree; 2; 3; 4; 5=Strongly agree You are confident that you can perform your job exceptionally well (in other words, much better than average) You are certain that you can solve big problems that occur at work You are confident that you can reach the larger goals you set for yourself at work You are certain that you can do your work well despite time pressures You are confident that you can do your work well even when you need to juggle work with non-work responsibilities (for example, in your family or community)	New question attempting to provide insight into the career status of the sample member’s current job, as well as the sample member’s career choice, based on a well validated construct (occupational self-efficacy) from social-cognitive career theory.
EC2	n/a	N		Your [job as a XXXX/primary job/current job/military job] allows you to… Response options: 1=Strongly disagree; 2; 3; 4; 5=Strongly agree Get respect from your friends and family Do work that you find satisfying Earn enough money to meet your needs Work with other people who share your values	Items based on outcome expectations construct from social cognitive career theory. Has been cognitively tested. All respondents in cognitive testing were able to answer this question.
EC3	n/a	N		Please indicate the extent to which you agree or disagree with each of the following statements about your job: Response options: 1=Strongly disagree; 2; 3; 4; 5=Strongly agree You are really interested in your work You often get totally absorbed or deeply focused in your job tasks You like the major tasks that make up your job Most people at work are pretty supportive of you There are people you can learn from at work There are people you can turn to for help in solving a work problem You feel fairly well satisfied with your present job Most days you are enthusiastic about your work You find real enjoyment in your work You plan to remain in your current job over the next year You don’t usually think about leaving this job You feel pretty strongly committed to keeping your current job	Items based on the constructs of occupational interests, work support, job satisfaction, and job persistence intentions from social cognitive career theory. Has been cognitively tested. All respondents in cognitive testing were able to answer this question.
EC4	AddHealth	P		Which of the following best describes your [job as a XXXX/primary job/current job/military job]? Response options: It is part of your long-term career or work goals; It is preparation for your long-term career or work goals; It is not related to your long-term career or work goals; You do not have long-term career or work goals	This measure performed very well. 99% of respondents who indicated current employment answered this question in Wave IV.
JT1	n/a	N		Now we would like to ask you about any formal job training you have received from your employer. Think about the skills that are needed for doing your current job as a [occupation]. In the last 12 months, have you participated in a formal training program offered by your employer or a union that helped you learn or improve the skills needed to do your job? Response options: Yes, No Before you continue to the next question, we would like to know if you had any difficulty understanding or answering this question so that we may improve it for future surveys. Did you have any difficulty such as not understanding question wording, being uncertain of the meaning of certain terms, or not having the information needed to answer the question? Response options: Yes; No If yes, please describe any difficulty you had. Please be as specific as possible. Response option: [textbox in which respondent can describe any difficulty]	Employer-provided job training of great concern to employers, employees, and policymakers. NCES’s Postsecondary, Adult, and Career Education Division (PACE) is developing new questions on this topic which we propose using in the ELS F3 field test. We are including a same-screen follow-up asking respondent if there was any difficulty in answering question; responses should aid in assessing performance of question.
JT2	n/a	N		About how long did that training program last? (Enter a number here, and then select the unit of time below). Response options: [     ] () days () weeks () months	As with JT1.
JT3	n/a	N		On average, about how many hours per [(time-period)] were spent on this training?	As with JT1.
EU1	BPS:09	R	Would you say your current job is related to the major or field of study you had when you were last enrolled at [primary undergraduate school] as an undergraduate?	Would you say your [job as a XXXX/primary job/current job/military job/most recent job] [is/was] related to the major or field of study you had when you were last enrolled at [last PS school / PS school from which respondent earned a credential]?	Revised question wording to reference the name of the respondent’s current or most recent job, so that the question is not limited to undergraduate education.
EU2	BPS:09	R	Would it be difficult for you to do your current job without having had the undergraduate courses you took at [primary undergraduate school]?	[Would it be/Was it] difficult for you to do your [job as a XXXX/primary job/current job/military job/most recent job] without having had the courses you took at [last PS school / PS school from which respondent earned a credential]?	Revised question wording to reference the name of the respondent’s current or most recent job, so that the question is not limited to undergraduate education.
EU3	BPS:09	R	Were any of the following required by your current employer as a condition for working?	[Are/Were any of the following required by your [primary/current/most recent] employer as a condition for working? / Are/Were any of the following required for your [job as a XXXX/primary job/current job/military job/most recent job]?	Added conditional wording such that the respondents’ actual current or most recent job title can be referenced in question wording.
EH1	NELS 94	R	For the next items, I want to ask about [your employment/any military or civilian employment] last year in 1999, and in the two years before that. Across all your jobs during the 1999 calendar year, how many weeks did you work for pay? Please include all paid time off such as vacations, sick leave, and family leave in your weeks spent working. (Do not include the time you spent out of work, between jobs, or without pay.) (Question references 1999.)	For the next items, I want to ask about [your employment/any military or civilian employment] last year in 2010, and in the two years before that. Across all your jobs during the 2010 calendar year, how many weeks did you work for pay? Please include all paid time off such as vacations, sick leave, and family leave in your weeks spent working. (Do not include the time you spent out of work, between jobs, or without pay.) (Question references 2010.)	Revised to reflect years appropriate for ELS F3 FT. This measure performed very well. 99% of eligible respondents answered this question.
EH2	NELS 94	R	How many hours did you work for pay at all jobs in a typical week in 1999?	How many hours did you work for pay at all jobs in a typical week in 2010?	Revised as with EH1. This measure performed very well. 98% of eligible respondents answered this question.
EH3	NELS 94	R	Now, I would like you to think back to the year before last. During the 1998 calendar year, were you employed [either by the military or in the civilian workforce] for six months or more during the year? Response options: Employed for 6 months or more; employed, but for less than 6 months; not employed at all	Now, I would like you to think back to the year before last. During the 2009 calendar year, were you employed [either by the military or in the civilian workforce] for six months or more during the year? Response options: Employed for 6 months or more; employed, but for less than 6 months; not employed at all	Revised as with EH1. This measure performed very well. 99% of eligible respondents answered this question.
EH4	NELS 94	R	For your employment in 1998, were you employed primarily full-time or part time? Response options: Full Time, Part Time, Not employed at all during 1998	For your employment in 2009, were you employed primarily full-time or part time? Response options: Full Time, Part Time, Not employed at all during 2009	Revised as with EH1. This measure performed very well. 99% of eligible respondents answered this question.
EH5	NELS 94	R	Now, I would like you to go back a year further to 1997. During the 1997 calendar year, were you employed [either by the military or in the civilian workforce] for six months or more during the year? Response options: Employed for 6 months or more; employed, but for less than 6 months; not employed at all	Now, I would like you to go back a year further to 2008. During the 2008 calendar year, were you employed [either by the military or in the civilian workforce] for six months or more during the year? Response options: Employed for 6 months or more; employed, but for less than 6 months; not employed at all	Revised as with EH1. This measure performed very well. 99% of eligible respondents answered this question.
EH6	NELS 94	R	For [this/any] employment in 1997, were you employed primarily full-time or part time? Response options: Full Time, Part Time, Not employed at all during 1997	For [this/any] employment in 2008, were you employed primarily full-time or part time? Response options: Full Time, Part Time, Not employed at all during 2008	Revised as with EH1. This measure performed very well. 99% of eligible respondents answered this question.
EH7	REFLEX	R	Have you ever been unemployed (that is, not employed and seeking employment) since graduation in 1999/2000? Response options: Yes, ___ number of times for a total of _____ months; No	Since January 2008, have you ever been unemployed (that is, not employed but seeking employment)? Response options: Yes; No	Revised question stem to reflect years appropriate for ELS F3 FT. Revised question stem to account for the fact that not all ELS respondents will be “graduates”. Split REFLEX question (asked as part of paper questionnaire, where respondents could ‘fill in the blanks’) into two questions
EH8	REFLEX	R	Have you ever been unemployed (that is, not employed and seeking employment) since graduation in 1999/2000? Response options: Yes, ___ number of times for a total of _____ months; No	Since January 2008, approximately how many times have you been unemployed (not employed but seeking employment), and for approximately how many months? Total number of times Total number of months	Revised question stem to reflect years appropriate for ELS F3 FT. Revised question stem to account for the fact that not all ELS respondents will be “graduates”. Split REFLEX question (asked as part of paper questionnaire, where respondents could ‘fill in the blanks’) into two questions
EH9	BPS:09	R	After your undergraduate enrollment at [primary undergraduate school], what was the longest period of time you were unemployed and looking for a job?	[Since January 2008, / Of the [X] number of times you mentioned being unemployed,] what was the longest period of time you were unemployed (not employed but looking for a job)?	Revised question wording to reflect the fact that not all ELS respondents are postsecondary attendees.
EF1	YDS	R	Since March 1999, have any of the following interfered with your work or your career plans? Are you concerned that any might interfere in the future? Response options: Has interfered during the past year; Concerned about the future; Not a concern	Since January 2005, have any of the following interfered with your work or career plans? Response options: Yes, No	YDS is a paper questionnaire, and, in this instance, is essentially asking two questions at once. Modified question stem for ELS to only ask about the first of these two questions; also modified response options accordingly. Also modified time point-of-reference.
EF2	NELS F4	P		What job or occupation do you plan to have when you are age 30? [to be coded using ONET occupation coder]	Percent of eligible respondents answering the question: ELS F2: 99% ELS F1: 98% ELS BY: 89% NELS F3: 90% NELS F2: 95% NELS F1: 84% Response rates to this item have increased over time. ELS F2 also used the O*NET coder, which considerably aided ease of responding and data quality.
EF3	ELS F2	P		How much education do you think you need to get the job you expect or plan to have when you are 30 years old? Response options: Less than high school graduation GED or other equivalency only High school graduation only Attend or complete a 1- or 2-year program in a community college or vocational school Attend college, but not complete a Bachelor’s degree Obtain a Bachelor’s degree Obtain a Master’s degree or equivalent Obtain a Ph.D., M.D., or other advanced degree; Don’t know	This item performs well. 98% of respondents answered this question in F2, and 99% answered this question in F1.
LF1	ELS F2	P		How many of each of the following people live with you? If you live by yourself please indicate so. You live alone (yes/no) Your spouse Your partner in a marriage-like relationship Your mother or female guardian Your father or male guardian Friends or roommates (including girlfriends/boyfriends) Brothers or sisters (including adoptive, step, and foster siblings) Children (biological, step, or adopted) Others not already listed	This question can help researchers understand potential barriers to persistence in education and its related outcomes. This household size variable was used in the highly policy-relevant poverty status derived variable. 99% of F2 respondents answered this question.
LF2	n/a	N		Do you live in your parent/guardian’s home, or [do they / does he/she] live in your home? Response options: live in parent/guardian’s home; parent/guardian’s live in my home Before you continue to the next question, we would like to know if you had any difficulty understanding or answering this question so that we may improve it for future surveys. Did you have any difficulty such as not understanding question wording, being uncertain of the meaning of certain terms or response choices, or not having the information needed to answer the question? Response options: Yes; No If yes, please describe any difficulty you had. Please be as specific as possible. Response option: [textbox in which respondent can describe any difficulty]	Easy question to answer, and allows analysts to distinguish between respondents “still living at home” for financial or other reasons, and respondents essentially “on their own” who may have brought their parent(s) into their own home to care for them. We are including a same-screen follow-up asking respondent if there was any difficulty in answering question; responses should aid in assessing performance of question.
LF3	NELS F4	R	Now I would like to get some information about your current dependents. Excluding yourself, [and excluding your spouse / and excluding your partner,] how many dependents do you currently support? (A dependent is a person for whom you pay at least half their expenses (food, shelter, clothing, health care, and schooling). This may include your children, parents, or others. Note that a dependent does not have to live with you). Number of dependents	Now we would like to get some information about your current dependents. Excluding yourself, [and excluding your spouse / and excluding your partner,] how many of each of the following types of dependents do you currently support? Enter ‘0’ where appropriate. (A dependent is a person for whom you pay at least half their expenses (food, shelter, clothing, health care, and schooling). This may include your children, parents, or others. Note that a dependent does not have to live with you). Number of dependent children (less than age 18) Number of dependent adults (age 18 or older)	NELS F4 asked for total number of dependents, with a separate follow-up question asking how many of those dependents were children. For ELS F3 FT we are proposing capturing both pieces of information on one form. Revised question stem to include guidance for self-administered respondents: “Enter ‘0’ where appropriate”.
LF4	NELS F4	P		Next, I’m going to ask you a few questions about your family life. What is your current marital status? Response options: Single, never married; Married; Divorced; Separated; Widowed; Not married, but partnered with significant other in a marriage-like relationship	This measure performed very well. 99% of eligible respondents answered this question. Of interest to researchers is the degree to which this statistic changes over time and its relation to other outcomes of education. In NELS F4, 47% of then 26 year-old respondents indicated a status other than ‘Single, never married.’
LF5	n/a	N		What is the highest level of education your spouse/partner has completed?	Spouse’s education provides context for the respondent’s current SES in that the spouse’s educational returns (e.g. income, but ultimately cultural and economic returns) may be shared with the respondent and form their common environment.
LF5a	n/a	N		Have you ever been legally married? Response options: Yes; No	Including this question (which will only be administered to respondents who indicate their current marital status is “in a marriage-like relationship”) such that those respondents can be properly routed to or around the marriage date (LF8) and “how did your marriage end” (LF9) questions.
LF6	NELS F4	P		Have you been married more than once? Response options: Yes, No	This measure performed very well. 99% of eligible respondents answered this question.
LF7	NELS F4	P		How many times have you been married?	This measure performed very well. 99% of eligible respondents answered this question.
LF8	NELS F4	P		What is the month and year of your [first/second/etc] marriage? Month, Year	This measure performed very well. 99% of eligible respondents answered this question.
LF9	NELS F4	P		Did your [first/second/etc.] marriage end in a… Response options: Divorce or annulment; Permanent or legal separation; Death	This measure performed very well. 97% of eligible respondents (who had indicated that they had been legally married before) answered this question.
LF10	ELS F2	P		Have you had any biological children [, that is, children born to you/, that is, children for whom you are the natural father/mother]? Response options: Yes, No	Of interest to researchers as a traditional demographic marker of adulthood, and as a potential barrier to persistence in postsecondary education and/or employment. 99% of F2 respondents answered this question. Only 9% of F2 respondents indicated that they had biological children as of 2006, but this percentage should be higher in F3.
LF11	ELS F2	P		How many biological children have you had?	99% of eligible respondents (those who indicated biological children) answered this question.
LF12	ELS F2	P		In what month and year was your [first/second/etc.] biological child born? Month, Year	98% of eligible respondents answered this question.
LF13	n/a	N		At the time of your [first/second/etc.] biological child’s birth, were you married to or partnered with your child’s father? Before you continue to the next question, we would like to know if you had any difficulty understanding or answering this question so that we may improve it for future surveys. Did you have any difficulty such as not understanding question wording, being uncertain of the meaning of certain terms, or not having the information needed to answer the question? Response options: Yes; No If yes, please describe any difficulty you had. Please be as specific as possible. Response option: [textbox in which respondent can describe any difficulty]	Responses may provide further context as to whether parenthood complicated plans for postsecondary education and/or employment. For the field test, we are including a same-screen follow-up asking respondent if there was any difficulty in answering question; responses to this follow-up should aid in assessing performance of the new question.
LF14-LF16	HS&B	P	(see “Justification” field)	Have you ever adopted a child? How many children have you adopted? In what month and year did you [first/next] adopt a child?	HS&B (a paper questionnaire) asked “Do you have any children?” followed by “Please describe all your children” – a section which included a subset of questions (e.g. what is child’s gender/age/relationship to you – bio, adopted, etc.) which the respondent was to answer for each child. Modified this approach such that it works for a web-based instrument. These items performed very well. Between 95%-99% of eligible respondents answered these questions.
A1	NELS F4	P		Including all of the wages, salaries, and commissions you earned in 2010, about how much did you earn from employment before taxes and all other deductions?	This measure performed very well. 93% of eligible respondents answered this question.
A1a	HSLS/ B&B12	N		We understand that you may not be able to provide an exact number for your job earnings. However, it would be extremely helpful if you would indicate which of the following ranges best estimates how much you earned from employment prior to taxes and deductions in calendar year 2010. Please include all wages, salaries, and commissions you earned in 2010. Response options: Less than $1,000; $1,000-$2,499; $2,500-$4,999; $5,000-$9,999; $10,000-$14,999; $15,000-$19,999; $20,000-$29,999; $30,000-$49,999; $50,000 and above	This question is only administered to those respondents who fail to provide a response to the open-ended income question (A1). This approach was used in ELS F2, and is currently being used in HSLS. The question stem shown here is borrowed from the BY HSLS parent interview; the categories are drawn from B&B:12, since that population is (and their incomes are) more in-line with the ELS F3 population. Categories can be refined for FS based on FT responses to A1.
A2	HS&B	P	(see “comment” field)	From which of these sources did you [or your spouse/partner] receive income during 2010? Response options: You received income from this source; Your spouse/partner received income from this source (this option to be hidden if respondent does not indicate that they have a spouse); [Neither you nor your spouse/partner received / You did not receive] income from this source	HS&B asked this question separately for the respondent and for the respondent’s spouse; modified for ELS to ask in one form in hopes of cutting down on interview time.
A3	NELS F4	P		Next, about how much did your spouse/partner earn from employment before taxes and all other deductions in 2010? Please include all wages, salaries, and commissions.	This measure performed well. 88% of eligible respondents (who had indicated they had a spouse/partner) answered this question.
A3a	HSLS/ B&B12	N		We understand that you may not be able to provide an exact number for your [spouse’s/partner’s] job earnings. However, it would be extremely helpful if you would indicate which of the following ranges best estimates how much your [spouse/partner] earned from employment prior to taxes and deductions in calendar year 2010. Please include all wages, salaries, and commissions they earned in 2010. Response options: Less than $1,000; $1,000-$2,499; $2,500-$4,999; $5,000-$9,999; $10,000-$14,999; $15,000-$19,999; $20,000-$29,999; $30,000-$49,999; $50,000 and above	This question is only administered to those respondents who do not provide a response to the open-ended income question (A3). This approach was used in ELS F2, and is currently being used in HSLS. The question stem shown here is borrowed from the BY HSLS parent interview; the categories, however are drawn from B&B:12, since that population is (and their incomes are) more in-line with the ELS F3 population. Categories can be refined for FS based on FT responses to A1.
A4	NELS F4	P		Without considering any 2010 earnings from employment, approximately how much did [you / you and your spouse / you and your partner] receive from other sources of income in 2010? (These sources might include stocks and bonds, savings interest, insurance, alimony or child support, family members, and disability payments.)	This measure performed very well. 94% of eligible respondents answered this question, and a large percentage (26%) indicated other sources of income than employment.
A5	NLSY	P		Now I would like to ask you some questions about any pension or retirement savings. Many employers and unions have pensions or retirement plans, some provide tax-deferred plans such as thrift/savings, 401 K's, profit sharing or stock ownership plans. Additionally, individuals can provide for their own retirement with IRA or Keogh plans. Do [you/you or your spouse/you or your partner] have any savings in these types of plans?	Labor market outcomes of analytical interest include current financial status (such as retirement savings); collecting such information allows for linkage with school quality, attributes, and what the student learns in school.
A6	NLSY	P		What is the total value of all the savings [you have/you have in your own and you have jointly with your spouse/partner] in these types of plans?	Labor market outcomes of analytical interest include current financial status (such as retirement savings); collecting such information allows for linkage with school quality, attributes, and what the student learns in school.
A7	HSLS:09	P		Do you… Response options: Pay mortgage towards or own your home; Rent your home, or; Have some other arrangement?	This measure performed very well. There is data for 93% of BY parent respondents.
A8	NLSY	R	What is the present value of [your house/the entire building/your unit/this property]? That is, about how much would it bring if it were sold today?	What is the present value of your home? That is, about how much would it bring if it were sold today?	Revised question stem to simply reference ‘home’ since in ELS F3 we will not know the respondents’ exact residence type.
A9	AddHealth	P		About how much do [you/you and your spouse/you and your partner] owe on the mortgage for your residence? (If none enter ‘0’).	This measure performed very well. 97% of respondents who indicated they had a mortgage (40% of the sample) answered this question in Wave IV.
A10	AddHealth	P		Now, think about your debts [besides any mortgage on your home.] How much do you and others in your household owe altogether? Include all debts, including all types of loans [except mortgage loans], credit card debt, medical or legal bills, etc.	This measure performed very well. 93% of respondents answered this question in Wave IV.
A11	AddHealth	P		Suppose you and others in your household were to sell all of your major possessions (including your home), turn all of your investments and other assets into cash, and pay off all your debts. Would you have something left over, break even, or be in debt? Response options: 1=Have something left over; 2=Break even; 3=Be in debt	This measure performed very well. 96% of respondents answered this question in Wave IV.
A12	n/a	N		How much would you [have left over / be in debt]?	Easy to answer, yielding useful outcome information on debt following education.
CP1	NELS F4	P		Are you currently registered to vote? Response options: Yes; No; Ineligible to vote	This measure performed very well. 99% of eligible respondents answered this question. This question is very similar to questions asked in the ANES and NLSY.
CP2	NELS F4	R	[Even if you are not currently registered to vote], did you vote in the 1996 presidential election?	[Even if you are not currently registered to vote], did you vote in the 2008 presidential election?	Revised to a year appropriate for ELS F3 FT. This question is very similar to questions asked in the ANES and NLSY. This measure performed very well. 99% of eligible respondents answered this question.
CP3	NELS F4	P		In the last two years, have you voted in any local, state, or national election? Response options: Yes, No	This question is very similar to questions asked in the ANES and NLSY. This measure performed very well. 99% of eligible respondents answered this question.
CP4	ELS F2	P		During the past two years, have you performed any unpaid volunteer or community service work through such organizations as youth groups, service clubs, church clubs, school groups, or social action groups? Response options: Yes, No	This question is very similar to questions asked in the ANES and NLSY. This item performed well. 99% of respondents answered this question. Importantly, 44% indicated volunteer work or community service in the last 2 years.
CP5	ELS F2	P		Which of the following types of organizations have you been involved with in your unpaid volunteer or community service work during the past two years? (Check all that apply) Youth organization School or community organization Political organization Church-related group Neighborhood or social action associations Hospital or nursing home Education organizations Conservation/environmental group Other community or service work (please specify: )	This question is very similar to questions asked in the ANES and NLSY. This item performed well. 99% of eligible respondents answered this question.
CP6	ELS F2	P		During the past two years, how often did you spend time volunteering or performing community service? Response options: Less than once a month; At least once a month, but not weekly; At least once a week	This question is very similar to questions asked in the ANES and NLSY. This item performs well in the ELS version. 99% of respondents who indicated they had volunteered in the past 2 years answered this question.
CP7	UniLOA	P		On a scale of 1 to 5, where 1 means ‘never’ and 5 means ‘always’, please indicate how often you contribute financially to causes you believe in; for example, making financial donations to charities, organizations, and causes.	This item is part of a construct (“Citizenship”) which has a Cronbach’s alpha of .85. There was not enough space on the questionnaire to include the entire construct, but it could be expected to be highly reliable.
LV1	ELS F2	P		Since January 2005, how many times, if any, have any of the following happened to you? Response options: Has not happened; Has happened once; Has happened more than once Your parents or guardians got divorced or separated One of your parents or guardians lost his or her job You lost your job One of your parents or guardians died A close relative or friend died You became seriously ill or disabled A family member became seriously ill or disabled You were the victim of a violent crime	This measure performed very well. 99% of eligible respondents answered this question in F2. A longer version of this question was asked in NELS F1 which was answered by 96% of eligible respondents, and in NELS F2 was answered by 98% of eligible respondents.
LV2	n/a	N		[In what month and year did event X occur? / In what month and year did event X first occur? / In what month and year did event X last occur?]	Collecting dates for these significant “life events” will allow analysts to better examine their impact on education- and/or employment-related outcomes.
LV3	ELS BY/F1, NELS F1/F2/F3	P		How important is each of the following to you in your life? Response options: Not important; Somewhat important; Very important Being successful in your line of work Finding the right person to marry or partner with and having a happy family life Having lots of money Having strong friendships Being able to find steady work Helping other people in your community Being able to give your children better opportunities than you’ve had Living close to parents and relatives Getting away from this area of the country Working to correct social and economic inequalities Having children Having leisure time to enjoy your own interests Becoming an expert in your field of work Getting a good education	This item has a long history of performing well on secondary longitudinal studies. Percent of eligible respondents answering the question: ELS F1: 99% ELS BY: 95% NELS F3: 99% NELS F2: 99% NELS F1: 99%

December 2010

Education Longitudinal Study: 2002
(ELS:2002)

Third Follow-up 2011 Field Test

OMB Supporting Statement

Part D

Linkages to Extant Data Sources

OMB# 1850-0652 v.7

National Center for Education Statistics

Institute of Education Sciences

U.S. Department of Education

D.1 Develop Linkages with Extant Data Sources

RTI recognizes the value added to the ELS:2002 data with the addition of data from specific extant data sources. RTI will develop linkages with several existing data sources to supplement the student interview data. Because certain data (for example, specific financial aid amounts and associated dates) can only be accurately obtained from sources other than the student or parent. Through our experience providing data for many National Center for Education Statistics (NCES) studies, including previous ELS:2002 data collections, Baccalaureate and Beyond Longitudinal Study (B&B), National Postsecondary Student Aid Study (NPSAS), and Beginning Postsecondary Student (BPS) study, we have gained considerable knowledge performing file merges with these sources of valuable data, including Department of Education’s (ED) Central Processing (CPS) for Free Application for Federal Student Aid (FAFSA) data, the National Student Loan Data System (NSLDS), and the General Educational Development (GED) Testing Service. For this study, we propose to perform file merges with the following datasets: GED, CPS, and NSLDS.

The Family Educational Rights and Privacy Act (FERPA), (34 CFR Part 99) allows the disclosure of information without prior consent for the purposes of ELS according to the following excerpts: 99.31 asks “Under what conditions is prior consent not required to disclose information?” and explains in 99.31 (a) an educational agency or institution may disclose personally identifiable information from an education record of a student without the consent required by 99.30 if the disclosure meets one or more specific conditions. ELS collection falls under Sec. 99.31 (a)( 3). The disclosure is, subject to the requirements of Sec. 99.35, to authorized representatives of--

(i) The Comptroller General of the United States;

(ii) The Attorney General of the United States;

(iii) The Secretary; or

(iv) State and local educational authorities.

ELS is collecting data under the Secretary’s authority. The personally identifiable information is collected from student record systems with adherence to the security protocol detailed in 99.35: “What conditions apply to disclosure of information for Federal or State program purposes?”

(a)(1) Authorized representatives of the officials or agencies headed by officials listed in Sec. 99.31(a)(3) may have access to education records in connection with an audit or evaluation of Federal or State supported education programs, or for the enforcement of or

compliance with Federal legal requirements that relate to those programs.

(2) Authority for an agency or official listed in Sec. 99.31(a)(3) to conduct an audit, evaluation, or compliance or enforcement activity is not conferred by the Act or this part and must be established under other Federal, State, or local authority.

(b) Information that is collected under paragraph (a) of this section must:

(1) Be protected in a manner that does not permit personal identification of individuals by anyone other than the officials or agencies headed by officials referred to in paragraph (a) of this section, except that those officials and agencies may make further disclosures of personally identifiable information from education records on behalf of the educational agency or institution in accordance with the requirements of Sec. 99.33(b); and

(2) Be destroyed when no longer needed for the purposes listed in paragraph (a) of this section.

(c) Paragraph (b) of this section does not apply if:

(1) The parent or eligible student has given written consent for the

disclosure under Sec. 99.30; or

(2) The collection of personally identifiable information is

specifically authorized by Federal law.

Secure Data Transfers. NCES has set up a secure data transfer system, using their NCES member site with Secure Sockets Layer (SSL) technology, described above. We will use this electronic system for submitting data containing potentially identifying information (such as SSNs, names, and dates of birth of our sample members) along with their survey ID (not the same ID that is available on the restricted-use data). Before being transmitted, files will be encrypted using FIPS 140-2 validated encryption tools. We will receive data from the NCES system as well. The system requires that both parties to the transfer be registered users of the NCES Members Site and that their Members Site privileges be set to allow use of the secure data transfer service as described above. This process will be used for all file matching procedures described below, except in instances when the vendor already has a secure data transfer system in place.

D.2 File Merge with ED Central Processing System (CPS)

RTI proposes to perform file merges with the CPS data containing federal student aid application information. The merge with CPS can occur at any time for any number of cases, provided that the case has an apparently valid SSN associated with it. RTI sends a file to CPS and receives in return a large data file containing all students who applied for federal aid. We already have existing programs and procedures in place to prepare and submit files according to rigorous CPS standards. Similarly, we have already developed programs and procedures to receive and process data obtained from CPS.

RTI will electronically upload a file on the FAFSA secure web-site for matching which contains SSN and the first 2 letters of the sample member’s last name (but no other information). Access to the site for the upload is restricted to authorized users who are registered and provide identification/authentication information (SSN, DOB, and personal identification number [PIN]) to the FAFSA data site. The file is retrieved by the Central Processing System or CPS (the FAFSA contractor data system) for linkage. The linked file, containing student aid applications for matched records, is then made available to us only through a secure connection (EdConnect) which requires username and password. All CPS files will be processed, edited, and documented for inclusion on the analytic data files.

D.3 File Merge with National Student Loan Data System Disbursement

RTI will also conduct a file merge with the NSLDS to collect federal loan and Pell grant data. The resulting file will contain cumulative amounts for each student’s entire postsecondary education enrollment. We have programs to create the files for the merge and also programs to read the data we receive. All matching processes are initiated by RTI staff providing a file with one record per sample member. File transfers will use the NCES secure data transfer system described above.

D.4 File Merge with General Educational Development Testing Service

RTI will also conduct a file merge with the GED testing service, as was done in ELS:2002/06 to obtain GED test dates and results. This will extend the coverage for the ELS:2002 sample regarding attempts for high school equivalency credentialing. File transfers will use the NCES secure data transfer system described above.

D.5 Processing Extant Data

The data from all of these sources, as allowed by the vendor, will be delivered in the restricted-use data files and will be useful for creating derived variables. The variables – both direct and derived – will be documented thoroughly for the data files.

December 2010

Education Longitudinal Study: 2002
(ELS:2002)

Third Follow-up 2011 Field Test

OMB Supporting Statement

Part E

Field Test Response Propensity Modeling Experiment and Case Assignment

OMB# 1850-0652 v.7

National Center for Education Statistics

Institute of Education Sciences

U.S. Department of Education

Background of Approach

RTI, under a contract with NCES, is currently undertaking an initiative, modeled on the Responsive Design methodologies developed by Groves (Groves and Heeringa, 2006), to develop new approaches to improve survey outcomes that incorporate different responsive and adaptive features.

RTI has implemented several of these procedures on recent studies and has published preliminary results (Rosen, et al, 2011; Peytchev, et al., 2010). RTI’s experimental approach aims to reduce nonresponse bias by using multiple sources of data to produce models that estimate a sample member’s response propensity prior to and following the early phase of data collections. After sample members with the lowest response propensities are empirically identified, they are targeted with interventions in an attempt to encourage participation. While ELS has historically made strategic decisions on targeting cases (e.g. dropouts), this new approach developed for ELS uses more data and aims to produce more precise estimates of which cases, based on their likelihood of response, should be considered for special treatment. The response propensity approach developed for the ELS Third Follow-up Field Test (FT) calls for the estimation of sample members’ response propensity prior to the commencement of data collection. This approach has been developed to determine what benefit can be gained, in terms of response rate improvement and bias minimization, by implementing a protocol to target low propensity cases using data from prior waves of a longitudinal study.

The approach will be implemented experimentally, with a random half of the low propensity cases assigned to an experimental group and the other random half to a control group.

Prior to Data Collection, Estimate Sample Member Response Propensity

The first step of the experiment is to estimate an initial response propensity for each sample member using the complete data that is available for all sample members (including both questionnaire respondents and nonrespondents from prior rounds). The employed data come from the base year, first follow-up and second follow-up waves of ELS; from the survey variables as well as sampling frame and “paradata” or data which describe the survey interviewing process.

To estimate a case’s response propensity prior to the start of the ELS Third Follow-up Field Test, a sample member’s eventual response status in the ELS Second Follow-up was predicted. A logistic regression model was fitted with the sample member’s ELS Second Follow-up response status as the dependent variable. As independent variables, a range of information known for all respondents and nonrespondents from each prior wave including information from batch tracing activities were examined for significance. The following variables were considered as predictors of a sample member’s Second Follow-up response outcome: base year response status, first follow-up response status, whether the respondent ever refused, whether the respondent has ever scheduled an appointment, whether the respondent was classified as hard to reach, the number of calls made to the respondent in F2, high school completion status, parental level of education, high school type, urbanicity, dropout status, and the sample member’s postsecondary aspirations.

No information about the race, gender, or any other demographic characteristics of the sample members was used for prediction.

Results of Response Propensity Estimates

Significant predictors of a sample member’s Second Follow-up response status were: base year response status, first follow-up response status, whether an appointment was made with the respondent, whether the respondent had ever refused to participate in a wave, the number of calls placed to a respondent in the second follow-up, whether mother graduated from college, and whether the sample member took an AP class in high school.

Predicted probabilities derived from the logistic regression model were used to get an estimate of a case’s response propensity. Sample members above the median predicted probability are classified as high propensity, and those below the median as low propensity. In total, 528 cases are classified as high propensity and 527 as low propensity. For the implementation of the experiment, the 527 low propensity cases will be randomly split into experimental and control groups. The goal is to examine how well low propensity cases using prior wave data can be predicted and how these cases can be treated in terms of bias minimization.

Since low propensity cases assigned to the experimental group will receive treatment, of interest is how those cases are distributed according to their prior response status. Exhibit 1 shows the distribution.

Exhibit 1. Distribution of Low Propensity Cases by Prior Response Status

	BY Response Status		First Follow-up Response Status		Second Follow-up Response Status
	% Respondent	% Nonrespondent	% Respondent	% Nonrespondent	% Respondent	% Nonrespondent
Third Follow Up FT Low Propensity Cases	85% (447)	15% (80)	78% (412)	22% (115)	55% (292)	45% (235)
Third Follow Up FT High Propensity Cases	96% (507)	4% (21)	100% (528)	0% (0)	92% (488)	8% (40)

Note: Actual counts of cases in parentheses.

As shown in Exhibit 1, the low propensity cases consist of both respondents and nonrespondents in all prior waves of ELS. Also, high propensity cases are not limited to 2^nd follow-up respondents. A number of nonrespondents are classified as high propensity. This suggests that for ELS, prior round response status, while important may not be sufficient as a predictor of response outcome in the 3^rd follow-up and should not be the sole basis for partitioning cases into propensity categories.

Exhibit 2 shows the distribution of the case propensities across some demographic characteristics of interest. From the data, it is clear that the demographic distribution of the propensities approximates the distribution in the overall FT sample. There is no obvious skewing across these demographic characteristics.

Exhibit 2. Distribution of Response Propensities by Sample Member and High School Characteristics

	Percent (and number) of cases in FT Sample	Percent (and number) of Cases in High Propensity Category
Sample Member Characteristics
Male	50.3 (531)	47.2 (249)
White	55.0 (550)	58.9 (293)
Black	18.8 (188)	17.7 (88)
Hispanic	19.4 (194)	15.7 (78)
Asian	6.2 (62)	7.0 (35)
School Characteristics
Urban	40.1 (431)	38.3 (202)
Public	84.3 (889)	86.5 (457)

Treatment for Low Propensity Cases

The basic premise of the response propensity approach is to identify low propensity cases as early as possible and assign to them “special treatments.” In theory, treating low propensity cases in the same manner as high propensity cases is inefficient and possibly harmful to overall data quality. The special treatment for ELS FT low propensity cases is a higher incentive. The incentive level of low propensity experimental cases will be $45 at the start of data collection (weeks 1-9) and go up to $55 starting at week 10. High propensity and control group cases will be offered $25 until the 10^th week of data collection, when the incentive will go up to $35. Exhibit 3 outlines the timing and levels of the different incentives.

Exhibit 3. ELS FT Treatment Schedule

	High Response Propensity	Low Response Propensity
Week	All High Cases	Control Group	Experimental Group
1-9	$25	$25	$45
10+	$35	$35	$55

Analysis Strategy

The experimental results will be evaluated by examining how well the models predict response outcomes and by investigating whether the incentive treatment minimized bias. First, the response rates will be examined for groups defined by estimated response propensity, i.e., how well the assigned response propensities actually predict the survey outcome. Then it will be examined whether the variance of the response propensity, was lowered and whether the association between the response propensity and any chosen survey variables, , was reduced, thus minimizing nonresponse bias in survey estimates of means and proportions.

In parallel with the ELS F3 FT data collection, where experimental interventions are planned on cases with low predicted response propensities, we will conduct analyses with the ELS F2 main study data. These main study analyses, in addition to the F3 FT results, where we have intervened on low propensity cases during data collection, will provide a more complete picture of the potential utility of this approach as well as refine the model for the main study. For the main study, sample weights will be incorporated into these analyses and the model-building exercise.

RTI is testing the response propensity approach to minimizing nonresponse bias in several field test studies for the first time. While there are variations in the study populations and in the “intervention” used in each, the goal of the approach is the same: identify cases with low response propensity, implement a targeted intervention to increase response propensity, and then evaluate the data to assess the impact to nonresponse bias. We plan to review all propensity modeling experiment results, their relation to sample bias, and identify the best approaches for each study to yield the most benefits for data quality in full scale collections.

References

Groves, R. M., & Heeringa, S. (2006). Responsive design for household surveys: tools for actively controlling survey errors and costs. Journal of the Royal Statistical Society Series A: Statistics in Society, 169(Part 3), 439-457.

Rosen, J. A., Murphy, J. J., Peytchev, A., Riley, S., & Lindblad, M. (2011). The effects of differential interviewer incentives on a field data collection effort. Field Methods, 23, 24–36. (doi:10.1177/1525822X10383390)

Peytchev, A., Riley, S., Rosen, J. A., Murphy, J. J., & Lindblad, M. (2010). Reduction of nonresponse bias in surveys through case prioritization. Survey Research Methods, 4(1), 21–29. http://w4.ub.uni-konstanz.de/srm/article/view/3037.

1 Blumberg, S.J., and Luke, J.V. (December 2009). Wireless Substitution: Early Release of Estimates From the National Health Interview Survey, January–June 2009. National Center for Health Statistics. Available from http://www.cdc.gov/nchs/nhis.htm.