Supporting Statement B (1220-0157)

OMB Clearance - NLSY97 Round 12 Main Fielding

Supporting Statement

National Longitudinal Survey of Youth 1997 (NLSY97)

A Survey of Persons who were Ages 12 to 16 on December 31, 1996

Rationale, Objectives, and Analysis of Content

B. Collections of Information Employing Statistical Methods

1. Respondent Universe and Respondent Selection Method

This section summarizes the primary features of the sampling and statistical methods used to collect data and produce estimates for the NLSY97. Additional technical details are provided in the NLSY97 Technical Sampling Report, available online at http://www.nlsinfo.org/preview.php?filename=nlsy97techsamprpt.pdf.

Additional information about statistical methods and survey procedures is available in the NLSY97 User’s Guide at:

http://www.nlsinfo.org/nlsy97/docs/97HTML00/97guide/toc.htm

The initial sample was selected to represent (after appropriate weighting) the total U.S. population (including military personnel) 12 to 16 years of age on December 31, 1996. The sample selection procedure included an overrepresentation of blacks and Hispanics to facilitate statistically reliable analyses of these racial and ethnic groups. Appropriate weights are developed after each round so that the sample components can be combined to aggregate to the overall U.S. population of the same ages. Weights are needed to adjust for differences in selection probabilities, subgroup differences in participation rates, random fluctuations from known population totals, and survey undercoverage. Computation of the weights begins with the base weight and then adjusts for household screener nonresponse, sub-sampling, individual nonresponse, and post-stratification of the nonresponse-adjusted weights. The number of sample cases in 1997, the first round, was 8,984. Retention rate information for subsequent rounds is shown in the table below. BLS anticipates approximately the same retention rate in Round 12 that was attained in Round 9. (We assume modest declines in response rates in Rounds 11 and 12 relative to Round 10. Round 11 will not be complete until spring 2008). Only sample members who completed an interview in Round 1 are considered in-scope for subsequent rounds. Even if NORC is unable to complete an interview for an in-scope sample member in one round, they attempt to complete an interview with that sample member in each subsequent round. The interview schedule is designed to pick up crucial information that was not collected in the missed interviews.

The schedule and sample retention rates of past survey rounds are shown in Table 3.

Table 3. NLSY97 Fielding Periods and Sample Retention Rates

Round	Months conducted	Total respondents	Retention rate	Number of deceased sample members	Retention rate excluding the deceased
1	February–October 1997 and March–May 1998	8,984	—	—	—
2	October 1998–April 1999	8,386	93.3	7	93.4
3	October 1999–April 2000	8,209	91.4	16	91.5
4	November 2000–May 2001	8,081	89.9	15	90.1
5	November 2001–May 2002	7,883	87.7	25	88.0
6	November 2002–May 2003	7,898	87.9	30	88.2
7	November 2003–July 2004	7,755	86.3	37	86.7
8	November 2004–July 2005	7,503	83.5	45	84.0
9	October 2005–July 2006	7,338	81.7	60	82.2
10	October 2006–May 2007	7,555	84.1	77	84.8
11	October 2007-June 2008	7,4181	82.61	901	83.41

Note: The retention rate is defined as the percentage of base year respondents who were interviewed in a given survey year.

¹ Preliminary.

2. Design and Procedures for the Information Collection

The NLSY97 includes personal interviews with all living Round 1 respondents, regardless of whether they subsequently become institutionalized, join the military, or move out of the United States. We employ a thorough and comprehensive strategy to contact and interview sample members. At each interview, detailed information is gathered about relatives and friends who could assist NORC field staff in locating respondents if they cannot readily be found in a subsequent survey round. Every effort is made to locate respondents. Interviewers are encouraged to attempt to contact respondents until they reach them. There is no arbitrary limit on the number of call-backs.

Preceding the data collection, the NORC interviewers are carefully trained, with particular emphasis placed on resolving sensitive issues that may have appeared in the pretest and in prior rounds. Most of the NORC interviewers have lengthy experience in the field from having participated in earlier NLSY97 rounds as well as from involvement with the NLSY79 and other NORC surveys. All new recruits are given one day of personal training on general interviewing techniques, followed by three days of personal training on the questionnaire and field procedures. Experienced interviewers receive self-study training consisting of over 8 hours on specially designed materials requiring study of the questionnaire and procedural specifications, with exercises on new or difficult sections and procedures.

Field interviewers are supervised by NORC Field Managers and their associates. NORC has divided the U.S. into 10 regions, each supervised by a Field Manager who is responsible for staffing and for the quality of field work in that region. A ratio of 1 supervisor to 15 interviewers is the standard arrangement. Field Managers are, in turn, supervised by one of the two Field Project Managers.

The interview content is prepared by professional staff at BLS, CHRR, and NORC. When new materials are incorporated into the schedule, special assistance is generally sought from appropriate experts in the specific substantive area.

Because sample selection took place in 1997 in preparation for the baseline interview, sample composition will remain unchanged.

3. Maximizing Response Rates

A number of the procedures that are used to maximize response rate already have been discussed in items 1 and 2 above. The other component of missing data is item nonresponse. Nonresponse is comprised of respondents refusing to answer or not knowing the answer to a question. Almost all items in the NLSY97 have low levels of nonresponse. For example, in prior rounds there was virtually no item nonresponse for basic questions like the type of residence respondents lived in (YHHI-4400) or the highest grade of school respondents had ever attended (YSCH-2857).

Cognitively harder questions like, “How many hours did you work per week?” (YEMP-23901) have low levels of nonresponse. In the hours per week example, 6 individuals out of 2,810 (0.2%) did not answer the question in Round 8.

Sensitive questions have the highest nonresponse. Table 4 presents examples of round 8 questionnaire items that are most sensitive or cognitively difficult. Even very personal questions about sex have low rates of nonresponse. The top two rows of the table show that the vast majority of respondents (over 90%) were willing and able to answer the question, “Did you ever have sexual intercourse?” The third row shows that only 1.4% of respondents did not respond to the question on marijuana usage since the last interview. The fourth row shows that very few respondents (0.8%) did not answer whether they had carried a handgun since the last interview. Lastly, almost all respondents (1.0% nonresponse rate) were willing to reveal whether they had earned money from a job in the past year, but many did not know or refused to disclose exactly how much they had earned (2% nonresponse rate). Because high nonresponse rates were expected for the income amount question, individuals who did not provide an exact answer were asked to estimate their income from a set of predetermined ranges. Including these range answers reduces nonresponse by 1,454 cases, effectively lowering the total nonresponse rate for this question from 26% to 1.2%.

Table 4. Examples of Nonresponse Rates for Some Round 6 Sensitive Questions

Q Name	Question	Number Asked	Number Refused	Number Don’t Know	% Nonresponse
YSAQ2-299	Ever Have Sexual Intercourse?1	950	67	5	7.6%
YSAQ2-299B	Have Sex Since Date of Last Interview?2	2149	189	27	9.0%
YSAQ-370C	Use Marijuana Since Date of Last Interview?	7,358	82	23	1.4%
YSAQ-380	Carry a Handgun Since Date of Last Interview?	7,415	40	16	0.8%
YINC-1400	Receive Work Income in 2003?	7,429	22	52	1.0%
YINC-1700	How Much Income from All Jobs in 2003?	5,990	33	1,464	25.0%
YINC-1800	Estimated Income from All Jobs in 2003?3	1,497	24	94	7.9%

¹ Asked of respondents who have never reported having sexual intercourse in a previous interview and who do not report a spouse or partner in the household.

² Asked of respondents who have previously reported having sexual intercourse who do not report a spouse or partner in the household.

³ Asked of respondents who were unable or unwilling to answer the previous question (YINC-1700).

The Round 10 questionnaire included a section entitled ‘Tell Us What You Think” through which BLS hoped to learn information about how questionnaire designers might improve respondents’ perceptions of the interview. Attachment 8 provides a brief report on the results of those experimental questions.

4. Testing of Questionnaire Items

BLS is cautious about adding items to the NLSY97 questionnaire. Because the survey is longitudinal, poorly designed questions can result in flawed data and lost opportunities to capture contemporaneous information about important events in respondents’ lives. Poorly designed questions also can cause respondents to react negatively, making their future cooperation less likely. Thus, the NLSY97 design process employs a multi-tiered approach to the testing and review of questionnaire items.

When new items are proposed for the NLSY97 questionnaire, we often adopt questions that have been used previously in probability sample surveys with respondents resembling the NLSY97 sample. We have favored questions from the other surveys in the BLS National Longitudinal Surveys program to facilitate intergenerational comparisons. We also have used items from the Current Population Survey, the Federal Reserve Board’s Survey of Consumer Finances, the National Science Foundation-funded General Social Survey, and other Federally funded surveys.

All new questions are reviewed in their proposed NLSY97 context by survey methodologists who consider the appropriateness of questions (reference period, terms and definitions used, sensitivity, and so forth). Questions that are not well-tested with NLSY97-type respondents undergo cognitive testing with convenience samples of respondents similar to the NLSY97 sample members. During Round 12 questionnaire development, for example, cognitive testing of a proposed module pertaining to fertility expectations revealed significant comprehension and interpretation difficulties among a convenience sample of 8 individuals in the NLSY97 age group. As a result of this testing, the module was excluded from the Round 12 interview.

Existing questions are also reviewed each year. Respondents age and their life circumstances change, as does the societal environment in which the survey is conducted. Reviews of the data help us to identify questions that may cause respondent confusion, require revised response categories, or generate questionable data. Sources of information for these reviews include the questionnaire response data themselves, comments made by interviewers or respondents during the course of the interview, interviewer remarks after the interview, interviewer inquiries or comments throughout the course of data collection, other-specify coding, and comparison of NLSY97 response data to other sources for external validation. We also watch carefully the “leading edge” respondents, who answer some questions before the bulk of the sample – for example, the first respondents to attend graduate school or to get a divorce. These respondents are often atypical, but their interviews can reveal problems in question functionality or comprehensibility.

A comprehensive pretest is carried out approximately four months preceding each round of the main NLSY97 to test survey procedures and questions. This pretest includes a heterogeneous sample of about 200 respondents of various racial, ethnic, geographic, and socio-economic backgrounds. On the basis of this pretest, the various questionnaire items, particularly those being asked for the first time, are evaluated with respect to question sensitivity and validity. When serious problems are revealed during the pretest, the problematic questions are deleted from the main NLSY97 instrument.

Although further edits to questionnaire wording are extremely rare, we monitor the first several hundred interviews each round with particular care. Based on this monitoring, field interviewers receive supplemental training on how best to administer questions that seem to be causing difficulty in the field or generating unexpected discrepancies in the data.

Round 12 questions that have not appeared in previous rounds of the NLSY97 include:

Internet job search questions in the Employment section. These questions are asked of a new reference period but repeat questions that have been asked in all prior interviews. There is a new question about usage of e-mail or websites. This question previously was a scripted probe that appeared on the questionnaire screen and in help screens, but it now appears as a separate questionnaire item.. There also is a new question about employer application requirements.

Internet usage questions in the second Self-Administered section. These questions have been refreshed from prior rounds. New items have been adopted from the Pew Internet and American Life Project, primarily its February-March 2007 Tracking survey (information available at http://www.pewinternet.org/dataset_display.asp?r=64).

Color of skin question in the Interviewer Remarks section. The Round 12 questionnaire includes an item in the interviewer remarks section asking the interviewer to code the respondent’s skin color on a scale from 0 to 10. This item has previously appeared in the New Immigrant Survey (http://nis.princeton.edu/) in the U.S. and in data collections in Latin American countries. Information about the New Immigrant Survey implementation of this question is available at http://nis.princeton.edu/downloads/NIS-Skin-Color-Scale.pdf. The New Immigrant Survey was conducted with funding from many Federal agencies, including the Department of Health and Human Services, the National Science Foundation, and U.S. Citizenship and Immigration Services.

Correctional settings questions embedded in the first Self-Administered section. Items were excerpted from a variety of surveys being conducted by the Urban Institute and the Research Triangle Institute, and by sociologist Bruce Western of Harvard University. The Urban Institute and the Research Triangle Institute are jointly conducting a multi-site evaluation of a federal re-entry initiative. An overview of that effort is available at: https://www.svori-evaluation.org/%5Cdocuments%5Creports%5CRRIA-SVORI_Overview.pdf. Additional details, including numbers of cases completed, can be found at the evaluation’s web-site:

https://www.svori-evaluation.org/.

Professor Western has conducted a study of parolees in Brooklyn. Results of his study are discussed in Charles J. Hynes’s “ComALERT: A Prosecutor’s Collaborative Model for Ensuring a Successful Transition from Prison to the Community” appearing in the Journal of Court Innovation (available at http://www.courtinnovation.org/_uploads/documents/ComALERT.pdf, see page 141 and forward).

Religion questions. The NLSY97 has included questions about religious identification and attendance in most rounds. The Round 12 questionnaire includes three questions on religion that have not previously been asked in the NLSY97. These questions are supported by funding from the John Templeton Foundation received by Professor Brad Wilcox of the University of Virginia. The self-administered section includes a questions about the importance of religious faith in daily life. This question comes from the National Survey of Youth and Religion, conducted by the University of North Carolina, Chapel Hill (wave 2 telephone survey, item F:1). Information about the National Survey of Youth and Religion, including fielded questionnaires and publications using the data, is available at http://www.youthandreligion.org. That site also describes 30 pilot interviews done during the questionnaire-design phase of the survey. Two other questions ask about respondents’ self-identification as born-again or evangelical, and about fundamentalism. The question about being born-again or evangelical is asked in the NLSY79 Young Adult questionnaire in 2008. It has been widely used in many other surveys, as documented on the Association of Religion Data Archives (http://www.thearda.com/Archive/). The questions on being born-again/evangelical and on fundamentalism have been used widely, including by the Pew Center (for example, in their 2001 Religion and Public Life Survey), the 2005 Baylor Religion Survey, and ABC News/Washington Post polls.

Self-description questions in the ‘Tell Us What You Think’ section. The Ten-Item Personality Inventory has been administered in the NLSY79 Young Adult questionnaire, and in other settings (see, for example: Gosling, et al, 2003, “A very brief measure of the Big-Five personality domains,” Journal of Research in Personality, 37, pp. 504-528.) Additional personality questions were selected by psychologists Angela Duckworth (University of Pennsylvania) and Sasha Chernyshenko (University of Canterbury, New Zealand) through empirical work they have done on previous administrations of broader sets of personality-related items. An extensive website documenting the Ten-Item Personality Inventory and literature using it is found at:

http://homepage.psy.utexas.edu/homepage/faculty/gosling/scales_we.htm#Ten%20Item%20Personality%20Measure%20(TIPI).

The industriousness and conscientious measures have all been previously administered in the U.S. and elsewhere. Chernyshenko, based on his empirical research, advised the BLS on which of the large pool of available items would be most reasonable to include. His research on this topic includes:

Chernyshenko, O.S., and Stark, S. (in press), “The Sixteen Factor Personality Questionnaire,” Encyclopedia of Career Development, London, Sage Publications.

Roberts, B., Chernyshenko, O.S., Stark, S., and Goldberg, L. (2005), “The construct of conscientiousness: The Convergence between Lexical Models and Scales Drawn from Six Major Personality Questionnaires,” Personnel Psychology, 58, pp. 103-139.

Roberts, B.W., Bogg, T., Walton, K., Chernyshenko, O.S., and Stark, S. (2004), “A Lexical Investigation of the Lower-order Structure of Conscientiousness, Journal of Research in Personality, 38, pp. 164-178.

Chernyshenko, O. S., Stark, S., Chan, K. Y., Drasgow, F., and Williams, B. A. (2001), “Examining the fit of IRT models to personality items,” Multivariate Behavioral Research, 36, pp. 523-562.

Duckworth has written using existing data, for example, in Borghans, L., Duckworth, A.L., Heckman, J.J., and ter Weel, B. (in press), “The Economics and Psychology of Personality Traits,” Journal of Human Resources.

A list of all changes to the NLSY97 questionnaire from rounds 11 to 12 is shown in attachment 7.

5. Statistical Consultant

Dr. Kirk Wolter

NORC

55 East Monroe Street

Chicago, IL 60603

(312) 759-4000

The sample design was conducted by NORC, which continues the interviewing fieldwork.