Note to Reviewer

Note to Reviewer:

Per the terms of clearance OMB NO.: 1220-0109 dated 09/15/2005, we have provided the following information.

1) BLS will provide a report on research being conducted on response bias due to incentives.

OMB asked about our plans to evaluate the impact of response fees on response accuracy. We currently have a project underway to look at the accuracy of respondent answers and, as a follow-on to that effort, we will look at how respondent fees influence accuracy.

The fundamental problem we face in evaluating data quality is that we typically do not know the correct answers to the questions. Having secured the approval of BLS and the Ohio Department of Job and Family Services, the Center for Human Resource Research at Ohio State University has clearance to match earnings data from unemployment insurance (UI) wage records from Ohio with our NLS data archive. While administrative data are not guaranteed completely accurate, they offer a credibly low-variance estimate of the true earnings and job-holdings for our respondents and, for the purposes here, will serve as the “gold standard” for earnings data to which we will compare survey information.

This data analysis project has just started, with the first step being to match employers listed by the respondent in the NLS interview with the employers in the UI wage data. Although often straightforward, this matching is sometimes very difficult due to firm’s corporate entity names often differing from their doing-business-as names. For example, the respondent may report working for Kentucky Fried Chicken, but the UI data may show employment with Yum Brands, Inc.

Once we have done the match and assessed the degree to which interview data match administrative data, will move on to assessing whether the quality of the match differs by respondent fee, holding constant interviewer-assessed quality of cooperation by the respondent. Because all respondents receive an incentive payment, we will divide the sample, round by round, into groups by the size of the payment and then perform standard tests for the admissibility of pooling versus separate estimates by fee stratum.

2) BLS will investigate the use of technologies such as T-ACASI to improve the quality of reporting on sensitive items.

In 2004, the NLS program did a proof-of-concept experiment to implement a system using voice recognition to collect sensitive information over the telephone for the NLSY97 interview. Because the NLS program uses large-scale decentralized CATI interviewing when it does phone interviews, the system needed to be integrated with the virtual call center technology that some Voice over Internet Protocol (VoIP) systems provide. The key process is that the phone call had to be switched from the interviewer to the voice recognition robot, which proceeded to ask the questions and recover the answers, and then switch the respondent back to the interviewer at the end of the self-interview section so she could complete the interview. We designed and implemented the system in an NLSY97 pretest and it functioned quite well. However, we did not implement this system in the main survey because we would have needed to mount a mode effect experiment to insure the sensitive information was being collected in a way that did not produce an artificial change due to the change in interview mode. The self-interview component of the NLSY97 is so substantial that this change in mode would be a considerable undertaking, not only in terms of assessing mode effects, but insuring that the various parts worked together seamlessly. T-ACASI is likely at its best when one only needs to ask a few questions as opposed to ten minutes of questions. This longer length creates much greater risk for things to go wrong.

The amount of sensitive information in the NLSY79 is very minor. The women in the sample are old enough that there are few abortions to ask about, and we have de-emphasized drug use in recognition of the life cycle stage of the respondents, with more of an emphasis on health, which does not create self-report problems.

The general lack of sensitive questions in the NLSY79 argues against T-ACASI, and if we looked into T-ACASI in the NLSY97, we would want to substantially shorten the self-administered supplement.

3) BLS will work to refine the incentive structure to offer more equal incentives to all respondents and open the early bird incentive to all respondents.

Please refer to the Information Collection Request document, Supporting Statement, Section A.9—Payment to Respondents which states that,

‘As OMB requested last round, we will make an Early Bird offer to all main youth respondents. We request permission to increase the Early Bird fee to $70 for those offered $60 last round, and to increase the fee to $90 for those offered $80 last round (the $80 fee was offered to those in the $80 cell in the original 2000 experiment). Again, this small $10 increase reflects the need to raise incentives over time, reflecting increases in inflation and cost of living. As in 2004, all main survey members who were in the same family in 1978 will be offered the same fee amount.’