August 2010
Education Longitudinal Study: 2002
(ELS:2002)
Third Follow-up
Supporting Statement
Part B
Supporting Statement Request
For OMB Review
OMB# 1850-0652 v.5
Submitted by
National Center for Education Statistics
Institute of Education Sciences
U.S. Department of Education
Table of Contents
Section Page
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS B-1
B.2. Statistical Procedure for Collecting Information B-2
B.2.a. Third Follow-Up Field Test Sample Design B-2
B.2.b. Third Follow-Up Main Study Sample Design B-2
B.3. Methods for Maximizing Response Rates B-2
B.4. Tests of Procedures and Methods B-5
B.5. Reviewing Statisticians and Individuals Responsible for Designing and Conducting the Study B-6
Appendix 1. Affidavit of Nondisclosure Confidentiality Agreement 1-1
Appendix 2. Locating Update Letters and Forms 2-1
Appendix 3. IRB Approval Letter 3-1
List of Exhibits
B-1. Tracing and Sample Maintenance Schedule for the Third follow-up Sample B-4
B-2. Consultants on statistical aspects of ELS:2002 B-6
B-3. Other contractor staff responsible for conduct of ELS:2002 B-6
This submission requests clearance for the ELS:2002 third follow-up field test and full-scale study panel maintenance activities with sample members and their parents. The respondent universe is described in the first section below. The second section describes the sampling and statistical methodology proposed for the field test and main study. The other sections describe methods for maximizing response rates, the special tests of procedures and methods, and the statisticians and other persons responsible for designing and conducting the study.
The target populations of the third follow-up field test study are the 2001 high school sophomore and the 2003 high school senior class. The sophomore cohort comprises students who were enrolled in the 10th grade in spring term 2001, and the 12th-grade cohort comprises seniors in the spring term of 2003. The sophomore cohort population includes high school dropouts.
The target populations of the third follow-up full-scale study are the 2002 sophomore cohort and the 2004 senior cohort. The sophomore cohort consists of those students who were enrolled in the 10th grade in the spring of 2002, and the 12th-grade cohort consists of those students who were enrolled in the 12th grade in the spring of 2004. The former population includes students who dropped out of school between 10th and 12th grade.
For both field test and full-scale, the sampling frame for the third follow-up will consist of students who were enrolled in the 10th grade in the base-year study and students who were enrolled in the 12th grade in the first follow-up study. The sampling frame will exclude students who were ineligible to participate in either the base year or the first follow-up, such as deceased students. Post-tracing, the questionnaire will only be fielded to sample members who remain “within scope” and will exclude those who are out-of-scope. Those who will be considered out of scope for the third follow-up round include all those who are found to be out-of-country, institutionalized or incarcerated, or “newly” incapable (having suffered a major cognitive or physical impairment). Hostile refusals or students who have asked that their data be withdrawn from the study are eligible members of the sample frame, but will be treated as permanent nonrespondents and not given a chance of selection for being fielded. Such eligible but nonfielded cases will be accounted for in the weighting and count against the response rate.
The ELS:2002 third follow-up field test sample will comprise respondents in the base year and/or first follow-up field test who may or may not also have been second follow-up field test respondents. The ELS:2002 second follow-up field test sample members were initially selected for the sample either in the base year when they were 10th graders in 2001, or they were added to the sample as freshened students when they were in the 12th grade in 2003.
The size of the field test sample (the subset of cases actively to be traced) is 1,060.
The ELS:2002 third follow-up full-scale sample will consist of approximately 16,200 sample members who were sophomores in 2002 or seniors in 2004 or both. The procedures that will be employed will target an overall response rate of 90 percent.
Our plan to maximize response rates focuses on two related goals of the ELS:2002 third follow-up field test and full-scale data collection: (1) successful locating of sample members, and (2) a successful data collection.
The first goal is successfully locating and contacting sample members by successfully implementing a tracing and sample maintenance plan. The tracing and sample maintenance plan includes, initially, the use of batch tracing services without direct respondent contact. Subsequently, direct mailings to sample members or their parents will take place. Batch tracing is a relatively low-cost method of updating addresses and telephone numbers for sampled individuals. The two primary batch tracing services include National Change of Address (NCOA) and Phone Append. NCOA will provide updated addresses for sample members, especially those who have recently moved. Phone Append will confirm or update the telephone number matched to each sample member at their most current known address. Using the updated address information, the batch tracing activity will be followed with a direct mailing to sample members and their parents.
The inclusion of parents is an important support to the tracing and contacting goals of the study. Parents have been deeply involved in the study since its onset: parent permission was sought for student participation in the base year and first follow-up, and a parent survey was conducted in the base year. The age of the sample population makes it very likely that many of the sample members have recently gone through major life transitions. For students who entered postsecondary education (74% at the second follow-up), they are now more than 5 years past their initial entry point into postsecondary education and most students who persisted in their education are likely to have completed their certificate, associate’s degree, or bachelor’s degree programs. Sample members are unlikely to still be attending the undergraduate institution they were attending at the time of the second follow-up. Many of the full-scale students are now 23 or 24 years of age (and will be 25 or 26 years of age at the time of data collection) and have entered the workforce and started their careers. Some students have married and changed names, making the task of updating their locating information somewhat more difficult. Another consideration is that 38 percent of adults aged 18 to 24 and 46 percent of adults aged 25 to 29 live in wireless telephone-only households,1 creating a greater locating challenge. The majority of the ELS sample will fall in the former age group as we begin sample maintenance and the latter group as we begin data collection. For these reasons, we expect sample members’ parents to be a useful source for updating contact information for their children. Because parents’ residences will likely be more stable than their children’s, they will be very important to the tracing effort for a significant number of sample members. For this reason, the batch tracing activities and the sample maintenance mailings described in this plan will include parents. Also, study contacting materials will be produced in English, Spanish, and four Asian languages, to also facilitate contacts with parents whose first language is not English.
This sample maintenance mailing (see Appendix 2 for text, and Appendix 3 for IRB approval certification) will include a letter that thanks sample members for their previous participation in ELS:2002, explains the data collection schedule and procedures for the third follow-up, and requests sample members to provide updated contact information. There are three means by which sample members or their parents may provide the needed locating information. They will be asked to verify or update contact information provided on a Contact Information Sheet and mail it to RTI in the provided envelope, to log into the ELS:2002 survey Web site at NCES to provide the information, or to call an RTI toll free number and leave the information in a voice-mail message or with Help Desk staff.
The schedule for panel maintenance activities is sketched in Exhibit B-1 below.
Exhibit B-1. Tracing and Sample Maintenance Schedule for the Third follow-up Sample
Date |
Activity |
September 2010 |
NCOA, Phone Append, and CPS batch tracing of field test and full scale sample members. Accurint and FastData searches for sample members not located thus far. |
October 2010 |
Direct mailing requesting updating of contact information via toll-free line or website to field test and full scale sample members/parents. |
April 2011 |
NCOA, Phone Append, and CPS batch tracing of field test sample members in advance of data collection. Accurint and FastData searches for sample members not located thus far. |
May 2011 |
Initial Contact mailings requesting updating of contact information to field test sample members/parents. |
June 2011 |
Pre-CATI intensive tracing for field test sample. |
July – October 2011 |
Intensive tracing during CATI for field test sample. |
September 2011 |
NCOA, Phone Append, and CPS batch tracing of full scale sample members. |
October 2011 |
Direct mailing requesting updating of contact information via toll-free line or website to full scale sample members/parents. |
March 2012 |
NCOA, Phone Append, and CPS batch tracing of full scale sample members in advance of data collection. Accurint and FastData searches for sample members not located thus far. |
April 2012 |
Initial Contact mailings requesting updating of contact information to full scale sample members/parents. |
May 2012 |
Pre-CATI intensive tracing for full scale sample. |
June – December 2012 |
Intensive tracing during CATI for full scale sample. |
August – December 2012 |
Field interviewing/tracing for full scale sample. |
Given the need to achieve high response rates, the interviewer training will focus considerable attention on enlisting cooperation in CATI and CAPI data collection. A large portion of the interviewer training for the third follow-up will concentrate on the most effective techniques for increasing participation. The two most important techniques on which interviewers will be trained are maintaining interaction with sample members and tailoring their approach to address the specific situation or concerns of potential respondents. An important part of these efforts are not only to highlight the importance of ELS:2002, but also to emphasize the importance of each respondent’s participation in the third follow-up survey.
In addition, a streamlined informed consent Web/CATI/CAPI protocol will be used that RTI designed for the first follow-up CATI data collection. This informed consent procedure is abbreviated for respondents who have received a lead letter, since much of the information required for informed consent is included in the letter. The lengthy informed consent used in the first follow-up field test provoked negative reactions among respondents (or their parents) that led to refusals to participate.
A final possible element of data collection strategy will be offering respondents incentive payments for their participation in the third follow-up as was done, for example, in the second follow-up. An incentives plan will be documented in the field test clearance package that will be submitted to OMB later this year.
The key to achieving a high response rate in the third follow-up data collection will be combining all survey design elements into a comprehensive and effective strategy.
Many of the procedures and methods developed for the ELS:2002 field test study have been developed and employed in prior NCES studies such as NELS:88/2000 or BPS and similar postsecondary studies. Given the mobility of the youthful population for the study, these methods include locating protocols as well as data collection systems and methodologies.
A number of individuals have consulted with NCES and RTI on the design and analysis plans for the ELS:2002. Members of the Technical Review Panel have been described in an earlier section of this document. In addition, Dr. Jeffrey A. Owings, Associate Commissioner for the Elementary/Secondary and Library Studies Division, at NCES has reviewed and approved the statistical aspects of the study. Other statistical reviewers at NCES include the project officer, John Wirt; the NCES Chief Statistician, Marilyn Seastrom; and the Disclosure Review Board chair, Neil Russell. Exhibit B-2 provides the names of RTI consultants on statistical aspects of ELS:2002, while Exhibit B-3 lists other principal RTI professional staff assigned to the study.
Exhibit B-2. RTI Consultants on statistical aspects of ELS:2002
Name |
Affiliation |
James Chromy |
RTI |
Steven J. Ingels |
RTI |
Peter H. Siegel |
RTI |
Daniel J. Pratt |
RTI |
John Riccobono |
RTI |
David Wilson |
RTI |
Exhibit B-3. Other contractor staff responsible for conduct of ELS:2002
Name |
Affiliation |
Erich Lauff |
RTI |
Donna Jewell |
RTI |
Tiffany Mattox |
RTI |
Jim Rogers |
RTI |
Ben Dalton |
RTI |
1 Blumberg SJ, Luke JV. Wireless substitution: Early release of estimates from the National Health Interview Survey, January-June 2009. National Center for Health Statistics. December 2009. Available from http://www.cdc.gov/nchs/nhis.htm.
| File Type | application/msword |
| File Title | March 2010 |
| Author | RTI_Doc |
| Last Modified By | #Administrator |
| File Modified | 2010-08-19 |
| File Created | 2010-08-19 |