ELS-2002-3rd-Follow-up-2012-Field-Test-PartB

The respondent universe for the ELS:2002 third follow-up field test and full-scale study data collection is described in the first section below. The second section describes the sampling and statistical methodology proposed for the field test and main study. The other sections describe methods for maximizing response rates, the special tests of procedures and methods, and the statisticians and other persons responsible for designing and conducting the study.

B.1 Respondent Universe

The target populations of the third follow-up field test study are the 2001 high school sophomore and the 2003 high school senior class. The sophomore cohort comprises students who were enrolled in the 10th grade in spring term 2001, and the 12th-grade cohort comprises seniors in the spring term of 2003. The sophomore cohort population includes high school dropouts.

The target populations of the third follow-up full-scale study are the 2002 sophomore cohort and the 2004 senior cohort. The sophomore cohort consists of those students who were enrolled in the 10th grade in the spring of 2002, and the 12th-grade cohort consists of those students who were enrolled in the 12th grade in the spring of 2004. The former population includes students who dropped out of school between 10th and 12th grade.

For both field test and full scale, the sampling frame for the third follow-up consists of students who were enrolled in the 10th grade in the base-year study, or students who were enrolled in the 12th grade in the first follow-up study, and participated, at minimum, in either the base year or first follow-up rounds. The sampling frame will exclude students who are deceased or were consistently (across rounds) physically or mentally incapable of participation. Post-tracing, the questionnaire will only be fielded to sample members who remain “within scope” and will exclude those who are out of scope. Those who will be considered out of scope for the third follow-up round include all those who are found to be: out of country, institutionalized, incarcerated, or “newly” incapable (having suffered a major cognitive or physical impairment). However, such sample members may be in-scope for a fourth follow-up, should a subsequent round be funded. (For example, an expatriate sample member might repatriate, and thus become in-scope in the future.) Sample members who have asked that their data be withdrawn from the study are eligible members of the sample frame, but will be treated as permanent nonrespondents and not fielded. Such eligible but nonfielded cases will be accounted for in the weighting and count against the response rate.

B.2 Statistical Procedure for Collecting Information

B.2.a Third Follow-Up Field Test Sample Design

The ELS:2002 third follow-up field test sample will comprise respondents in the base year and/or first follow-up field test who may or may not also have been second follow-up field test respondents. The ELS:2002 second follow-up field test sample members were initially selected for the sample either in the base year when they were 10th graders in 2001, or they were added to the sample as freshened students when they were in the 12th grade in 2003.

The size of the field test sample (the subset of cases actively to be pursued for data collection) is 1,060. For purposes of cost containment, while collecting sufficient observations to evaluate questionnaire performance, the target yield for the first follow-up sample is approximately 500 individuals.

B.2.b Third Follow-Up Main Study Sample Design

The ELS:2002 third follow-up full-scale sample will consist of approximately 16,200 sample members who were sophomores in 2002 or seniors in 2004 or both. The procedures that will be employed will target an overall response rate of 90 percent.

B.3 Methods for Maximizing Response Rates

Our plan to maximize response rates focuses on two related goals of the ELS:2002 third follow-up field test and full-scale data collection: (1) successful locating of sample members, and (2) a successful data collection.

The first goal is successfully locating and contacting sample members by successfully implementing a tracing and sample maintenance plan. A successful locating effort is dependent on a multitude of factors including the characteristics of the population, the age of the locating information for the population, and the completeness and accuracy of that information. The locator database for the cohort includes critical tracing information for most of the sample members, including their previous residences and telephone numbers. Moreover, Social Security numbers are available for the majority of the sample members (89 percent of the field test sample members, 91 percent of the full scale sample members).

The tracing and sample maintenance plan includes, initially, the use of batch tracing services without direct respondent contact. Subsequently, direct mailings to sample members or their parents will take place. Batch tracing is a relatively low-cost method of updating addresses and telephone numbers for sampled individuals. The two primary batch tracing services include National Change of Address (NCOA) and Phone Append. NCOA will provide updated addresses for sample members, especially those who have recently moved. Phone Append will confirm or update the telephone number matched to each sample member at their most current known address. Cases whose contact information is not confirmed or updated by these steps will be sent for Accurint batch tracing. Using the updated address information obtained from these steps, the batch tracing activity will be followed with a direct mailing to sample members and their parents.

The inclusion of parents is an important support to the tracing and contacting goals of the study. Parents have been deeply involved in the study since its onset: parent permission was sought for student participation in the base year and first follow-up, and a parent survey was conducted in the base year. Parents have also previously been contacted for tracing and locating purposes. The age of the sample population makes it very likely that many of the sample members have recently gone through major life transitions. For students who entered postsecondary education (74 percent at the second follow-up), they are now more than 5 years past their initial entry point into postsecondary education and most students who persisted in their education are likely to have completed their certificate, associate’s degree, or bachelor’s degree programs. Sample members are unlikely to still be attending the undergraduate institution they were attending at the time of the second follow-up. Many of the full-scale students are now 23 or 24 years of age (and will be 25 or 26 years of age at the time of data collection) and have entered the workforce and started their careers. Some students have married and changed names, making the task of updating their locating information somewhat more difficult. Another consideration is that 38 percent of adults aged 18 to 24 and 46 percent of adults aged 25 to 29 live in wireless telephone-only households,^¹ creating a greater locating challenge. The majority of the ELS sample will fall in the former age group as we begin sample maintenance and the latter group as we begin data collection. For these reasons, we expect sample members’ parents to be a useful source for updating contact information for their children. Because parents’ residences will likely be more stable than their children’s, they will be very important to the tracing effort for a significant number of sample members. For this reason, the batch tracing activities and the sample maintenance mailings described in this plan will include parents. Also, study contacting materials will be produced in English, Spanish, and, on an at-need basis, up to four Asian languages, to also facilitate contacts with parents whose first language is not English. Panel maintenance activities to facilitate sample locating were described fully in an earlier submission to OMB, that obtained permission to contact sample members or their parents and are currently ongoing.

We will also conduct intensive in-house tracing at RTI during data collection for cases we cannot locate. The goal of intensive tracing is to obtain a telephone number at which the sample member can be reached. Tracing procedures may include (1) Directory Assistance for telephone listings at various addresses, (2) criss-cross directories to identify (and contact) the neighbors of sample members, (3) calling persons with the same unusual surname in small towns or rural areas to see if they are related to or know the sample member, and (4) contacting the current or last known residential sources such as the neighbors, landlords, and current residents of the last known address. Other more intensive tracing activities could include (1) database checks for sample members, parents, and other contact persons, (2) credit database and insurance database searches, (3) drivers’ license searches through the appropriate state departments of motor vehicles, (4) calls to colleges, military establishments, and correctional facilities to follow up on leads generated from other sources, (5) calls to alumni offices and associations, and (6) calls to state trade and professional associations based on information about field of study in school and other leads. The same vendors that were mentioned in the approved panel maintenance submission to OMB will be utilized.

Communication with sample members prior to and during data collection will promote cooperation by offering multiple methods for them to contact us (see Appendix 6 for contacting materials). The lead letters will provide the study website where sample members can find more information about the study and participating (by completing the questionnaire) via the Web. The letters will also provide telephone numbers respondents can use to contact RTI staff. Follow-up telephone calls will also be used to determine whether sample members have in fact received the materials we will have mailed to them or visited the study website. Contact materials and the web site have been designed so as to be attractive and informative in a manner that will enhance participation in ELS:2002.

For those with Internet access, the first opportunity will be to complete a self-administered Web questionnaire. Instructions for completing the questionnaire via the website will also need to be as clear and simple as possible to facilitate maximum web participation. Because it is not possible to ensure that all potential respondents fully understand the instructions or that computer glitches will not occur, we will also make it clear to potential web respondents that help desk staff will be available to them if and when they need them. Simply offering sample members a self-administered option is likely to increase response by allowing respondents greater discretion over how and when they can participate.

Despite the many advantages of offering a web survey mode, previous studies indicate that response rates are somewhat higher in interviewer-administered surveys than in self-administered surveys. Further methods of contact will include phone interviews (and in full scale in-person interviews) to increase contact with sample members over mail and Web contacts. Given the need to achieve high response rates, the interviewer training will focus considerable attention on enlisting cooperation. A large portion of the interviewer training for the third follow-up will concentrate on the most effective techniques for increasing participation. The two most important techniques on which interviewers will be trained are maintaining interaction with sample members and tailoring their approach to address the specific situation or concerns of potential respondents. An important part of these efforts are not only to highlight the importance of ELS:2002, but also to emphasize the importance of each respondent’s participation in the third follow-up survey. Exhibit B-1 summarizes the third follow-up full-scale data collection schedule and projected responses by survey mode.

Exhibit B-1. Third Follow-up Full-Scale Data Collection Schedule and Projected Response by Survey Mode

The use of cell phone calling and text messaging is a relatively new means for contacting sample members. Little research has been conducted on the effects of text messaging on participation rates. Research conducted by Brick et al. suggests that text messaging as a method of prenotifying sample members has nearly equal response rates as control group counterparts (Brick, Brick, Dipko, Presser, Tucker, and Yuan 2007). According to Lambries et al., those households using primarily cell phones required more attempts to contact than those using both landline and cell phones and those using landline only (Lambries, Link and Oldendick 2006). Households that used cell phones primarily showed differences of 1.1 more attempts than landline-only household and 0.8 more attempts than both landline and cell phone households.

However, text messaging has some advantages as the first means of contacting sample members. Text messaging may help identify working numbers and, in turn, increase the efficiency of the calling process (Steeh, Buskirk and Callegaro 2007). The research by Steeh et al. concludes that text messages have two advantages as the first means of contact: outcome rates are substantially improved and information about the working status of the number is obtained. Further research in this field is needed to better understand the effects of cell phone calling and text messaging on participation rates. The previously approved panel maintenance mailings include a question asking sample members if they would like to receive a text message when data collection is about to begin.

A final element of data collection strategy will be offering respondents incentive payments for their participation in the third follow-up as was done, for example, in the second follow-up. A propensity modeling experiment including differential telephone prompting schedules and incentive amounts are documented in Part A of this clearance package.

The key to achieving a high response rate in the third follow-up data collection will be combining all survey design elements into a comprehensive and effective strategy.

B.4 Tests of Procedures and Methods

Many of the procedures and methods developed for the ELS:2002 field test study have been developed and employed in prior NCES studies such as NELS:88/2000 or BPS and similar postsecondary studies. Given the mobility of the youthful population for the study, these methods include locating protocols as well as data collection systems and methodologies.

B.5 Reviewing Statisticians and Individuals Responsible for Designing and Conducting the Study

A number of individuals have consulted with NCES and RTI on the design and analysis plans for the ELS:2002. Members of the TRP have been described in Part A of this submission. In addition, Dr. Jeffrey A. Owings, Associate Commissioner for the Elementary/Secondary and Library Studies Division at NCES, has reviewed and approved the statistical aspects of the study. Other statistical reviewers at NCES include the project officer, John Wirt; the NCES Chief Statistician, Marilyn Seastrom; the Disclosure Review Board chair, Neil Russell; and Elise Christopher. Exhibit B-1 provides the names of RTI consultants on statistical aspects of ELS:2002, while Exhibit B-2 lists other principal RTI professional staff assigned to the study.

Exhibit B-2. RTI Consultants on Statistical Aspects of ELS:2002

Name	Affiliation
James Chromy	RTI
Steven J. Ingels	RTI
Daniel J. Pratt	RTI
John Riccobono	RTI
Peter H. Siegel	RTI
David Wilson	RTI

Exhibit B-3. Other Contractor Staff Responsible for Conduct of ELS:2002

Name	Affiliation
Ben Dalton	RTI
Donna Jewell	RTI
Erich Lauff	RTI
Tiffany Mattox	RTI
Jim Rogers	RTI

1 Blumberg, S.J., and Luke, J.V. (December 2009). Wireless Substitution: Early Release of Estimates From the National Health Interview Survey, January–June 2009. National Center for Health Statistics. Available from http://www.cdc.gov/nchs/nhis.htm.

File Type	application/msword
File Title	December 2010
Author	cannada
Last Modified By	#Administrator
File Modified	2011-02-15
File Created	2011-02-15