OMB83 C Memo for 2010 Change Request

MEMORANDUM OMB # 1850-0852 v.6

DATE: August 16, 2010

TO: Brian Harris-Kojetin, Shelly Martinez

Office of Information and Regulatory Affairs, Office of Management and Budget

FROM: Laura LoGerfo

National Center for Education Statistics

THROUGH: Kashka Kubzdela

National Center for Education Statistics

SUBJECT: HSLS:09 First Follow-up School Recruitment Procedures and Materials

              

The High School Longitudinal Study (HSLS:09) is a nationally representative, longitudinal study of more than 21,000 9^th graders as they proceed through secondary and postsecondary education. The first round of data collection occurred in the fall of 2009, and the next round of data collection is scheduled for the spring of 2012 when most students will be in 11^th grade. The field test that will verify the procedures and vet the instruments for this 11^th grade data collection begins March 2011. The field test will comprise a subsample of the base year field test schools with 24 of the 41 participating schools within five states (yielding approximately 500 participating students). The field test for the first follow-up (FFU) data collection will closely resemble the base year field test in approach, burden, and materials used. The brochure was changed slightly to exclude 12^th graders from the field test sample and update the field test schedule. The approach to student data collection will be the same: computer administration of both math assessment and survey in students’ schools. Schools will be asked to provide enrollment status information, a task already approved by OMB, and to provide an 11^th grade enrollment list from which students who were not 9^th graders in the U.S. will be identified as part of the sample freshening process. This change memo requests clearance of the first follow-up field test recruitment procedures and materials (included as an appendix).

Explanation of Gifts/Payments

This section discusses the monetary honoraria and incentives for the school, school coordinator, and students participating in the in-school data collection.

School Coordinators – As in the base year, we are proposing to offer one School Coordinator at each school a base honorarium of $100 with the opportunity to earn an additional $25 for achieving at least an 85% student participation rate or an additional $50 for achieving a student response rate of 92% or better at the school. The School Coordinator role is labor intensive and involves providing enrollment information about students sampled in the base year, an 11^th grade enrollment list from which a sample can be drawn for student freshening, and information about the students selected for freshening. Additionally, the School Coordinator arranges the logistics of the in-school session, distributes parental consent forms and tracks their return, and assists with the session on its scheduled date. This honorarium will not reimburse the school coordinator for his or her time, but will serve as a token of our appreciation for his or her significant efforts toward making the study a success.

School – School officials frequently asked “What’s in it for the school” throughout the recruitment effort in the base year field test and full scale studies. However, an experiment conducted in the field test of the utility of a $500 school-level incentive did not demonstrate that this incentive was effective. For the first follow-up field test, schools will be offered a modest token of appreciation with an estimated value of $50 per school. For the field test, this token will be in the form of a choice of 1-year science- or math-related magazine subscriptions for the school media center. During the field test recruitment effort, we plan to ask schools to suggest additional, alternative low-cost options that would be meaningful to the schools. We will use these suggestions to make a list of 4-5 options to present to main study schools, pending OMB approval, to assist with the school recruitment/retention effort for the first follow-up.

In-School Students – As in the base year field test, in-school student respondents will be offered a $10 incentive to participate. We have seen more students reluctant to leave class to participate in voluntary research and have proposed a $10 incentive per student for in-school participation to offset the perceived stress of missing class to take another assessment. Since the students received this same incentive during the last round, they will expect similar compensation for this phase of the study.

Burden

For the recruitment effort, the respondent burden is for school district staff and the school coordinator. School district staff will review the notification letter, and a subset of the school districts will review a research application. School coordinators at the schools will verify the enrollment status of the base year sampled students and prepare the 11^th grade enrollment list. Additional burden for the school coordinator will be realized at the data collection stage, which will be included in the full field test OMB package along with burden estimates for survey respondents (students, parents, school administrators, school counselors). Estimated burden for the school coordinator for school recruitment activities is shown in Table 1. The cost to school staff to coordinate the data collection effort is estimated at $20 per hour. For the school coordinator, the total estimated cost is $1,920 for the field test recruitment activities.

Table 1. Estimated burden for First Follow-Up Field Test recruitment

Respondent	Number of respondents	Average burden/response	Range of response times	Total burden (hours)
School District: Review Notification Letter	38	3 minutes	2-4 minutes	2
School District: Application Review	5	240 minutes	210-270 minutes	20
School Coordinator: enrollment lists, pre-data-collection logistics	24	240 minutes	120-360 minutes	96
Total	67			118

Methods to Maximize Participation

Procedures for maximizing response rates at the institution level are based on successful strategies and experiences from the base year of HSLS:09, predecessor studies (e.g., ELS:2002), and other similar studies. In this section, methods for maximizing response rates for school recruitment are discussed. Methods for maximizing response rates among students, parents, and school staff will be discussed in the subsequent submission to OMB.

Achieving high participation rates on voluntary school–based research studies has proven increasingly difficult in recent years. Recent experience has shown that many schools already feel burdened by mandated “high stakes” testing and, at the same time, are hampered by fiscal and staffing constraints. Moreover, there are roadblocks not only at the school level, but also at the district level, where research studies must comply with stringent requirements and must submit formal detailed applications (similar to IRB applications) before schools can even be contacted.

Additionally, in the 2009 base year data collection of HSLS:09, a large number of schools rescinded their agreement to participate during the recruitment and data collection periods due to various factors. The keystone of the plan to work with school districts and schools in this round is to build upon the relationships developed with school and district personnel in the base year, and to demonstrate the importance of the study while maintaining flexibility in negotiations with school districts and schools.

Sample materials to be sent to districts and schools are provided in appendix A. Recruitment would commence with a notification letter to school districts. The letter will thank the district for their support in the base year and inform them that base year participating schools will be contacted for the first follow-up activities. Where we know that a district requires a formal application to conduct research in the schools and our existing application has expired, a renewal application or new application, as appropriate, will be prepared and submitted. Once approved by the district, schools will receive a letter reintroducing the study, inviting the schools to participate, and initiating the enrollment status update and the 11^th grade enrollment list collection.

Within a few days of receiving the materials, a trained recruiter will contact the school district or school to discuss their participation in the study. Recruiters are hired for their knowledge, skill, and articulation with the proven ability to develop relationships with district and school contacts that will foster participation and persist throughout the in-school follow-ups for the longitudinal study.

As much as possible, burden will be shifted from the school to research staff. Possible ways of shifting the burden include scheduling survey administrations to best fit the school calendar, mailing consent forms directly to parents, providing compensation for time and/or help in completing forms, offering a session administrator to come to the school to compile sampling information, and having a session administrator coordinate all aspects of survey day (e.g., posting reminders, processing consents, and gathering students). These options have proven helpful on similar studies to gain cooperation in schools that express scheduling, burden, or staffing concerns.

To reduce costs, a subset of participating base year field test schools will participate in the first follow-up field test study with an expected yield of 24 participating schools and 500 participating students. It is planned to release all 41 participating base year field test schools and to recruit schools to yield 24 participating schools. Approximately four additional schools will be recruited to allow for the possibility of schools rescinding their participation prior to data collection. Once the targeted number of schools is recruited, recruitment efforts will cease for the field test. By initiating recruitment with each of the 41 schools, we expect to learn and benefit from our discussions with each of the participating base year field test schools to formulate strategies for overcoming challenges in the main study recruitment effort.

In the full scale study, students from schools that decline to participate in the in-school component of the study will be contacted outside of school to complete a questionnaire and assessment via web, telephone, or field interview. RTI will also attempt to collect a school administrator questionnaire and a school counselor questionnaire from school staff at schools that decline an in-school session for students. It is planned to test these out-of-school data collection procedures with just one refusing school in the field test.

Based on information obtained as part of the Enrollment Status Update, it is possible that a small set of schools may be added to the sample as “new” or “convenience” schools. New schools are defined as schools that have newly opened due to a change in the composition of the base year school (e.g., a base year school has been split into multiple schools) where a significant percentage of the students from the base year school have been moved en masse to the new school. New schools will be asked to participate following the same protocol as base year schools. Convenience schools are schools that may be contacted if four or more students from one or more base year schools have transferred to a particular school that was not in the base year sample. When it is determined that a transfer school has enough base year students enrolled to warrant contact, RTI will invite the school to conduct an in-school session with base year students who have transferred to the school. School staff will not be asked to complete a questionnaire at convenience schools.

Student Sample Freshening

The student target population for the HSLS:09 FFU includes all 9^th-grade students in study eligible schools as of fall 2009 (in the full scale; 2008 for field test) and all 11th-grade students enrolled in study eligible schools as of spring 2012 (in the full scale; 2011 for field test). To maximize coverage of the target population, RTI will “freshen” the base-year field-test and full-scale samples with new study-eligible 11th-grade students who were not eligible for the study during the base-year period (e.g., home schooled; out of the country; attending junior high schools without an 11th grade; incarcerated; not attending 9th grade in base year but has advanced/held back to the 11th grade in FFU). RTI will freshen the two HSLS:09 Base-Year student samples within approximately 24 base-year field test schools and all 944 base-year full-scale schools using a four-step process. Based on the experience in the Education Longitudinal Study of 2002¹, we project that the freshening procedure will add approximately 15 eligible students to the FFU field test sample and approximately 300 eligible students to the FFU full-scale sample. Freshening will not occur within any of the convenience schools.

Step 1: Specifications for Enrollment Lists. RTI recruiters will request from school coordinators an electronic list of all 11th grade students currently enrolled at the school containing the following information also obtained in the base year:

• unique student ID (e.g., school ID);

• full name;

• race/ethnicity²;

• sex³; and

• birth month/year⁴.

The protocol will follow the procedures developed and validated during the base year to ensure confidentiality of sample members’ information.

Step 2: Quality Assurance. The HSLS:09 sampling staff will examine the quality of the 11th grade enrollment lists. Critical to the sampling task is (i) a unique link to the 9th grade enrollment list and (ii) race/ethnicity information (if provided on the base-year list). If the critical items are missing or if enrollment counts by race/ethnicity fall outside a tolerable range from CCD or PSS data⁵, then the school coordinator will be recontacted to obtain updated information or to verify the enrollment information.

Step 3: Selection of Freshening Sample. After verifying the quality of the enrollment lists, the HSLS:09 sampling staff will sort the 11th-grade enrollment list in the same order as the 9th-grade enrollment within the race/ethnicity strata. Using a half-open-interval (HOI) technique (Kish 1965), all new students immediately following a sampled student on the 11th-grade enrollment list and also listed before a base-year non-sample student will be selected for the freshening sample. The following is a pictorial representation of this method:

Student ID	New Student?	Base-year Sample Student?	Freshening Sample?	FFU Sample?
1		‘yes		‘yes
2		‘no
3		‘no
4	‘yes	-
5		‘yes		‘yes
6		‘no
7		‘yes		‘yes
8	‘yes	-	‘yes	‘yes
9	‘yes	-	‘yes	‘yes
10		‘no

If the last student within a stratum was selected for the base-year study, then the HOI procedure continues at the top of the list in a circular fashion. The freshening-sample student records are assigned the base weight calculated for the listed base-year sample student.

Step 4: Freshening Sample Eligibility. The identification of the freshening-sample students will be relayed to the school coordinator through a secure vehicle that ensures confidentiality. To determine eligibility, RTI recruiters will obtain the following information:

• The student’s grade-level at the time of the base-year study.

• If the base-year grade level is nine, then the education status of the student (e.g., school, home schooled, out of the country, incarcerated, etc.) and whether the student attended or did not attend a school that also had 11^th grade.

• If applicable, the name and physical location of the U.S. school.

Parent name and contact information will be requested for those in the freshening sample identified as study-eligible.

1 ELS:2002, http://nces.ed.gov/surveys/els2002/.

2 As in ELS:2002 and the base year of HSLS:09, four categories of race/ethnicity (Hispanic, Asian, Black, and Other) are used to define the second-stage sampling strata. Note that to ensure adequate representation of the groups defined as Hispanic, Asian, and Black, the four race/ethnicity categories are used for sampling stratification only, not for reporting purposes. Data collection and reporting follow the standard 2 ethnicity and 5 racial categories.

3 Sex, as well as the race/ethnicity variable, is useful for calculating nonresponse weight adjustments.

4 Birth month and year was requested on the 9th-grade enrollment lists and will be used in the FFU to ensure a match between the two enrollment lists.

5 The CCD (Common Core of Data) and the PSS (Private School Survey) files were used as the first-stage sampling frame.