OMB83 MGLS 2017 OFT2 Tracking & MS Base-year Recruitment Change Memo

MEMORANDUM OMB # 1850-0911 v.15

DATE: June 9, 2017

TO: Robert Sivinski and E. Ann Carson

Office of Information and Regulatory Affairs, Office of Management and Budget (OMB)

FROM: Carolyn Fidelman

National Center for Education Statistics (NCES)

THROUGH: Kashka Kubzdela

National Center for Education Statistics (NCES)

SUBJECT: Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) Operational Field Test First Follow-up (OFT2) Tracking and Main Study Base-year Recruitment Change Request

The Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) is the first study conducted by the National Center for Education Statistics (NCES) to follow a nationally representative sample of students as they enter and move through the middle grades (grades 6-8). The data collected through repeated measures of key constructs will provide a rich descriptive picture of the academic experiences and development of students during these critical years and will allow researchers to examine associations between contextual factors and student outcomes. The study will focus on student achievement in mathematics and literacy along with measures of student socioemotional wellbeing and other outcomes. The study will also include students with disabilities for whom descriptive information on their outcomes, educational experiences, and special education services will be collected. Base-year data for MGLS:2017 will be collected from a nationally representative sample of sixth-grade students beginning in January 2018, with annual follow-ups beginning in January 2019 and in January 2020 when most of the students in the sample will be in grades 7 and 8, respectively. In preparation for the national data collection, referred to as the Main Study, the data collection instruments and procedures must be field tested. An Item Validation Field Test (IVFT) was conducted in the winter/spring 2016 to determine the psychometric properties of items and the predictive potential of assessment and survey items so that valid, reliable, and useful assessment and survey instruments can be developed for the Main Study. An Operational Field Test (OFT) began in January 2017 to test the near-final instruments and the recruitment and data collection processes and procedures in preparation for the Main Study. District and school recruitment for the Main Study began in April 2017. OMB approved the OFT data collection, Main Study recruitment, and OFT First Follow-up (OFT2) tracking materials and procedures in December 2016 with the latest change request approved in April 2017 (OMB# 1850-0911 v.11-14).

This request is to (1) utilize a more intensive refusal conversion approach to protect the validity of estimates based on MGLS:2017 data; (2) update instructions on school enrollment lists that pertain to inclusion of students with Individualized Education Plans (IEPs); (3) submit revised data collection materials to be used in the Operational Field Test First Follow-up (OFT2); and (4) extend the Main Study data collection period for parents and staff through July 2018. These revisions do not affect the estimated burden to respondents or the total cost to the federal government for MGLS:2017.

1. More intensive refusal conversion

The MGLS:2017 Main Study sampling plan is designed to achieve 900 participating schools distributed across 16 sampling strata. A sample of 3,710 schools was selected for MGLS:2017, and under the assumption that 70 percent of schools would agree to participate, a subsample of 1,236 of those 3,710 schools was selected for initial recruitment. The set of 2,474 schools not selected for initial recruitment defines a set of reserve schools from which additional schools will be selected for recruitment if the desired number of participating schools are not achieved in one or more of the 16 sampling strata.

A more intensive refusal conversion strategy will be employed during the recruitment of the initial set of 1,236 sampled schools to reduce the degree to which sampling of reserve schools is required and to help obtain the target number of students in the study’s focal disability groups set for the base year. The set of pending refusal schools will be reviewed and a subset of districts and schools will be selected to receive interventions designed to increase the likelihood of their participation.

During our recently completed Operational Field Test (OFT), school-level response was lower for schools classified as having a higher prevalence of students in the focal disability groups. The study sampling design identifies schools as either high prevalence or low prevalence with respect to the number of students in the focal disability groups. High prevalence schools are public schools where the expected number of students whose primary IEP is autism or emotional disturbance exceeds the 95^th percentile (or 17 students) across all schools in the MGLS:2017 Main Study sampling frame. Only 13 of 51 high prevalence public schools (25.5 percent) in our initial OFT sample agreed to participate. This compared with 17 of 36 low prevalence public schools (47.2 percent) in our initial OFT sample that agreed to participate (p < .036). Most of the refusals experienced among high prevalence schools were received at the district level, where 25 districts, representing 32 high prevalence schools, declined participation. At the district level, with our initial OFT sample, 11 of the 38 districts with high-prevalence schools (28.9 percent) allowed us to contact their schools about the study; 20 of 32 districts with low-prevalence schools (62.5 percent) allowed us to contact their schools (p < .005). Furthermore, OFT districts that declined participation for their schools have on average a higher percentage of high-prevalence schools (32 percent high-prevalence schools) than do OFT districts that allowed us to contact their schools for participation (20 percent high-prevalence schools). Given the rarity of students in two of the three focal disability groups – namely students whose primary IEP designation is autism or emotional disturbance – these refusal conversion strategies will be targeted to achieve higher response among those public schools classified as “high prevalence.”

In the OFT, schools were randomly assigned to a school-level participation incentive of a $200 monetary incentive or either a $400 monetary or non-monetary equivalent incentive. For high prevalence schools, if the district participated, the $400 incentive level yielded 9 participating schools of 11 (82 percent); the $200 level yielded 4 participating schools of 9 (44 percent). Given the apparent effectiveness of the additional $200 in increasing response, we propose providing a boost of an additional $200 to pending refusal schools. For each additional high-prevalence school that agrees to participate, we estimate bringing in on average an additional 8 students with autism or emotional disturbance IEP designation.

Although the $200 boost will be provided to schools, because the district is a gatekeeper and was a significant factor in nonresponse during the OFT, we propose to offer the additional $200 in districts that contain high prevalence schools. This is intended to encourage the district to open the door for us to contact schools directly, where we have greater likelihood of gaining cooperation. Upon OMB approval of this plan, we will identify all pending refusal districts containing high prevalence schools on an ongoing basis. We will offer all schools in those districts, including low prevalence schools, an additional $200 monetary incentive or $200 monetary equivalent in goods or services – for a total of $600 for the targeted schools – to achieve district participation. The purpose of offering consistent levels of incentives across schools in a district is to avoid treating schools in the same district differentially. We will also offer site visits and tele- or video conferences (which may include school administrators in addition to district officials) to explain the details of the study, to address any concerns that the districts and schools may have, and to improve the likelihood of district participation. We will compare the districts that receive the intensified effort (including the nonmonetary measures) to districts where schools received $400 to evaluate effectiveness.

2. School Enrollment List Instructions

Schools are asked to provide enrollment lists of all students in grade 6. The MGLS:2017 seeks to collect data on students with disabilities, and asks that schools indicate on the enrollment lists which students have IEPs, and whether they have been identified as students with autism, emotional disturbance, or specific learning disabilities. Some students with disabilities or IEPs may spend a portion of the school day receiving instruction at another school. We would like to clarify for schools that we want those students included on the enrollment lists, if enrolled at their schools, even if they are not at the school for the whole day. We have changed the instructions on the enrollment lists as shown in Table 1.

Table 1. Edits to the MGLS:2017 School Enrollment List Instructions Between OFT1 and Main Study Base Year

OFT1	Main Study Base Year
Request for Student Roster Information From Schools (MS1-S)
Not all students in your school are needed for participation. We will be selecting a sample. In order to select the sample, we are requesting a complete roster of the students currently enrolled in sixth grade at your school, including ungraded students whose primary IEP designations are one of the 3 focal IEP categories.	Not all students in your school will be asked to participate in the study. We will be selecting a sample. In order to select the sample, we are requesting a complete roster of the following: Students currently enrolled in sixth grade at your school Ungraded students at ages 11-13 currently enrolled at your school whose primary IEP designation is one of the three focal IEP categories: Autism, Emotional Disturbance, or Specific Learning Disability. Please include in your enrollment list students with an IEP classification of Autism, Emotional Disturbance, or Specific Learning Disability (whether in sixth-grade or in an ungraded setting) who are enrolled at your school, even if they may spend time at another school. These students may take academic or non-academic classes; or they may be in a “school within a school,” or a school co-located with your school.
Grade level	Grade level (6th or U=ungraded ages 11-13 and have a primary IEP classification of Autism, Emotional Disturbance, or Specific Learning Disability)
Appendix T Student Rostering Form
For each student currently enrolled in grade 6 in your school, please provide the below information. Please be certain to include all children in each grade. General Student Information: Grade 6	For each student currently enrolled in grade 6 at your school, please provide the below information. Please be certain to include all students currently enrolled in sixth grade at your school. Please include ungraded students at ages 11-13 currently enrolled at your school whose primary IEP designation is one of the three focal IEP categories: Autism, Emotional Disturbance, or Specific Learning Disability. Please include in your enrollment list students with an IEP classification of Autism, Emotional Disturbance, or Specific Learning Disability (whether in sixth-grade or in an ungraded setting) who are enrolled at your school, even if they may spend time at another school. These students may take academic or non-academic classes; or they may be in a “school within a school,” or a school co-located with your school. General Student Information: Grade 6 or U=ungraded

3. OFT2 Data Collection Materials

As previously approved by OMB (1850-0911 v.13), Operational Field Test First Follow-up (OFT2) will be conducted in spring 2018 in schools that participated in the OFT Base Year. Recruitment materials were approved as Appendix OFT1-W (OMB# 1850-0911 v.13). The current request is for a revision to the contacting materials to reflect a shorter student session (75 instead of 90 minutes) and the omission of the height and weight measurement collection and of the teacher and parent surveys. In addition, this request includes the incentives plan for OFT2.

OFT2 data collection activities will now include a 75-minute student session and a 40-minute school administrator survey. Schools included in in-school data collection activities, including the request for the administrator survey, during OFT2 will be OFT base-year schools, transfer schools, and destination schools that have 4 or more sampled students. Transfer- and destination-schools with fewer than 4 sampled students will not be asked to complete an administrator survey. Students participating in OFT2 at their school will be allowed to keep the earbuds used during the assessment as a token of thanks for their participation. Students who have left their base-year school and are unable to participate in a destination or transfer school will be invited to participate via web outside of school and will be offered $20 for their participation. The option of out-of-school self-administration proved valuable in the High School Longitudinal Study of 2009 (HSLS:09), during which 9 percent of the HSLS:09 first follow-up eligible student sample (and 11 percent of the respondents) participated outside of school via web self-administration. The monetary incentive offered to these students is designed to encourage them to incur the burden of participating in the study on their own time. OFT2 consists of a shorter student session and of fewer staff surveys than in OFT1. However, there is still considerable burden for the school to provide tracking information and to coordinate student sessions. We will offer schools $200 in cash or cash equivalent to encourage their participation in OFT2. As with the Base Year, school coordinators will be offered a $150 monetary incentive. They play an especially important role in the study and are critical to its success. The coordinator in each participating school will coordinate logistics with the data collection contractor; compile and supply enrollment status update information for each sampled student; communicate with teachers, students, and parents about the study to encourage their participation; distribute and collect parental consent forms; and assist the test administrator in ensuring that the sampled students attend the testing sessions.

These changes are listed in Table 2 and reflected in the attached Appendices OFT2-A – OFT2-H. These Appendices now also include a school administrator contacting letter (OFT2-B1), which was originally mistakenly omitted.

Table 2. Edits to the Last Approved (1850-0911 v.13) MGLS:2017 OFT Contacting Materials

Appendix #	Last Approved (1850-0911 v.13)	Revised (1850-0911 v.15)
OFT2-A	Data collection will consist of a 90-minute student survey and assessment, student height and weight measurements, and web surveys by principals, math and special education teachers/providers, and parents. As with the base year study, students will be asked to complete the data collection activities in the school on Chromebooks, tablet-like computers with touchscreen capability and an attached keyboard, provided by RTI. Participating students will receive a small token of appreciation for their participation. The school coordinator will receive $150 as a thank you for making the study successful in the school. Participating teachers will receive $20 for completing the questionnaire plus an additional $7 for each student about whom the teacher provides input. As a thank you for your school’s continued support of this important research, participating schools will receive <incentive fill>.	Data collection will consist of a 75-minute student survey and assessment, as well as a 40-minute web survey for the school administrator. As with the base-year study, students will be asked to complete the data collection activities in the school on Chromebooks, tablet-like computers with touchscreen capability and an attached keyboard, provided by RTI. Participating students will receive a pair of earbuds as a token of thanks for their participation. The school coordinator will receive $150 to thank them for making the study successful in the school. As a thank you for your school’s continued support of this important research, your school will receive $200.
OFT2-B1	n/a	New letter
OFT2-B2	Formerly: OFT2-B	Now: OFT2-B2
OFT2-F	MGLS:2017 data collection will be administered by RTI International on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. Data collection will occur on a date convenient for the school, anytime between January and May 2018, and will consist of a 90-minute student session that includes a student survey, student assessments, and height and weight measurements. A trained session facilitator will be sent to the school to conduct the student session. The student session will be conducted on Chromebooks, tablet-like computers with touchscreen capability and an attached keyboard, provided by RTI for students to use. School administrators will be asked to complete a 40-minute survey. The math teachers of students selected for the study will complete web-based questionnaires about their background and classroom characteristics (approximately 20 minutes), as well as answer questions about the skills and abilities of specific students in the study (about 10 minutes per student). The special education teacher/provider for selected students with an Individualized Education Program (IEP) will be asked to complete web-based questionnaires about their background and the special education services they provide (about 10 minutes), as well as answer questions about the skills and abilities of specific students in the study (about 25 minutes per student). Parents will be asked to complete a 40-minute survey. Participating students will receive a small token of appreciation for their participation. The person coordinating the session at the school will receive $150 as a thank you for their assistance. Teachers will receive $20 for completing the questionnaire and an additional $7 for each student about whom they provide input. Schools will receive <incentive> as a token of our appreciation for their participation. Parents will receive <incentive> for their participation.	MGLS:2017 data collection will be administered by RTI International on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. Data collection will occur on a date convenient for the school, anytime between January and May 2018, and will consist of a 75-minute student session that includes a student survey and assessments in reading, mathematics, and executive function. A trained session facilitator will be sent to the school to conduct the student session. The student session will be conducted on Chromebooks, tablet-like computers with touchscreen capability and an attached keyboard, provided by RTI for students to use. School administrators will be asked to complete a 40-minute survey. Participating students will receive a pair of earbuds as token of thanks for their participation. The person coordinating the session at the school will receive $150 to thank them for their assistance. Schools will receive $200 as a token of our appreciation for their participation.
OFT2-G	MGLS:2017 data collection will be administered by RTI International on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. Data collection will occur on a date convenient for the school anytime between January and May 2018 and will consist of a 90-minute student session that includes a student survey, student assessment, and height and weight measurements. A trained session facilitator will be sent to the school to conduct the student session. The student session will be conducted on Chromebooks, tablet-like computers with touchscreen capability and an attached keyboard, provided by RTI for students to use. School administrators will be asked to complete a 40-minute survey. The math teachers of students selected for the study will complete web-based questionnaires about their background and classroom characteristics (approximately 20 minutes), as well as answer questions about the skills and abilities of specific students in the study (about 10 minutes per student). The special education teacher for selected students with an Individualized Education Program (IEP) will be asked to complete web-based questionnaires about their background and the special education services they provide (about 10 minutes), as well as answer questions about the skills and abilities of specific students in the study (approximately 25 minutes per student). Parents will be asked to complete a 40-minute survey. Participating students will receive a small token of appreciation for their participation. The person coordinating the session at the school will receive $150 as a thank you for their assistance. Teachers will receive $20 for completing the survey and an additional $7 for each student about whom they provide input. Schools will receive <incentive> as a token of our appreciation for their participation. Parents will receive <incentive> for their participation.	MGLS:2017 data collection will be administered by RTI International on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. Data collection will occur on a date convenient for the school anytime between January and May 2018 and will consist of a 75-minute student session that includes a student survey and assessment. A trained session facilitator will be sent to the school to conduct the student session. The student session will be conducted on Chromebooks, tablet-like computers with touchscreen capability and an attached keyboard, provided by RTI for students to use. School administrators will be asked to complete a 40-minute survey. Participating students will receive a pair of earbuds as a token of thanks for their participation. The person coordinating the session at the school will receive $150 to thank them for their assistance. Schools will receive $200 as a token of our appreciation for their participation.
OFT2-H	MGLS:2017 data collection will be administered by RTI International on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. Data collection will occur on a date convenient for the school, anytime between January and May 2018, and will consist of a 90-minute student session that includes a student survey, student assessment, and height and weight measurements. A trained session facilitator will be sent to the school to conduct the student session. The student session will be conducted on Chromebooks, tablet-like computers with touchscreen capability and an attached keyboard, provided by RTI for students to use. School administrators will be asked to complete a 40-minute survey. The math teachers of students selected for the study will complete web-based questionnaires about their background and classroom characteristics (approximately 20 minutes), as well as answer questions about the skills and abilities of specific students in the study (about 10 minutes per student). The special education teacher for selected students with an Individualized Education Program (IEP) will be asked to complete web-based questionnaires about their background and the special education services they provide (about 10 minutes), as well as answer questions about the skills and abilities of specific students in the study (approximately 25 minutes per student). Parents will be asked to complete a 40-minute survey. Participating students will receive a small token of appreciation for their participation. The person coordinating the session at the school will receive $150 as a thank you for their assistance. Teachers will receive $20 for completing the survey and an additional $7 for each student about whom they provide input. Schools will receive <incentive> as a token of our appreciation for their participation. Parents will receive <incentive> for their participation.	MGLS:2017 data collection will be administered by RTI International on behalf of the National Center for Education Statistics (NCES), part of the U.S. Department of Education. Data collection will occur on a date convenient for the school anytime between January and May 2018 and will consist of a 75-minute student session that includes a student survey and assessment. A trained session facilitator will be sent to your school to conduct the student session. The student session will be conducted on Chromebooks, tablet-like computers with touchscreen capability and an attached keyboard, provided by RTI for students to use. School administrators will be asked to complete a 40-minute survey. Participating students will receive a pair of earbuds as a token of thanks for their participation. The person coordinating the session at your school will receive $150 to thank them for their assistance. Schools will receive $200 as a token of our appreciation for participating.

4. Extend Main Study Data Collection Period

The MGLS:2017 Main Study data collection was planned to be conducted between January and June 2018, but is now planned to be extended through July 2018 for staff and parent data collection to allow them sufficient time to respond, given that teacher and parent lists are submitted on a flow basis throughout the in-school data collection period.

2