OMB83C Change Memo v19

Memorandum United States Department of Education

Institute of Education Sciences

National Center for Education Statistics

DATE: April 18, 2018

TO: Robert Sivinski, OMB

THROUGH: Kashka Kubzdela, NCES

FROM: Carolyn Fidelman, NCES

SUBJECT: Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) Main Study Base Year (MS1) Out-of-School Change Request (OMB# 1850-0911 v.19).

The Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) is the first study conducted by the National Center for Education Statistics (NCES) to follow a nationally representative sample of students as they enter and move through the middle grades (grades 6-8). The data collected through repeated measures of key constructs will provide a rich descriptive picture of the academic experiences and development of students during these critical years and will allow researchers to examine associations between contextual factors and student outcomes. The study will focus on student achievement in mathematics and literacy along with measures of student socioemotional wellbeing and other outcomes. The study will also include students with disabilities for whom descriptive information on their outcomes, educational experiences, and special education services will be collected. Base-year data for MGLS:2017 will be collected from a nationally representative sample of sixth-grade students beginning in January 2018, with annual follow-ups beginning in January 2019 and in January 2020 when most of the students in the sample will be in grades 7 and 8, respectively. In preparation for year two of the national data collection, referred to as the Main Study, the data collection instruments and procedures must be further field tested. An Item Validation Field Test (IVFT) was conducted in the winter/spring 2016 to determine the psychometric properties of items and the predictive potential of assessment and survey items so that valid, reliable, and useful assessment and survey instruments could be developed for the Main Study. An Operational Field Test (OFT) was conducted between January and May 2017 to test the near-final instruments and the recruitment and data collection processes and procedures in preparation for year one of the Main Study. District and school recruitment for the Main Study began in April 2017. OMB approved the Main Study Base Year (MS1), Operational Field Test First Follow-up (OFT2), and Tracking and Recruitment for Main Study First Follow-up (MS2) in October 2017 with the latest change requests in January 2018 (OMB# 1850-0911 v.16-18).

This request is (1) to add out of school data collection opportunities for students in schools that are unable to participate and (2) to revise panel maintenance materials for parents. The requested changes do not affect the approved overall estimated respondent burden nor the total cost to the federal government.

(1) Out of school data collection. To mitigate school nonresponse, we plan to offer schools the option of providing the student list and contact information so that MS1 students may be contacted outside of school to participate in the study. Students invited to participate outside of school will be asked to take about 55 minutes to complete the student survey, the math assessment, and the first stage of the reading assessment. The second stage of the reading assessment, the executive function activity, and the height and weight collection would not be administered to students participating outside of school. Data collection protocols will follow those previously approved for OFT2 (OMB# 1850-0911 v. 16). All contacts with the student will go through the parent. As part of this submission, we have added (a) a letter to the parents of students who will be asked to participate out of school (appendix MS1-G4), which will include (b) an envelope with a letter to the student (appendix MS1-G4a) providing information on how to log into a student session, and we added (c) email reminders (appendices W-5a through W-5c). Students will be offered a check for $20 for participation outside of school.

(2) Revisions to the panel maintenance materials. We revised the panel maintenance form used in OFT prior to implementing the panel maintenance activities for the Main Study. The changes include removal of the request for student address; addition of a checkbox to the parent/guardian address information for the parent to indicate the student’s primary residence; and moving the student’s school enrollment status items to the start of the form. These changes are recommended based on questions received from parents completing panel maintenance materials and to reduce submissions of incomplete information.

The edits made to the last approved MGLS:2017 clearance documents to reflect these changes are listed below. Text marked in red font below reflects the additions and revisions that were made.

Part A

Text was added to section A.3 Use of Improved Information Technology (page 5) to reference the web-based student data collection:

The parent and school staff questionnaires will be fielded as web surveys. Web surveys will also be used for MS1 students in schools that only allow an “out-of-school” data collection and for OFT2 students that require “out-of-school” data collection because they attend schools with fewer than 4 sample members.

In section A.9 Payments or Gifts to Respondents, in Table 2 (page 9) and in the associated text (page 10), text was added to reflect the addition of out-of-school data collection and the associated student incentive to be offered:

Instrument/Activity	Administration Time*	MS1 and OFT2** Incentives
MS1 Data Collection
Student return of parent consent forms (explicit consent schools only)	10 minutes	Food event at school (e.g., pizza, bagels, etc.) sponsored by the study
Student Assessments and Survey – In-school administration (Mathematics, Reading, Executive Function, Height, Weight, & Survey)	90 minutes	Earbuds used during assessment, plus choice of (average value $0.50 each) 1) mechanical pencil, 2) mobile device screen cleaner, 3) sun catcher, 4) slap bracelet
Student Assessments and Survey – Out-of-school administration (Mathematics, Reading, & Survey)	55 minutes	$20

Main Study Base Year (MS1): A choice of up to four token incentive items will be offered to students participating in the Main Study data collection in school. In addition, students will be allowed to keep the earbuds used during the assessment. For students at schools where explicit parental permission is required (i.e., return of parental consent form), students who return the form by a set date will be offered a food event sponsored by the study (e.g., pizza, bagels, etc.).

To increase response rates, schools that cannot permit an in-school session will be offered the option of providing student roster information and having the students participate outside of school. As was approved for OFT2 (OMB# 1850-0911 v.15), students participating in MGLS:2017 MS1 outside of school will be offered $20 to encourage their participation in the study on their own time.

In section A.12 Estimates of Burden, text was added to describe the out-of-school student session (page 16) and Table 6 was updated to reflect the difference in assessment length between the in-school vs. out-of-school participants:

• The in-school student session, which includes the survey and student assessments, will be approximately 90 minutes, 20 of which will be for the student survey portion. The out-of-school student session, which will include the survey and a subset of the assessments, will take approximately 55 minutes, 20 of which will be for the student survey portion.

Table 6. MS1, MS2, and OFT2 Burden Estimates¹

MGLS:2017 Activity	Sample Size	Expected Response Rate	Number of Respondents	Number of Responses	Average Burden Time (minutes)	Total Burden (hours)	Estimated Respondent Average Hourly Wage1	Estimated Respondent Burden Time Cost
Student Assessment (in-school)3	26,100 25,520	85%	22,185 21,692	22,185 21,692	70	25,883 25,307	‒	‒
Student Assessment (out-of-school)3	580	85%	493	493	35	288	‒	‒

Part B

In section B.2 Procedures for the Collection of Information, under MS1 School Recruitment Approach (page 9) added:

Recruitment of Schools for Out of School Student Data Collection. As a last ditch effort to secure the participation of schools and their students, we will offer the possibility of collecting student data outside of school. This option will be offered to schools that are unable to otherwise schedule study participation into the school calendar. Schools allowing an out-of-school student data collection will be asked to provide a student roster to select students into the sample along with contact information to invite the parent and student to participate via Web and teacher information in order to invite teachers to participate in the teacher surveys. Student data collection will be conducted using the out-of-school data collection procedures described below.

On page 12, a phrase was added to subheader Student Survey and Assessments under MS1 Data Collection Approach to differentiate the in-school data collection procedures from the out of school data collection approach:

Student Survey and Assessments (In-School).

Subsequently, also on page 12, a section was added to document the out-of-school data collection approach:

Student Survey and Assessments (Out-of-school). The MS1 out-of-school data collection will be conducted with students whose schools are unable or unwilling to schedule an in-school session. As with the in-school sessions, a school coordinator will be appointed to prepare and submit a student roster, which will be used to sample the students for the study and provide parent and teacher contact information.

• Because MGLS:2017 MS1 students are minors, all communication with them will go through their parents or guardians. The parent or guardian of the study student will receive a letter and/or email telling him or her about the study and inviting the study student to participate via Web (appendix MS1-G4). As an enclosure in the parent mailing, the parent will receive an envelope addressed to the student. This envelope will contain a letter to the student inviting him or her to participate via Web and will contain information to log in to the session (appendix MS1-G4a). By the parent passing the information on to the student for participation, the parent implies permission for the student to participate.

• The student will complete a 55-minute session which will comprise the student survey, math assessment, and part 1 of the reading assessment.

• Parents of nonresponding students will receive reminder emails, letters, and postcards with information repeating the instructions on how the study student may access the survey. Emails will be sent approximately every 6-10 days and letters approximately every 2-3 weeks. Telephone prompting may also take place.

A bullet was added in Section B.3 Methods to Secure Cooperation under Maximizing School Participation (Page 20):

• We will offer an out-of-school student data collection to schools unwilling or unable to fit MGLS:2017 into the school schedule.

Part C – No Changes

Appendices A-S

Letters to parents and students were added for the out of school data collection in MS1-G4 and MS1-G4a (pages 31-33), and revisions were made to the panel maintenance materials, in MS2-K and MS2-L (pages 115-118), for clarity and to minimize item nonresponse. The panel maintenance materials changes include removing the request for child’s address, adding a check box under each parent address to indicate if it is the child’s primary address, and moving the enrollment status and school information to the top of the form.

Appendices T-V – No Changes

Appendices W

Emails were added in Appendix MS1-W.5a, MS1-W.5b, and MS1-W.5c (page 17-18) to contact parents for the MS1 out-of-school data collection.

3