Change Sheet Memo

Memorandum United States Department of Education

Institute of Education Sciences

National Center for Education Statistics

DATE: January 17, 2020

TO: Robert Sivinski, OMB

THROUGH: Kashka Kubzdela, NCES

FROM: Carolyn Fidelman, NCES

SUBJECT: Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) Main Study First Follow-up (MS2) Data Collection Change Request (OMB# 1850-0911 v.25)

The Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) is the first study conducted by the National Center for Education Statistics (NCES) to follow a nationally representative sample of students as they enter and move through the middle grades (grades 6–8). The data collected through repeated measures of key constructs will provide a rich descriptive picture of the academic experiences and development of students during these critical years and will allow researchers to examine associations between contextual factors and student outcomes. The study focuses on student achievement in mathematics and literacy along with measures of student socioemotional wellbeing and other outcomes. The study includes students with disabilities for whom descriptive information on their outcomes, educational experiences, and special education services are being collected.

The study conducted an Item Validation Field Test (IVFT) in Winter/Spring 2016 to determine the psychometric properties of assessment and survey items and the predictive potential of items so that valid, reliable, and useful assessment and survey instruments could be developed for the Main Study. An Operational Field Test (OFT) was conducted in Winter/Spring 2017 with a follow-up in Winter/Spring 2018 to test the near-final instruments and recruitment and data collection procedures and materials in preparation for the MGLS:2017 Main Study Base Year (MS1). The MS1 data collection took place in Winter/Spring 2018. OMB has approved recruitment activities for the MS2 data collection, tracking of the MS1 sample, and plans for the MS2 data collection. (OMB# 1850-0911 v.20-24).

This current request is to (1) reduce the time of the out-of-school student session from 75 minutes to 45 minutes and (2) to add language to introductory screen on each web survey to ensure the security of survey responses. The change in time for the out of school student session impacts assessments only, the time for which is not considered when specifying burden, and therefore does not involve any changes to the estimated respondent burden. This request also does not impact the cost to the federal government for this study.

1. Reduction in time for the out-of-school student session. To increase participation among those students who participate outside of school, the out-of-school session was reduced from 75 minutes to 45 minutes. Students participating outside of school were not slated to complete the brain games or height and weight measurements. We propose further reducing the out-of-school session to remove the second stage portions of the reading and mathematics assessment.

2. Additional data security language. We propose adding language to the introductory screen of our web surveys to ensure that web survey respondents fully close out of their browser when they complete the survey to ensure the security of their responses. In addition, a pop-up will appear after 10 minutes of inactivity to alert the respondent that the session will close in 2 minutes.

The revisions made to the last approved MGLS:2017 clearance documents (OMB# 1850-0911 v.24) are reflected below. Blue text indicates the document and green text the section of the document in which a change was made, followed by notes regarding the edits. Black font demarks the text that remains unchanged from the last approved, while text in red font shows the revisions that were made.

Part A

Preface was updated to reflect the status of the previous OMB approval and the purpose of the current memo. The following text was added to this section:

This submission is to conduct the The MS2 data collection, to be conducted from January through July 2020 (when most sample students will be in the eighth grade), was approved in November 2019 (OMB# 1850-0911 v.24). The latest revision to this submission is to reduce the out-of-school data collection student session timing from 75 minutes to 45 minutes and to add language to the online surveys to ensure the security of survey responses.

Section A.9’sTable 2 was updated to reflect the change in administration time for the student out-of-school administration.

Table 2. MS2 and OFT3 Tracking Activities, Data Collection Instruments, and Incentive Amounts

Instrument/Activity	Administration Time*	MS2 and OFT1** Incentives
MS2 Tracking, Recruitment, and Data Collection
Student return of parent consent forms (explicit consent schools only)	10 minutes	Food event at school (e.g., pizza, bagels, etc.) sponsored by the study
Student Assessments and Survey – In-school administration (Mathematics, Reading, Executive Function, Height, Weight, & Survey)	90 minutes	Earbuds used during assessment and a certificate for 2 hours of community service from the U.S. Department of Education
Student Assessments and Survey – Out-of-school administration (Mathematics, Reading, & Survey)	4575 minutes	$20 and a certificate for 2 hours of community service from the U.S. Department of Education
Parent Panel Maintenance	5 minutes	$10
Parent Survey	35 minutes	$20 to $30 (one parent per student)

*Note that the assessment administration time may be longer for students with disabilities.

** Final incentive amounts were determined based on the outcome of the field tests and main study base year.

In section A-12, new text was added under “MS2 Data Collection” reflect the out-of-school administration.

MS2 Data Collection: Students who participate in school will complete a 90-minute session consisting of math and reading questions, a survey, two brain games, and height and weight measurements. Students who participate outside of school will spend about 45 minutes as they will not complete the brain games, the second-stage portions of the reading or math assessment, or height and weight measurements, so their session will take about 45 minutes.

Part B

The introduction to Part B was updated to reflect the status of the previous OMB approval and the purpose of the current memo.

Part B of this submission presents information on the collection of information employing statistical methods for the Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) Main Study First Follow-up (MS2) Data Collection. MS2 recruitment, which began in began in January 2019, was approved in December 2018 with the latest update an update approved in May 2019 (OMB# 1850-0911 v.21-23). MS2 data collection was approved in November 2019 (OMB# 1850-0911 v. 24), and the latest revision to this submission is to reduce the out-of-school data collection student session timing from 75 minutes to 45 minutes and to add language to the online surveys to ensure the security of survey responses.

Section B.3 was updated to reflect that students participating outside of school will complete a shorter session.

Student Survey and Assessments (Out-of-school). An important aspect of a longitudinal study is to follow students regardless of where they attend school after the base year. This enables examination of the trajectory and associated success of all students regardless of their path through the education system. Out-of-school data collection was conducted successfully during the High School Longitudinal Study of 2009 (HSLS:09) first follow-up with students in modal grade 11 and was field tested in the MGLS:2017 OFT2. The out-of-school student session will take approximately 45 75 minutes and include only the first stage of the mathematics and reading assessments and the student survey. Height and weight measurements and executive function tasks will not be collected from students participating outside of school.

Appendices A-S

In the following appendices, the session length was changed from 75 minutes to 45 minutes. Only the number was changed.

Appendix MS2B-K. MGLS:2017 School FAQs Text

Appendix MS2B-Q7b. Out-of-School Student Instructions

Appendix MS2B-R1. Parent and Out-of-School Student Data Collection – Parent Letter Initial

Appendix MS2B-R1a. Parent and Out-of-School Student Data Collection – Parent Letter Initial – Spanish

Appendix MS2B-R3. Out-of-school Data Collection – Student Letter

Appendix MS2B-R4. Out-of-school Data Collection – Student Letter Reminder

Appendix MS2B-R8g. MGLS:2017 Parent FAQs Text – Out-of-School Participation

Appendix MS2B-R8g1. MGLS:2017 Parent FAQs Text – Out-of-School Participation – Spanish

Appendices T-V

The survey instrument for students who will complete the survey outside of school added a timeout warning that also includes language to ensure that student respondents who are not completing the survey in school¹ know to close all browser windows to ensure the security of its answers, as follows (p. 4):

AFTER 8 MINUTES OF NO ACTIVITY, DISPLAY:

Due to inactivity, your session will close in 2 minutes. Press Continue if you need more time.

[AFTER 10 MINUTES OF NO ACTIVITY, DISPLAY:]

Your session has timed out. We’re very sorry! Your session has been idle for more than 10 minutes. Please click below to log back into the survey.

Click below to return to the login page.

If you need to take a break, CLOSE ALL browser windows to keep your responses secure. For example, if you used Chrome or Safari to open the survey, make sure no Chrome or Safari windows are open after you end the survey. Not closing all browsers may allow someone else to see your responses.

For all other web instruments, the instruction page was modified to include language to ensure that respondents know to close all browser windows to ensure the security of its answers, as follows (pp. 35, 227, 271, and 333):

When you have finished or if you need to take a break before finishing, please click the LOG OUT button and CLOSE ALL browser windows to keep your responses secure. For example, if you used Chrome or Safari to open the survey, make sure no Chrome or Safari windows are open after you end the survey. Not closing all browsers may allow someone else to see your responses. When you log in again, you can resume where you left off.

The same warning is also now available for the Spanish Translation of the Parent Interview Survey (p. 124):

Cuando haya terminado o si necesita tomarse un recreo antes de terminar, por favor haga clic en el botón de CERRAR SESIÓN y CIERRE TODAS las ventanas del navegador para mantener sus respuestas protegidas. Por ejemplo, si usó Chrome o Safari para abrir la encuesta, asegúrese de que no quede abierta ninguna ventana o pantalla de Chrome o Safari después que usted termine la encuesta. Si usted no cierra todos los navegadores, alguien tal vez pueda ver sus respuestas. Al iniciar la sesión de nuevo, usted puede continuar a partir de donde estaba.

The following timeout message was added as a pop up after 8 minutes of inactivity (pp. 36, 227, 271, 332):

Warning: Due to inactivity, your session will close in 2 minutes. Press continue if you need more time.

The following timeout message was added as a pop up after 10 minutes of inactivity (pp. 36, 227, 271, 332):

Your session has timed out. We’re very sorry! Your session has been idle for more than 10 minutes. Please click below to log back into the survey.

Click below to return to the login page.

The same warnings are also now available for the Spanish Translation of the Parent Interview Survey (p. 124):

AFTER 8 MINUTES OF NO ACTIVITY, DISPLAY:

Debido a la inactividad, la sesión será finalizada en 2 minutos. Haga clic en “Continuar” si usted necesita más tiempo.

  

[AFTER 10 MINUTES OF NO ACTIVITY, DISPLAY:]

La sesión finalizó. ¡Lo sentimos mucho! La sesión ha sido inactiva por más de 10 minutos. Por favor haga clic abajo para volver a ingresar a la encuesta.

Haga clic abajo para volver a la página de iniciar sesión.

Finally, the timeout warnings that appeared in some surveys have now been changed to reflect that the instruments will time out after 10 minutes, rather than 20 (pp. 42, 228, 272, 335):

To protect your data, you will be logged off if you are idle for more than 20 10 minutes and will need to log back in.

The same change was also made to the Spanish Translation of the Parent Interview Survey (p. 129):

Para proteger su información, la sesión terminará si usted está inactivo(a) por más de 20 10 minutos y tendrá que volver a ingresar.

Appendix W

In the following appendices, the session length was changed from 75 minutes to 45 minutes. Only the number will be changed.

Appendix MS2-W.3.a. Invitation to Survey E-mail, Parent Survey and Out-of-School Student Data Collection

Appendix MS2-W.3.a.1. Invitation to Survey E-mail, Parent Survey and Out-of-School Student Data Collection – Spanish Translation

Appendix MS2-W.3.c Reminder E-mail, Parent Survey and Out-of-school Student Data Collection

Appendix MS2-W.5a. Email Invitation for Out-of-School Participation, Student Respondent

Appendix MS2-W.5b. Email Reminder for Out-of-School Participation, Student Respondent

1 Students who complete the student survey in school will be closely supervised, so they do not require this warning.