MGLS Summary of Changes

Memorandum United States Department of Education

Institute of Education Sciences

National Center for Education Statistics

DATE: March 20, 2019 (updated May 1, 2019)

TO: Robert Sivinski, OMB

THROUGH: Kashka Kubzdela, NCES

FROM: Carolyn Fidelman, NCES

SUBJECT: Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) Main Study First Follow-up (MS2) Tracking and Recruitment and Operational Field Test Second Follow-up (OFT3) Update – Summary of Changes (OMB# 1850-0911 v.23)

The Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) is the first study conducted by the National Center for Education Statistics (NCES) to follow a nationally representative sample of students as they enter and move through the middle grades (grades 6–8). The data collected through repeated measures of key constructs will provide a rich descriptive picture of the academic experiences and development of students during these critical years and will allow researchers to examine associations between contextual factors and student outcomes. The study focuses on student achievement in mathematics and literacy along with measures of student socioemotional wellbeing and other outcomes. The study includes students with disabilities for whom descriptive information on their outcomes, educational experiences, and special education services are being collected. In preparation for the Main Study (MS), the data collection instruments and procedures were field tested. An Item Validation Field Test (IVFT) was conducted from January through May 2016 to determine the psychometric properties of assessment and survey items and the predictive potential of items so that valid, reliable, and useful assessment and survey instruments could be developed for the Main Study. The MGLS:2017 Operational Field Test (OFT) Base Year (OFT1) data collection was conducted from January through May 2017 to test the near-final instruments and recruitment and data collection procedures and materials in preparation for the MGLS:2017 Main Study Base Year (MS1). The MS1 data collection took place from January to August 2018, and the OFT First Follow-up (OFT2) data collections from February to May 2018. OFT2 was conducted primarily to obtain information on recruiting, particularly for students in three focal IDEA-defined disability groups: specific learning disability, autism, and emotional disturbance; obtain a tracking sample that can be used to study mobility patterns in subsequent years; and test protocols, items, and administrative procedures.

Originally, NCES planned for MGLS:2017 to conduct annual main study follow-up data collections first beginning in January 2019 and next beginning in January 2020, when most of the students in the sample will be in grades 7 and 8, respectively. In September 2018, OMB approved NCES’s request (OMB# 1850-0911 v.20) to revise the study design due to lower than expected response rates experienced in the sixth-grade data collection and to: (1) drop the originally planned seventh-grade round of data collection and conduct the Main Study First Follow-up (MS2) data collection in January-July 2020 (when most sample students will be in the eighth grade), (2) notify participating districts and schools of this change in data collection schedule, (3) discontinue the procedures designed to oversample students in specific IDEA-defined disability groups, and (4) conduct MS2 and OFT Second Follow-up (OFT3) tracking activities. In November 2018, OMB approved NCES’s request (OMB# 1850-0911 v.22) to make non-substantive edits to the 2018 MS2 and OFT3 tracking and enrollment update procedures and materials to be deployed in November 2018, and in December 2018, OMB approved NCES’s request (OMB# 1850-0911 v.21) to further modify the study design for MS2 to augment the sample with additional schools to achieve about 776 participating schools in MS2.

This current request is to: (1) update contact materials to reference the administrative records collection, (2) introduce a $10 incentive for panel maintenance participants, (3) update the student roster form in Appendix T for the fall 2019 MS2 tracking and recruitment activities, (4) introduce a flyer designed to encourage the return of parent permission forms in explicit permission schools, (5) update CIPSEA confidentiality law citations to reflect changes in law, and (6) update text to reflect that district staff will no longer be asked to sign affidavits of nondisclosure prior to receiving the list of schools sampled in the district.

This request does not involve any changes to the estimated respondent burden or to the cost to the federal government for this study.

(1) Administrative Records Collection. To add context and comparison to data provided in MS1 and MS2, after the January-July 2020 MS2 data collection, when most study students will be in grade 8, NCES is pursuing the possibility of collecting administrative records for sampled MS1 and MS2 students in the fall of 2021, when most study students will be in grade 10. Although the 2021 MGLS:2017 data collection will be part of later clearance requests (in May 2020 and March 2021), the already approved 2019 communications with schools regarding the 2020 MS2 data collection, need to include references to the 2021 MGLS:2017 administrative records collection so that schools are aware that the 2020 MS2 data collection may not be the final contact. If realized, the 2021 administrative records collection will include courses and grades from grade levels 6 through 8 as well as test scores from grades 8 and 9. Records collection would begin at the state and district level, with recruiters contacting schools as needed. For each student, the study may need to collect records from multiple sources. In this submission, we have revised MGLS:2017 MS2 communication materials where necessary to mention the 2021 student administrative records collection.

(2) Incentives for Panel Maintenance Participants. Considering the value that the base-year students provide for the utility of MGLS:2017 data, and given how obtaining contact information from students’ parents affects their subsequent participation, maximizing the amount of contact information we obtain from parents is crucial to the success of the study. Therefore, to maintain high response rates, we propose offering a $10 incentive for parent participants in the MS2B tracking activity, as has been done in prior NCES studies such as ELS and HSLS with good results. Based on the results from these two studies, we expect that a $10 incentive will increase response rates for the panel maintenance activity, particularly among base-year nonrespondents. In addition, we hope that the incentive will increase results to the follow-up survey, as was seen in ELS and is described in detail in the Supporting Statement Part B of this submission.

(3) Student Roster Form. The student roster has been modified to include a column to determine whether the student was enrolled in grade 6 at that particular school during the 2017-2018 school year. This will help confirm eligibility for participation in the MS2 student session. A second column, “student’s math teacher”, has been modified to clarify that we are asking about the student’s highest mathematics teacher. Some advanced students have multiple math teachers at a time and, with this clarification on the roster, we can better ensure we collect student data from the teacher who has taught the student at his/her highest level.

(4) Flyer. The flyer is another way for us to encourage student participation in MGLS:2017 at schools that require explicit parental consent permission forms. These flyers can be posted or distributed to the sampled students to help encourage students and parents to return the parental consent for, and thereby boost student participation in the student session.

The revisions made to the last approved MGLS:2017 clearance documents (OMB# 1850-0911 v.21-22) are reflected below. Blue text indicates the document and green text the section of the document in which a change was made, followed by notes regarding the edits. Black font demarks the text that remains unchanged from the last approved, while text in red font shows the revisions that were made.

Part A

Preface was updated to reflect study activities that have been completed by the end of December 2018. The following text was added to this section:

In November 2018, OMB approved NCES’s request (OMB# 1850-0911 v.22) to make non-substantive edits to the 2018 MS2 and OFT3 tracking and enrollment update procedures and materials to be deployed in November 2018, and in December 2018, OMB approved NCES’s request (OMB# 1850-0911 v.21) to further modify the study design for MS2 to augment the sample with additional schools to achieve about 776 participating schools in MS2. This current request is to: (1) update contact materials to reference the administrative records collection, (2) introduce a $10 incentive for panel maintenance participants, (3) update the student roster form in Appendix T for the fall 2019 MS2 tracking and recruitment activities, and (4) introduce a flyer designed to encourage the return of parent permission forms in explicit permission schools.

Section A.9’sTable 2 was updated to add MS2 2019 panel maintenance administration time and incentive.

Table 2. MS2 and OFT3 Tracking Activities, Data Collection Instruments, and Incentive Amounts

Instrument/Activity	Administration Time*	MS2 and OFT1** Incentives
MS2 Tracking, Recruitment, and Data Collection
MS2 2019-20 Tracking and Data Collection
Student return of parent consent forms (explicit consent schools only)	10 minutes	Food event at school (e.g., pizza, bagels, etc.) sponsored by the study
Student Assessments and Survey – In-school administration (Mathematics, Reading, Executive Function, Height, Weight, & Survey)	90 minutes	Earbuds used during assessment and a certificate for 2 hours of community service from the U.S. Department of Education
Student Assessments and Survey – Out-of-school administration (Mathematics, Reading, & Survey)	75 minutes	$20 and a certificate for 2 hours of community service from the U.S. Department of Education
Parent Panel Maintenance	5 minutes	$10
Parent Survey	40 minutes	$20 to $30 (one parent per student)

*Note that the assessment administration time may be longer for students with disabilities.

** Final incentive amounts were determined based on the outcome of the field tests and main study base year.

Also, in section A-9, new text was added under “Parents” to justify the payment of the $10 panel maintenance incentive.

Parents

Main Study First Follow-up (MS2): Parent survey response rates have declined over the past decade. The ECLS-K:2011 baseline (fall 2010) parent survey response rate was more than 10 percentage points lower (74 percent) than the parent survey rate in the corresponding 1998 wave of the ECLS-K (85 percent). Additionally, the ninth-grade parent survey response rate for the HSLS:09 baseline was 68 percent. The MGLS:2017 parent survey is a key component of the data being collected. In MS1, a differential incentive was offered to parents of students with Emotional Disturbance (EMN). Because we are no longer oversampling students with EMN, we will offer the same incentive to all MS2 parents as was used for MS1 parents of students without EMN – $20 with a $10 boost for nonresponse offered mid-way through the data collection period.

In addition, we will offer a $10 incentive for parents to complete the second panel maintenance materials in the fall of 2019. About 21 percent of parents participated in the 2018 panel maintenance activity and, of those, about 2 percent of parents were base-year nonrespondents. The ELS:2002/12 experience found that a $10 incentive for panel maintenance response increased response by about 5 percent. Moreover, parents who responded to the panel maintenance request were more likely to participate in the survey than those who did not participate in panel maintenance. The addition of a $10 incentive for the fall 2019 panel maintenance is expected to increase response for panel maintenance while also increasing response to the MS2 parent survey.

In section A-10, CIPSEA law citations were updated to reflect changes in law.

8. Confidential Information Protect and Statistical Efficiency Act of 2002;

9. E-Government Act of 2002, Title V, Subtitle A;

9. Foundations of Evidence-Based Policymaking Act of 2018, Title III, Part B, Confidential Information Protection;

Part B

Section B.2 was updated to reflect that district staff will no longer be asked to sign affidavits of nondisclosure prior to receiving the list of schools sampled in the district.

Recruitment of New Districts or Diocese. For school districts new to the study, school districts of augmentation sample public schools and dioceses of sample Catholic schools (if district or diocese affiliation exists) will receive a mailing about the study. The district introductory information packet includes a cover letter (Appendix MS2B-H), a colorful recruitment-oriented brochure (Appendix MS2-A1), and a sheet of Frequently Asked Questions (FAQs) about the study (Appendix MS2B-K). Three days after mail delivery of the packet, a recruiter calls to secure the district’s cooperation and answer any questions the superintendent or other district staff may have. The staff person working with us from the school district is asked to sign an affidavit of nondisclosure (NDA) prior to receiving the list of schools sampled in the district. Once the signed nondisclosure affidavit is received, we We also discuss the sampled schools, confirm key information about the schools (e.g., grades served, size of enrollment), and discuss obtaining the students’ IEP information that is necessary for drawing the MS1 student sample. Information collected during this call is used to confirm which schools in the district are eligible for participation in the study, and to obtain contact and other information helpful in school recruitment.

Section B.2 was updated to mention and later describe the potential 2021 administrative records data collection.

B.2 Procedures for the Collection of Information

(…)

Administrative Records Data Collection

To add context and comparison to data provided in MS1 and MS2, after the January-July 2020 MS2 data collection, when most study students will be in grade 8, NCES is pursuing the possibility of collecting administrative records for sampled MS1 and MS2 students in the fall of 2021, when most study students will be in grade 10. Any sampled students (minus study withdrawals) from either MS1 or MS2 would be eligible for records collection. If realized, the 2021 administrative records collection will include courses and grades from grade levels 6 through 8 as well as test scores from grades 8 and 9. Records collection would begin at the state and district level, with recruiters contacting schools as needed. For each student, the study may need to collect records from multiple sources. In this submission, we have revised MGLS:2017 MS2 communication materials where necessary to mention the 2021 student administrative records collection.

Section B3 was updated to add detail about the addition of a $10 incentive during MS2 panel maintenance and how it has been successfully used in other studies.

OFT3 and MS2 Tracking

Obtaining high response rates, and thereby minimizing sample attrition, is always a paramount objective in longitudinal studies. It is all more important for the first follow-up of MGLS:2017, MS2, however, because the final base year sample size is lower than originally targeted. Considering the value that the base-year students provide for the utility of MGLS:2017 data, and given how obtaining contact information from students’ parents affects their subsequent participation, maximizing the amount of contact information we obtain from parents is crucial to the success of the study. Therefore, to maintain high response rates, we propose offering a $10 incentive for parent participants in the MS2B tracking activity, as has been done in prior NCES studies such as ELS and HSLS with good results.

This proposed incentive is modeled after what was successfully used in HSLS:09 (OMB# 1850-0852 v.11) and in ELS:2002/12. Furthermore, in ELS:2002/12, NCES conducted an experiment with its field test sample that demonstrated the effectiveness of a $10 incentive offer to increase participation in the panel maintenance. In this experiment, half of the sample members in the field test sample were offered a $10 check if they or their parents confirmed or updated their contact information. No incentive was offered to the other half of the sample. A cost-benefit analysis was also conducted to evaluate the difference between the cost of the incentive offer and the difficulty of cases that responded. The impetus behind this analysis was to determine whether information was received from more difficult cases, because the benefit would be reduced if the “easy-to-track” cases were the ones to respond. Overall, the $10 treatment group had a higher participation rate (25 percent) than the control group (20 percent, t=1.90, p < .05). Higher panel maintenance participation for the treatment group as compared with the control group was also observed by various characteristics of cases, such as those with postsecondary education experience, males, and those with a high school diploma. Further evaluation of the data indicates that the contact information provided was largely new information not already in the study database; for 82 percent of the responding cases, at least one new address, phone number, or email address was provided for the student, parent, or both. Being able to make direct contact with the sample student during data collection saves time, reduces costs, and is likely to increase interview participation.

The $10 incentive offer was then implemented with the ELS:2002/12 main study sample based on the field test results. Forty percent of the ELS:2002/12 third follow-up main study sample participated in panel maintenance at some point. Among that group, 97 percent responded to the third follow-up survey compared to a 74 percent response rate among those that did not participate in panel maintenance. Among the sample members that responded to one or more panel maintenance requests, the ELS:2002/12 survey response rate exceeded 90 percent across numerous categories including, but not limited to: those without known postsecondary experience (91%), those without a regular high school diploma (93%), and males.

In looking at the recent respondents to the fall 2018 MS2 panel maintenance emails and letters (Appendices MS2A-I through MS2A-L), we have received responses, either new contact information or confirmation that the contact information we had stored is still accurate, from approximately 31 (2%) of the 1,490 base-year non-respondents to be pursued in MS2. At the same time, the base-year respondents have been more likely to respond to the panel maintenance emails and letters with approximately 3,300 of the 14,373 (23%) base-year respondents confirming or submitting new contact information. Based on the aforementioned results from HSLS:09 and ELS:2002/12, we expect that a $10 incentive will increase response rates for the MGLS:2017 MS2 panel maintenance activity, particularly among base-year nonrespondents. In addition, we hope that the incentive will increase results to the follow-up survey, as was seen in ELS.

Appendices A-S

In Appendix MS2-B1 (MGLS:2017 Recruitment Website Text), text was removed to reflect that district staff will no longer be asked to sign affidavits of nondisclosure prior to receiving the list of schools sampled in the district.

Confidentiality:

The Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) is conducted by the National Center for Education Statistics (NCES). Under contract with NCES, RTI International, a U.S.-based nonprofit research organization, administers MGLS:2017 on behalf of NCES. NCES is authorized to conduct MGLS:2017 by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9543) and to collect students’ education records from education agencies and institutions for the purposes of evaluating federally supported education programs under the Family Educational Rights and Privacy Act (FERPA, 34 CFR §§ 99.31(a)(3)(iii) and 99.35). All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

All RTI project staff have signed confidentiality agreements and affidavits of nondisclosure and are prohibited by law from using the obtained information for any purposes other than this research study.

You may review the Confidentiality Agreements and Affidavit of Nondisclosure statements (pdf, 81KB) that are signed by project staff.

A nondisclosure form is also required before informing districts which schools are selected for the study. We take the confidentiality of participating schools seriously and limit disclosure to those who need to know to make informed decisions about school and student participation in the study. A copy of the nondisclosure agreement may be accessed here.

You may also review the procedures that govern NCES and RTI International to protect identifiable information.

In Appendices MS2B-E and MS2B-F (District Letter), the text was added: “The data collection will occur on a date the school(s) select(s) between January and June of 2020 and administrative records may be collected in fall 2021.”

In Appendix MS2B-G, 2B-I1, and MS2B-I2 (School Letter), the following bullet point was added:

Study Overview:

• Your school selects a day between January and June of 2020 for the 90-minute student session.

• Project staff will provide all equipment and the Internet connectivity to run the student session.

• Students will respond to survey questions, math and reading, and executive function. They will also provide height and weight measurements.

• Math teachers, the administrator, and one parent for each student will receive a survey invitation.

• If district/school policy permits, your school will receive a $400 check or $400 in goods/services for participating, the school coordinator will receive $50 to thank them for the enrollment update and $100 for the data collection logistics.

• Student administrative records may be collected in fall 2021.

In Appendices MS2B-H1 and MS2B-H2 (District Letter), the following sentence was added: “In addition to the 2019-20 data collection, we may also collect administrative records information in fall 2021.”

In Appendix MS2B-L (Letter to Parents from Schools), the following sentence was added: “Later, in fall 2021, student administrative records may be collected.”

In Appendix MS2B-M (Panel Maintenance Letter), the following sentence was added: “Once the information is updated and received, we will send you a $10 check as a token of our appreciation.”

In Appendix MS2B-N (Panel Maintenance Email), the following sentence was added: “Upon submitting your son/daughter’s updated contact information, we will send you a $10 check as a token of our appreciation.”

In Appendix MS2B-O (Panel Maintenance Webpage Contents), the following sentence was added: “Upon submitting your son/daughter’s updated contact information, we will send you a $10 check as a token of our appreciation.”

and the following item 8 was added to the end of the address update form:

8. You’re finished! The last thing we need to know is where you want us to send the $10 check for completing this activity. If the address below is right, press Next. If this address below is not right, please fix it, and then press Next. If you do not want to receive this check, please click the box below, and then press Next to continue.

                Name: [________________]

                Street address1: [___________________]

                Street address2: [___________________]

                City: [___________________]

                State: [___________________]

                Zip code: [___________________]

 I do not want to receive any money for completing this survey.

In Appendix MS2B-P (Panel Maintenance Hardcopy), the following sentence was added: “Upon submitting your son/daughter’s updated contact information, we will send you a $10 check as a token of our appreciation.”

and the following item 6 was added to the end of the address update form:

6. You're finished! The last thing we need to know is where you want us to send the $10 check for completing this document.

Send check to address/corrected address in Question 4.

Name:                    __________________________

Street address:     __________________________

Street address2:  __________________________

City:                        __________________________

State:                      ___________________

Zip code:               ___________________

In Appendix MS2B-Q1 and MS2B-Q2 (Parent Permission Letter), the following sentence was added: “Later, in fall 2021, student administrative records may be collected.”

And to the fields asking for parent’a information, a field was added for parent’s home address and his/her son/daughter’s school name:

Please Print:

Student name: __________________________________________

School name: ____________________________________________

Parent name: ___________________________________________

Parent telephone: _________________________________________

Parent email: _____________________________________________

Parent home address: __________________________________________________________________

Street address City State Zip

Appendix MS2B-Q2A was added to the package. This flyer will be used to help increase student participation.

MGLS:2017 Student Participation Flyer (Explicit Permission Schools)

NATIONAL CENTER FOR EDUCATION STATISTICS/U.S DEPARTMENT OF EDUCATION

MIDDLE GRADES LONGITUDINAL STUDY of 2017-18

[PIZZA PARTY!/GIFT CERTIFICATE!]

[A food event will be held for/a $3 gift certificate to the school cafeteria will be given to] ALL students whose parent/guardian returns a completed Middle Grades Longitudinal Study of 2017-18 (MGLS:2017) permission form! [The party will be held at your child’s school/The certificate will be given out] on or about the day the session will be held.

What’s the study about?

• Selected students spend approximately 90 minutes on a computer answering questions about themselves, completing math and reading items, and playing a “brain game”. Height and weight measurements will be collected during the session.

Why should my child participate?

• This is an exciting and unique chance to provide your opinion, input, and experiences about middle grades education. We are collecting information from students and their parents, teachers, and administrators to find out what you think is working well and what is not working well. The input we receive could help to improve education for students now and in the future.

What are the benefits of participation?

• Excitement – the feeling of being part of one of the largest national longitudinal studies of the middle grades by the U.S. Department of Education and the National Center for Education Statistics (NCES).

• Contribution – you can take pride in knowing that you are making a positive contribution to the field of middle grades education.

• Tokens of Appreciation– as a thank you for participating, your child will be given a pair of earbuds and a pencil used during the session. He/She will also receive a certificate for 2 community service hours.

Please complete the enclosed permission form and return it to [Fill] in the return envelope today!

Appendices T-V

In Appendix MS2-T2 (Student Rostering Form), a field to capture whether the student was in grade 6 at that particular school during the 2017-18 school year and clarification on the rostering form that the math teacher’s name added should be the student’s highest math teacher.

Name

Student was in 6th grade in 2017-18 at this school?

Parent Contact information

Student's Math Teacher (Highest Math Class)

Student's Special Education Teacher, if applicable

Student name

Y/N

Parent's First Name

Parent's Last Name

Parent's Address

Parent City

Parent State

Parent Zip

Parent's Email

Parent's Home Phone

Parent's Cell Phone

MathTeacherFirstName

MathTeacherLastName

MathTeacherEmail

Math Course Name

Math Course Period/Section

SpecialEDTeacherFirstName

SpecialEDTeacherLastName

SpecialEDTeacherEmail