NAEP 2020 LTT Update Response to Emergency Clearance Public Comments

Public Comments Received During the Emergency Clearance Comment Period

April 2019

National Assessment of Educational Progress (NAEP) 2020 LTT

ED-2019-ICCD-0032 Comments on FR Doc # 2019-05403

Document: ED-2019-ICCD-0032-0005

Name: Anonymous

As a student who is originally from outside of the united states, I think it is pointless to compare the test results of the American children to children of other countries. What is the conclusion that this test is trying to get to? The education system in the U.S. is vastly different than, for example, in China. In some countries, students are more likely to be good test takers since they are trained to take tests, and scores are very important to them; while in the States, I believe students are taught less rigidly and teachers value creativity. So there is really no point to compare test results internationally since there are different standards.

_______________

NCES Response:

First, thank you for taking the time to review the materials and for providing feedback. We appreciate your comments regarding your concern for using the National Assessment of Educational Progress (NAEP) test results to compare the U.S. to other countries. NAEP is the only nationally representative assessment in the United States of what students know and can do in various subjects, over time. The results from NAEP are for U.S. students and are not used as a direct comparison to students in other countries.

Document: ED-2019-ICCD-0032-0006

Name: Beth LaDuca

Colt Gill

Director of the Oregon Department of Education

April 12, 2019

Thank you for the opportunity to comment on the burden estimates for the NAEP 2019 and 2020 Long-Term Trend 2020 Update Emergency Clearance (OMB# 1850-0928 v.14). The burden estimate for school personnel to input data for students with disabilities and English language learners (SD and ELL students) is 10 minutes per sampled student. For some schools, this translates to considerable burden, as much as 400 minutes for one Oregon school in 2018-2019. Another Oregon school principal, with 29 SD and ELL students selected for NAEP 2019, reported: I had to spend an entire day locked in my office preparing for all of the student lists and accommodations. Because most of our students are either SD, ELL, or both, I had so many lists to go through. This doesn’t take into account the other prep work, like sending out letters to parents, teachers, students . . .All of that was totally doable. But having to hand-enter in each student’s info was a lot of work. Even for SBAC, I don’t have to enter all that information. It seems like a data person could have uploaded a file or something.

As is often the case in Oregon, this principal completed all the NAEP planning activities. The official burden estimate for this work is 4.5 hours, plus the 4 hours and 50 minutes (290 minutes) required to enter the SD and ELL student information. The total burden estimate of 9 hours and 20 minutes represents a substantial burden for a school administrator.

The Oregon principal quoted above offers a very reasonable option for reducing burden: if NAEP were to allow a download of the SD and ELL student questions into an Excel spreadsheet, the school coordinator could enter the data much more efficiently and then upload the completed spreadsheet to the NAEP planning system. Having the SD and ELL questions in Excel would also allow school or district data specialists the option of entering the data automatically from a student information system. Currently, the NAEP planning system requires the school coordinator to answer the SD and ELL questions individually for each student in the online system. There are from six to eleven questions for each SD and ELL student. Reducing the burden for providing SD and ELL student information would be a meaningful improvement for NAEP 2020.

Respectfully submitted,

Beth LaDuca

NAEP State Coordinator

255 Capitol St NE, Salem, OR 97310 | Voice: 503-947-5600 | Fax: 503-378-5156 | www.oregon.gov/ode

_______________

NCES Response:

Thank you for taking the time to review the materials and for providing valuable feedback. We appreciate your comments regarding the amount of time, burden, and methods involved in completing the Student with Disabilities (SD) and English Language Learners (ELL) information for Long-Term Trend. Due to time constraints and having to administer the first set of tests in October of 2019, making any changes to the information collected and/or the method of data entry is not feasible at this time. We can assure you, as we prepare for the 2021 and future administrations, serious consideration will be given to how we can streamline this process to reduce the burden and improve the data entry procedure.

Document: ED-2019-ICCD-0032-0007

Name: Anonymous

I do not think that the National Assessment of Educational Progress and Long Term Trend testing should continue, as I do not believe the testing to be an effective representation of students' progress and overall data. Coming from personal experience, having recently graduated high school, throughout my time in school, I took countless standardized tests meant to collect student data. Many of my peers, before we even entered middle school, began to take standardized tests less seriously, or not try their hardest, as they had realized that there was not a direct benefit to them to do well. Additionally, the data collection questions that are presented before the test (including questions about your race, parental education, etc), are sometimes given as optional--or people simply do not fill them out. These factors may actually distort the data collected from the sample. While I cannot provide specific data on how drastically these factors would skew the data collected, from my own personal experience I believe that these tests would not be an accurate representation of student progress and data.

_______________

NCES Response:

Thank you for taking the time to review the materials and for providing feedback. We appreciate your comments regarding the motivation of students involved in the NAEP assessments. Please be assured that our recruiting efforts do encourage student participation and engagement, and some survey questions ask about student motivation and perceptions about the subject and the assessment. In addition, student non-response to the survey questions is considered in the statistical analyses.

2