ED Response to Public 60-day Comments

Public Comment Received During the 60-day Comment Period and NCES Responses

Docket: ED-2015-ICCD-0139 Comment on FR Doc # 2015-31521
National Assessment of Educational Progress (NAEP) Assessments for 2017-2019

Comment related to burden for school personnel

Document: ED-2015-ICCD-0139-0005 and ED-2015-ICCD-0139-0007

Submitter Information

Name: Beth LaDuca, Oregon Department of Education

Comment

Thank you for the opportunity to comment on the burden estimates for the NAEP Assessments for 2017-2019 (OMB# 1850-NEW v.1). The burden estimate for school personnel to conduct NAEP pre-assessment and assessment activities is three hours. The Oregon Department of Education (ODE) does not believe that this burden estimate is accurate. The following document for NAEP 2016, developed by a work group of NAEP State Coordinators with support from NAEP contractor Westat, outlines expected time commitments of over 20 hours for school coordinators to conduct pre-assessment and assessment activities. These activities will be substantially the same for the NAEP 2017-2019 Assessments.

Response

We thank you for your thoughtful and careful review of the NAEP 2017-2019 package. We have carefully considered your comments and have provided further justification for why we have not made changes.

The estimates developed by the NAEP State Coordinators were initial estimates as the new MyNAEP system and tasks were being developed and implemented. Now that the MyNAEP system has been used for the 2015 and 2016 assessments, we can estimate the time for the future assessments based on actual data. Specifically, the time spent by the school coordinates on Tasks 3, 5-10 (as listed in the comment) was a little over an hour. Adding the additional time for Tasks 1 and 2 and the Wrap-up (estimated at 10-15 minutes each, similarly to as is listed above) and Assessment Day (providing logistical support, estimated at 20-30 minutes) yields a total burden estimate for the school coordinator at a little over 2 hours. To allow for individual fluctuations, we have used an estimated time of 3 hours for these activities. Note, the time for completing the student SD and ELL information (described in Task 4, above) is included separately at 10 minutes per student, and is not included in the 3 hours for the school coordinator.

Comment related to burden for teachers and school administrators

Document: ED-2015-ICCD-0139-0005 and ED-2015-ICCD-0139-0007

Submitter Information

Name: Beth LaDuca, Oregon Department of Education

Comment

The Oregon Department of Education also questions the burden estimates for the school and teacher questionnaires. ODE received multiple complaints about the length school and teacher questionnaires from principals and teachers in schools selected for the NAEP 2015 program. According to principals, the school questionnaire requested detailed data that took considerable time to gather. Several principals reported spending several hours, rather than the estimated 30 minutes, to complete the school questionnaire. Multiple teachers reported that the teacher questionnaires were simply too long. For the NAEP 2017 - 2019 Assessments, the burden estimate for the teacher questionnaires is 30 minutes despite the fact that the National Assessment Governing Board Background Information Framework for the National Assessment of Educational Progress states:

In constructing questionnaires it is important to place strict limits on the respondent burden they impose. As much data as possible should be obtained from school records and other reliable sources. The average individual response time to answer background questionnaires for each assessment, as calculated in accordance with Office of Management and Budget (OMB) procedures, shall be limited as follows: ten minutes for each student, 20 minutes for each teacher, and 30 minutes for each school. (p. 10)

The contractors for NAEP 2015 have data that NCES should analyze to review the burden estimates for NAEP 2017 - 2019 school and teacher questionnaires. The majority of NAEP 2015 questionnaires were completed online, so the contractor should be able to provide response time estimates. Also, the contractor responsible for the online assessment planning system MyNAEP collected feedback from users, including feedback about school and teacher surveys.

Please review all the available data and, if needed, adjust the burden estimates for pre-assessment and assessment activities and for school and teacher questionnaires so that school personnel are well-informed about the time necessary to complete them. Please ensure that that the teacher questionnaires adhere to the National Assessment Governing Board limit of 20 minutes. If possible, please reduce the burden placed on school personnel conducting NAEP pre-assessment and assessment activities.

Response

We thank you for your thoughtful and careful review of the NAEP contextual questions submitted for 2017-2019. We have carefully considered your comments and have provided further justification for why we have not made changes.

Currently, NCES is compiling and analyzing timing data related to the 2015 and 2016 assessments to examine the time necessary to complete the questionnaires. In developing the teacher and school questionnaires for 2017-2019, we will consider the number of items from the library (i.e., Appendix F) that can be included so that the final selection of items does not exceed the proposed time estimates. In addition, please note that the times listed in NAGB’s Background Information Framework are guidelines only. As such, NAEP does not have a policy which limits the amount of time to be spent on survey questionnaires.

2