Public Comments Received During the 30-day Comment Period
April 2023
National Assessment of Educational Progress (NAEP) 2024
ED-2022-SCC-0141 Comments on FR Doc # 2023-02414
NCES and the staff of The National Assessment of Educational Progress want to thank all public commenters for your feedback responding to a request for comments on NAEP 2024 published in the Federal Register. The National Center for Education Statistics (NCES) appreciates your interest in our work. The Paperwork Reduction Act (PRA) provides an opportunity for an open and public comment period where comments on collections can be made. We are grateful for this process and your comment and hope that you will continue to follow our work.
Thank you for the opportunity to comment on the information collection “National Assessment of Educational Progress (NAEP) 2024,” OMB # 1850-0928. Supporting Statement Part B, section B.2.b describes a new Assessment Management System (AMS) as an “online secure site that provides participating schools with a convenient way to prepare for the upcoming assessment.”
Unfortunately, the development of the AMS during 2023 for use in the NAEP 2023 field test and NAEP 2024 has not delivered the improvements long requested by many school administrators and NAEP State Coordinators. Specifically, the AMS does not offer users the ability to upload information about multiple students with disabilities and English learners in one file. In fact, the current version of the AMS lacks the functionality of the old system for schools to print worksheets with the names of these students that could be used to collect the necessary data offline from the appropriate teacher or specialist. The new AMS actually reduces the flexibility of school personnel to collect and input information.
The Oregon Department of Education has expressed the need for improvement in the collection of data for students with disabilities and English learners since April 2019, in a comment on OMB# 1850-0928 v.14. That comment is as relevant today as it was in 2019, since the issue has not been addressed; however, the burden estimate for inputting data for students with disabilities and English learners has increased from 10 to 15 minutes per student.
The Oregon Department of Education commented in April 2019:
“The burden estimate for school personnel to input data for students with disabilities and English language learners (SD and ELL students) is 10 minutes per sampled student. For some schools, this translates to considerable burden, as much as 400 minutes for one Oregon school in 2018-2019. Another Oregon school principal, with 29 SD and ELL students selected for NAEP 2019, reported:
I had to spend an entire day locked in my office preparing for all of the student lists and accommodations. Because most of our students are either SD, ELL, or both, I had so many lists to go through. This doesn’t take into account the other prep work, like sending out letters to parents, teachers, students . . .All of that was totally doable. But having to hand-enter in each student’s info was a lot of work. Even for SBAC, I don’t have to enter all that information. It seems like a data person could have uploaded a file or something.
As is often the case in Oregon, this principal completed all the NAEP planning activities. The official burden estimate for this work is 4.5 hours, plus the 4 hours and 50 minutes (290 minutes) required to enter the SD and ELL student information. The total burden estimate of 9 hours and 20 minutes represents a substantial burden for a school administrator.
The Oregon principal quoted above offers a very reasonable option for reducing burden: if NAEP were to allow a download of the SD and ELL student questions into an Excel spreadsheet, the school coordinator could enter the data much more efficiently and then upload the completed spreadsheet to the NAEP planning system. Having the SD and ELL questions in Excel would also allow school or district data specialists the option of entering the data automatically from a student information system. Currently, the NAEP planning system requires the school coordinator to answer the SD and ELL questions individually for each student in the online system. There are from six to eleven questions for each SD and ELL student. Reducing the burden for providing SD and ELL student information would be a meaningful improvement for NAEP 2020.”
Reducing the burden on school personnel by enhancing the new AMS to allow upload of information for multiple students in one file would be a meaningful improvement for NAEP 2024. The Oregon Department of Education requests that the NAEP program prioritize development of this functionality in the new AMS.
Respectfully submitted,
Beth LaDuca
NAEP State Coordinator
----------------------------------------------------------------
Dear Ms. LaDuca,
We thank you for your thoughtful and careful review of the NAEP 2024 package and your public comment on the NAEP burden the principals encountered when entering SD and EL information.
Based on previous valuable feedback you provided via the Federal Register and comments from other stakeholders, we have recently made several significant improvements in the process for entering SD and EL information to reduce burden, such as simplifying the user-interface to reduce the number of pages and actions required to provide this information. We agree that the use of the worksheets was helpful and plan to reinstitute a similar process in a future iteration of AMS. We have not yet implemented the use of importing or uploading spreadsheets to the system, though we are developing that function and hope to have it in place for the 2025 NAEP program. Our main concern is ensuring that we implement an upload or import function which accounts for the variation in accommodation conventions across states and does not increase burden to the schools. As you know, the naming and implementation of NAEP accommodations and state accommodations can differ across the states; therefore, it is necessary for NAEP state coordinators states to cross-walk the state accommodations that schools know and understand with the accommodations as defined in NAEP. The cross-walk is necessary if states want to reduce burden to schools by letting the schools use the state or local accommodations instead of the NAEP defined accommodations. We started this process in 2023 with about five states and plan to continue it in 2024. This cross-walk process needs to be implemented before an upload process could be implemented in the AMS. We feel it is too risky to implement this complex importing or uploading feature in a large NAEP year with state and TUDA level results this would be an appropriate task for a small NAEP year, such as NAEP 2025. We will continue to evaluate and make efficiencies to the system to reduce the burden on schools.
We continue to look for alternatives to minimize the burden on schools for the collection of SD and ELL information, particularly when there is a large number of identified students. In addition, we are in the process of conducting a survey with all of the school coordinators participating in the 2023 Field Test. We will use this information, as well as your feedback, to evaluate additional ways to reduce the burden. Additionally, we may adjust the burden estimate, based on these survey results. Any changes to the AMS or the associated burden for the NAEP 2024 assessment will be reflected in a future amendment.
Sincerely,
Daniel J. McGrath, Ph.D.
Delegated Authority of Associate Commissioner
Assessment Division
National Center for Education Statistics
Tel: 202 710 8753
GLSEN is pleased to submit this comment regarding measures to advance inclusion and equity for LGBTQ+1 students and educators in the National Assessment of Educational Progress of 2024 (NAEP 2024) administered by the National Center for Education Statistics (NCES). As the leading national organization on LGBTQ+ issues in K-12 education, GLSEN urges the inclusion of LGBTQ+ demographic measures in the NAEP 2024 student and educator questionnaires.
The Federal Evidence Agenda on LGBTQI+ Equity directs federal agencies to “consider prioritizing assessments of programs and services for LGBTQI+ people in areas where known disparities exist, such as in education.”2 As a nationally representative sample of students and, periodically, representative at the state level as well, the NAEP 2024 provides critical insight into student achievement and learning experiences across academic subjects that inform education policy and practice. Given that available data consistently show disparities impacting LGBTQ+ students in K-12 schools,3 including specifically with regard to conventional educational metrics like GPA and absences,4 the NAEP 2024 should include LGBTQ+ demographic measures. Doing so would also support the NCES in “better determin[ing] how well education is meeting the needs of all students,”5 including LGBTQ+ students of color and others who experience marginalization across multiple aspects of their identity.
In response to GLSEN’s earlier comment on the NAEP of 2024 urging the addition of measures that allow students and teachers to self-report their gender identity, NCES stated that it “is actively working towards including more gender identity options in future NAEP data collections both from school records (where we get student gender information) and teacher self-reports via the teacher survey questionnaire).”6 NCES is not required to collect gender data through student records (20 U.S. Code § 9622) and other data on population characteristics are already collected through student questionnaires. Because collecting comprehensive gender identity data via student records is not currently possible and raises unique student privacy and safety concerns,7 GLSEN urges the use of student self-reports instead of (exclusively) relying on student records and implementation of existing measures for teacher self-reports.
Specifically, GLSEN recommends the following:
The NAEP student survey should include measures that allow students to self-report their gender identity rather than relying exclusively on student records for gender data. NCES should prioritize the development and testing of such measures in coordination with the U.S. Census Bureau, if needed, including by requesting funds for this purpose;
NCES should allow for accurate reporting of data on nonbinary students through school records where schools have implemented nonbinary-inclusive student records and have enrolled nonbinary students; and
The NAEP teacher survey should include a measure of gender identity like that included on the proposed National Teacher and Principal Survey of 2023-2024 (NTPS of 2023-2024).8
To elaborate on GLSEN’s second recommendation above, it is important to know that at least 14 states allow students to use an “X” or another gender marker besides “M” and “F” and at least nine state education agencies have published data on nonbinary student enrollments state reports.9 Requiring schools participating in the NAEP to report nonbinary students as male or female creates a burden for schools that have implemented more inclusive student information systems and leads to the collection of data that is known to be inaccurate. For example, Corvallis School District in Oregon implemented a more inclusive student information system, allowing students to record their gender as “X” instead of “M” or “F,” with the support of the Oregon Department of Education, only to encounter a problem when ask reporting enrollment data to the U.S. Department of Education. As one school personnel explained, “[t]he federal government wants me to make a decision on whether or not a student is male or female. And I don’t think I should do that.”10
In response to GLSEN’s earlier comment recommending that measures of sexual orientation be included on surveys of students in grades 8 and 12 and for all teachers, NCES stated that “NAEP does not collect information related to sexual orientation. The content of NAEP questionnaires is outlined in the Education Sciences Reform Act (P.L. 107-279).” 20 U.S. Code § 9622(b)(2)(G) directs the Commissioner for Education Statistics “in carrying out the measurement and reporting” for the NAEP to “include information on special groups, including, whenever feasible, information collected, cross tabulated, compared, and reported by race, ethnicity, socioeconomic status, gender, disability and limited English proficiency.”
Collecting data on sexual orientation is not prohibited by the Education Sciences Reform Act or any other law and no special provisions are required for including a measure of sexual orientation on the NAEP survey secondary students.11 In light of the Federal Evidence Agenda on LGBTQI+ Equity and President Biden’s Executive Order on advancing equality for lesbian, gay, bisexual, transgender, queer, and intersex individuals12, GLSEN urge NCES to:
Include a measure of sexual orientation on the NAEP questionnaire for students in grade 12 that allows students to self-report their sexual orientation. NCES should prioritize the development and testing of such measures by requesting funds for this purpose and coordinating with the U.S. Census Bureau, if needed, including by requesting funds for this purpose;13 and
Add the measure of sexual orientation included on the proposed NTPS of 2023-2024 to the NAEP teacher questionnaire.14
NCES did not respond to GLSEN’s recommendation regarding measuring variations in sex characteristic or intersex status. Comprehensive LGBTQ+ inclusive data collection must include measures for intersex individuals, a population that may comprise as much as 1.7% of the population and faces documented but understudied health and social disparities.15 GLSEN urges NCES to prioritize the development and testing of such measures in coordination with the U.S. Census Bureau and other relevant federal agencies, including by requesting funds for this purpose.
Revising NAEP questionnaires with existing demographic measures to include LGBTQI+ demographic measures will improve our understanding of how well our schools are meeting the needs of all students. Thank you for considering these recommendations. To discuss the recommendations in this comment, please contact me at [email protected].
Sincerely, Aaron Ridings
Chief of Staff and Deputy Executive Director for Public Policy and Research
----------------------------------------------------------------
Dear Mr. Ridings,
Thank you for your feedback dated March 2023, responding to a 30-day request for comments on the proposed National Assessment of Educational Progress (NAEP) 2024 Clearance Package. The National Center for Education Statistics (NCES) appreciates your interest in NAEP and NCES data collections as well as the references you provided as part of your feedback.
NCES recognizes the importance of providing response options reflective of our nation’s population, including those who do not identify as exclusively male or female. As such, we are actively working towards including more gender identity options in future NAEP data collections both from school records (where we collect student gender information) and teacher self-reports via the teacher survey questionnaire. In support of this, we are pursuing ways to disaggregate student record data into binary and non-binary, as well as working with the National Assessment Governing Board (NAGB) to add a gender identity option to the teacher questionnaires. Both initiatives—supporting accurate reporting of data on nonbinary students through school records where schools have implemented nonbinary-inclusive student records and including on the teacher questionnaire a gender identity question that allows for responses besides “male” or female,” as has recently been proposed for the 2023-24 National Teacher and Principal Survey (NTPS)—are currently under review for use in the 2024 NAEP assessments. We anticipate including them in later amendments to this package. We appreciate your valuable feedback and will consider it during our reviews of these items.
NCES does not ask students about gender or sexual orientation on studies like NAEP because of concerns about the privacy and comfort of the responding students, who are taking these assessments and surveys in classrooms with many other students and adults present. NAEP currently does not collect other data from students that could affect their access to programs and facilities within their schools.
The drafted NAEP teacher survey questions do not include the NTPS questions that measure differences between gender assigned at birth and current gender identify or that measure sexual orientation. This is because of important differences in the purposes of the two studies. NTPS is intended to provide a nationally representative profile of teachers and the teaching profession in the Unites States. Teacher respondents for NTPS know that they are responding to a questionnaire intended to provide this profile. NAEP is not nationally representative for teachers. Teachers are asked to participate in NAEP so that they can provide contextual information to help understand student achievement. We know from other Federal data collections that context is important for respondents when it comes to sexual orientation measures16. The authorizing legislation for NAEP17, Section 303(b)(5), limits data collection in NAEP to questions that are directly related to student academic achievement. NCES believes a better place to collect data on sexual orientation is through our sample surveys, which are designed for this kind of research.
I want to thank you again for your work in supporting federal efforts to improve equitable data collection on sexual orientation and gender identity (SOGI). As you may know, the NCES Commissioner Dr. Peggy Carr is a co-Chair of the White House Equitable Data Working Group Subcommittee on Equitable Data, and several other NCES staff serve on the Subcommittee on SOGI data, working on efforts to produce more robust data while ensuring important safeguards for respondents. We will continue these efforts in the years to come.
Sincerely,
Daniel J. McGrath, Ph.D.
Delegated Authority of Associate Commissioner
Assessment Division
National Center for Education Statistics
Tel: 202 710 8753
1 GLSEN uses “LGBTQ+” to refer to sexual and gender minority populations, including but not limited to individuals who are lesbian, gay, bisexual, transgender, queer, nonbinary, Two-Spirit, and intersex.
2 Subcommittee on Sexual Orientation, Gender Identity, and Variations in Sex Characteristics (SOGI) Data Subcommittee on Equitable Data, “Federal Evidence Agenda on LGBTQI+ Equity,” January 2023, available at https://www.whitehouse.gov/wp- content/uploads/2023/01/Federal-Evidence-Agenda-on-LGBTQI-Equity.pdf.
3 Johns, M. M., Lowry, R., Haderxhanaj, L. T., Rasberry, C. N., Robin, L., Scales, L., Stone, D., & Suarez, N. A. (2020). Trends in Violence Victimization and Suicide Risk by Sexual Identity Among High School Students - Youth Risk Behavior Survey, United States, 2015-2019. MMWR supplements, 69(1), 19–27. https://www.cdc.gov/mmwr/volumes/69/su/su6901a3.htm?s_cid=su6901a3_w. Johns MM, Lowry R, Andrzejewski J, et al. Transgender Identity and Experiences of Violence Victimization, Substance Use, Suicide Risk, and Sexual Risk Behaviors Among High School Students — 19 States and Large Urban School Districts, 2017. MMWR Morb Mortal Wkly Rep 2019;68:67–71. DOI: http://dx.doi.org/10.15585/mmwr.mm6803a3
4 Kosciw, J. G., Clark, C. M., & Menard, L. (2022). The 2021 National School Climate Survey: The experiences of LGBTQ+ youth in our nation’s schools. New York: GLSEN. https://www.glsen.org/research/2021-national-school-climate-survey.
5 Institute for Education Science, National Center for Education Statistics (NCES). Survey Questionnaires: Questionnaires for Students, Teachers, and School Administrators. https://nces.ed.gov/nationsreportcard/experience/survey_questionnaires.aspx (Accessed January 4, 2023).
6 NCES. (2023) Response to 60 Day Comments. Available at: https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=202211-1850-002.
7 Washick, B., Ridings, A., Juste, T. (2022). LGBTQ+ Data Inclusion: Advancing Intersectional Equity in K-12 Education Systems (Issue Brief). New York and Washington, DC: GLSEN. Available at https://eric.ed.gov/?id=ED623159.
8 National Center for Education Statistics (NCES), Department of Education (ED). Agency Information Collection Activities; Comment Request; National Teacher and Principal Survey of 2023-2024 (NTPS 2023-24) Data Collection. 88 Fed. Reg. §14 (January 23, 2023). https://www.federalregister.gov/documents/2023/01/23/2023-01157/agency-information-collection-activities- comment-request-national-teacher-and-principal-survey-of. NTPS instrument available at https://www.regulations.gov/document/ED-2023-SCC-0019-0004.
9 California, Connecticut, District of Columbia, Illinois, Maryland, Massachusetts, New Jersey, New Mexico, New York, Oregon, Pennsylvania, Utah, Virginia, and Washington currently allow for the use of an “X” or another gender marker other than “M” or “F.” The state education agencies of California, Connecticut, District of Columbia, Illinois, Massachusetts, New Jersey, New Mexico, Oregon, Utah, and Washington have published data on nonbinary student enrollments in state reports.
10 Bryant, L. “Nonbinary students aren’t reflected in federal civil rights data. That might change.” May 10, 2022, Chalkbeat. https://www.chalkbeat.org/2022/5/10/23063639/nonbinary-student-federal-civil-rights-data-collection.
11 The Protection of Pupils Rights Amendment requires prior parental consent before a student is required to “submit to a survey, analysis, or evaluation that reveals information concerning” that includes enumerated topics. Sexual orientation and variations in sex characteristics are not among the enumerated topics. However, this further evaluation of topics enumerated under the PPRA is irrelevant on surveys that are voluntary. As noted in the NAEP FAQ, “Federal law specifies that NAEP is voluntary for every student, school, school district, and state.” NCES. (N.D.) NAEP Participation. https://nces.ed.gov/nationsreportcard/participating/.
12 Executive Order 14075 of June 15, 2022. Advancing Equality for Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex Individuals. 87 Fed. Reg. §118 (June 21, 2021). https://www.govinfo.gov/content/pkg/FR-2022-06-21/pdf/2022-13391.pdf.
13 GLSEN’s Research Brief, Considerations for Measuring Sexual Orientation and Gender Identity in Surveys of Secondary School Students, and earlier comment provides sample measures of sexual orientation for use in general population surveys of secondary students. Clark, C. M. & Kosciw, J. G. (2022). Considerations for Measuring Sexual Orientation and Gender Identity in Surveys of Secondary School Students (Research Brief). New York: GLSEN. https://www.glsen.org/SOGI-measurement.
14 NCES ED. Agency Information Collection Activities; Comment Request; NTPS 2023-24 Data Collection. 88 Fed. Reg. §14 (January 23, 2023). https://www.federalregister.gov/documents/2023/01/23/2023-01157/agency-information-collection-activities- comment-request-national-teacher-and-principal-survey-of. NTPS instrument available at https://www.regulations.gov/document/ED-2023-SCC-0019-0004.
15 Rosenwohl-Mack, A., Tamar-Mattis, S., Baratz, A. B., Dalke, K. B., Ittelson, A., Zieselman, K., & Flatt, J. D. (2020). A national study on the physical and mental health of intersex adults in the US. PloS one, 15(10), e0240088. Zeeman, L., & Aranda, K. (2020). A systematic review of the health and healthcare inequalities for people with intersex variance. International Journal of Environmental Research and Public Health, 17(18), 6533. National Academies of Sciences, Engineering, and Medicine. (2020). Understanding the Well-Being of LGBTQI+ Populations. Washington, DC: The National Academies Press. Available at https://doi.org/10.17226/25877. National Academies of Sciences, Engineering, and Medicine. (2022). Measuring Sex, Gender Identity, and Sexual Orientation. Washington, DC: The National Academies Press. Available at https://doi.org/10.17226/26424.
16 See, for example: The Impact of Sexual Orientation and Gender Identity (SOGI) and Disability Questions on Response Rates: Initial Results, https://www.fcsm.gov/assets/files/docs/2022-conference-docs/H2.3_Young.pdf
17 TITLE III- NATIONAL ASSESSMENT OF EDUCATION PROGRESS https://www.nagb.gov/naep/the-naep-law.html
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Spiegelman, Maura |
| File Modified | 0000-00-00 |
| File Created | 2023-09-22 |