NE 5.1.3 Supporting Statement A rev3 comments and response

Regional Education Laboratory Northeast & Islands

Supporting Statement Part A

Table of Contents

Justification 3

1. Circumstances Necessitating Collection of Information 3

Statement of Need 3

Overview of Study Design 4

Overview of Data Collection 7

2. How, by Whom, and for What Purpose Information Is to Be Used 8

3. Use of Automated, Electronic, Mechanical, or Other Technological Collection Techniques 8

4. Efforts to Avoid Duplication of Effort 9

5. Sensitivity to Burden on Small Entities 9

6. Consequences to Federal Program or Policy Activities if the Collection Is Not Conducted or Is Conducted Less Frequently Than Proposed 10

7. Special Circumstances 10

8. Federal Register Announcement and Consultation 10

9. Payment or Gift to Respondents 10

10. Data Confidentiality 11

11. Additional Justification for Sensitive Questions 12

12. Estimates of Hour Burden 12

13. Estimate of Total Cost Burden to Respondents or Record-Keepers 15

14. Estimates of Annualized Cost to the Federal Government 16

15. Reasons for Program Changes or Adjustments 16

16. Plan for Tabulation and Publication and Schedule for Project 16

17. Approval Not to Display the Expiration Date for OMB Approval 17

18. Exception to the Certification Statement 17

References 17

Justification

The U.S. Department of Education (ED) requests OMB clearance for data collection related to the Regional Educational Laboratory (REL) program. ED, in consultation with REL Northeast & Islands (REL-NEI) under contract 91990022C0013 has planned a study of the efficacy of a professional development (PD) course for fourth and fifth grade math intervention teachers in the state of Massachusetts. Researchers at WestEd will carry out a school-level randomized controlled trial (RCT) to evaluate the efficacy of the PD course and investigate how and whether the PD course is helping teachers improve their practices and boost student learning. OMB approval is being requested for REL-NEI’s data collection for this project, including pre and post measures of student skills and attitudes, and teacher knowledge, beliefs, and feedback regarding instructional practice and implementation of the PD course. 

The study will also draw upon administrative data that the Massachusetts Department of Elementary and Secondary Education (DESE) collects, including student characteristics such as race, ethnicity, special education eligibility, multilingual learner status, economic disadvantage status, and prior achievement on the state’s standardized assessment. OMB approval is not being sought for this administrative data since these are collected by the state and not REL-NEI. 

1. Circumstances Necessitating Collection of Information

This data collection is authorized by the Educational Sciences Reform Act (ESRA) of 2002 (see Appendix A). Part D, Section 174(f)(2) of ESRA states that as part of their central mission and primary function, each regional educational laboratory “shall support applied research by . . . developing and widely disseminating, including through Internet-based means, scientifically valid research, information, reports, and publications that are usable for improving academic achievement, closing achievement gaps, and encouraging and sustaining school improvement, to—schools, districts, institutions of higher education, educators (including early childhood educators and librarians), parents, policymakers, and other constituencies, as appropriate, within the region in which the regional educational laboratory is located.”

Statement of Need

Nationally, elementary math achievement is a topic of national concern. Students who leave elementary school with a strong foundation in mathematics, particularly fractions, are better prepared to succeed in middle school mathematics and algebra (National Mathematics Advisory Panel, 2008). Algebra is a well-documented gatekeeper to advanced high school math coursework, which is consistently linked to increased post-secondary opportunities and earnings (Goodman, 2019). On the 2019 NAEP report card, only 41% of grade 4 students scored Proficient or above in math, and these numbers are lower for students from marginalized backgrounds (NCES, 2019). Unfortunately, these opportunity gaps persist at the secondary level, highlighting the importance of addressing these gaps as soon as they appear. 

Given this need, district leaders in Massachusetts are seeking ways to bolster learning for students who struggle with mathematics. Some of the approaches districts have adopted to achieve this goal target core instruction, but districts are also interested in strengthening their approaches to intervention.

Together with the Massachusetts Partnership to Support Student Learning Through Math Intervention, REL-NEI is designing a PD course to build educators’ knowledge of and ability to implement the recommendations of a new What Works Clearinghouse (WWC) Practice Guide, Assisting Students Struggling with Mathematics: Intervention in the Elementary Grades (Fuchs et al., 2021). The recommendations include providing systematic instruction; using clear and concise mathematical language; using different types of representations, including the number line, to deepen student thinking and prepare them for advanced mathematics; providing deliberate instruction on word problems; and using timed activities to build fluency. By using the course’s extensive resources, districts will have the tools to provide teachers with in-depth PD on the recommendations and how to implement them by using specific instructional practices. Teachers will learn to use these practices effectively, helping them to optimize intervention time to improve student learning. Their students will be actively engaged in doing math, communicating their ideas, and progressing toward clear learning goals with feedback and support. 

Although the WWC Practice Guide on which the PD course is based is relevant for students in grades 3-6, the study will focus on grades 4 and 5 for two reasons. First, it is unlikely that students in grade 3 would have prior math achievement scores to be used as a covariate in the proposed impact analysis (testing often begins in grade 3). Second, students in grade 6 are often in middle school, which would complicate the implementation of the PD course. The course emphasizes fractions, rather than all topics in grades 4 and 5, because the topic is a major curricular focus in those grades and is critical to success in the gatekeeper course of algebra in middle and high school.

Overview of Study Design

The proposed efficacy study will measure the implementation and impact of the PD course on teacher and student outcomes following the logic model in exhibit 1.

Exhibit 1. Study Logic Model

A. Inputs: Core Components	B. Outputs: Use of Components	C. Short-Term Outcomes: Teacher/Leader Knowledge	D. Medium-Term Outcomes: Teacher/Leader Practices		E. Long-Term Outcomes: Student Outcomes
PD Course Resources A1. Teachers - 1 Intro session (sync) - 5 Online sessions (async) w/videos, reading, math tasks - Participant Guide with: - PD overview & handouts - 10 PLC session resources - 5 Try It! Assignments A2. Leaders/Facilitators - Intro video for Leaders - Facilitator Guide with slides, agendas, PLC protocols, handouts, tools, tips	B1. Teachers participate in: - 16 Course sessions (A1) - 1 Intro session - 5 Online sessions - 10 PLC sessions - 5 Try It! assignments (A1) B2. Leaders/Facilitators use: - Intro video & online sessions to learn about practices (A1, A2) - Facilitator Guide to lead Intro session, 6 PLC sessions (A2)	C1. Increase in teachers’ mathematical knowledge for teaching (MKT) on the math topics covered by the Toolkit (A1) C2. Increase in teachers’ knowledge of evidence-based strategies for supporting student learning (A1) C3. Increase in teachers’ self-efficacy for using evidence-based practices with students (A1)	D1. Increase in teachers’ ability to implement the evidence-based strategies in math intervention (A1) D2. Demonstrate use of practices with students in math intervention (A1)		E1. Positive change in students’ self-efficacy for learning mathematics concepts and skills E2. Increase in student ability to use the focus strategies of connecting representations for fractions, using number lines for fractions, and solving word problems. E3. Increase in student knowledge of fractions

The study’s research questions address both impact and implementation, with the first two questions designated as confirmatory:

• RQ1a) What is the impact of the PD course on students’ self-efficacy toward math?

• RQ1b) What is the impact of the PD course on students’ math knowledge of fractions?

• RQ2a) What is the impact of the PD course on teachers’ self-efficacy for assisting students with mathematics difficulties?

• RQ2b) What is the impact of the PD course on teachers’ math knowledge for teaching fractions?

• RQ2c) What is the impact on the PD course on teachers’ instructional practices designed to support students receiving math intervention?

• RQ3a) To what extent is the PD course implemented with fidelity?

• RQ3b) What challenges did teachers face in implementing the PD course activities and how did they overcome those challenges?

The school-level design will randomly assign schools to participate in the PD course for grade 4 and 5 (intervention group) teachers or continue business-as-usual professional learning activities for grade 4 and 5 teachers (comparison group). The proposed study will be conducted in 60 schools across 15 districts in Massachusetts. Based on a scan of schools in partnership districts, we are assuming an average of 1.5 teachers and 30 grade 4 and grade 5 students who are eligible to participate in the intervention per school. These assumptions mean that the proposed efficacy study would include roughly 90 teachers and 1,800 students.

The PD course includes both online self-paced activities and in-person activities and includes five modules that are designed to be delivered over 6–7 months. It begins with an introductory online session (2 hours) followed by modules spaced about 4–5 weeks apart that focus on each respective WWC practice guide recommendation. Each module begins with teachers participating in an online session to learn about the recommended strategies through readings, videos, examples, and math activities. Each self-paced online session will provide 2 hours of asynchronous PD activities to be completed over 2 weeks, offering teachers flexibility to work at times of their choosing and at their own pace. Next, teachers will participate in a professional learning community (PLC) session (in-person or virtual) to discuss the recommendation and prepare to apply it with their students. After the PLC session, teachers will apply the strategies with students by using the Try It! Assignment. All the professional development content is contained in PD course and the PD course resources include all materials necessary for facilitators to lead a PLC session and implement the PD course. The PD course sequence is presented in exhibit 2.

Exhibit 2. PD Course Sequence

Topic	Intro	Module 1 Mathematical Language	Module 2 Representations	Module 3 Number Line	Module 4 Word Problems	Module 5 Systematic Instruction
PD Format	Intro Session	Online 1 PLC 1-A Try It! 1 PLC 1-B	Online 2 PLC 2-A Try It! 2 PLC 2-B	Online 3 PLC 3-A Try It! 3 PLC 3-B	Online 4 PLC 4-A Try It! 4 PLC 4-B	Online 5 PLC 5-A Try It! 5 PLC 5-B

The estimated time for teachers to complete the full PD course is 30 hours. This design reflects research recommendations for PD to consist of at least 20 hours (Garet et al., 2016; Yoon et al., 2007), which allows time for teachers to learn new content and apply those learnings in their classrooms. The content and design of the PD course is also well-aligned with one of the few causal studies of a PD program for elementary math teachers that improved student outcomes. Perry and Lewis (2011) evaluated a lesson study resource kit for elementary teachers that focused on the linear measurement of fractions using the number line. The study included two counterfactual conditions, lesson study on a topic other than fractions (lesson study only) and business-as-usual professional development (BAU). Using a researcher-created end-of-year fractions assessment, the study found effects of 0.84 (lesson study only) and 0.71 (BAU) for students in grades 2, 3, and 5. Thus, providing elementary math teachers with PD that is more than 20 hours and includes resources to support fractions and use of the number line supports the pacing and course components outlined in exhibit 3.

Exhibit 3. PD Hours and Pacing of Course Components

Course Component	PD Hours and Pacing
Introductory Session	2 hours (in-person/virtual with online components)
5 Online Sessions	10 hours (2 hours per session over 2–3 weeks; online asynchronous)
10 PLC Sessions	12.5 hours (1–1.5 hours per session; meets every 2 weeks; in-person or virtual)
5 Try It! Assignments	5 hours (1 hour each; planning/reflection time, not teaching time)
Total PD hours	30 hours over 6–7 months

Overview of Data Collection

Several data sources will be used to answer the study’s impact and implementation questions during the 2024/25 school year. The student and teacher impact measures (RQs 1 and 2) will be administered before and after the PD course is implemented. This allows for comparisons between treatment and comparison students and teachers after the PD course has been implemented, and pre-post comparisons for students and teachers in the treatment group.

To assess the impact of the PD course on students, the study will administer one survey measure of students’ self-efficacy (RQ1a) and one measure of students’ fraction knowledge (RQ1b). These two measures will be administered by the study team in early fall of 2024, before the PD course is implemented, and again in spring 2025, after the PD course has been completed by intervention group teachers.

Similarly, to assess the impact of the PD course on teacher outcomes, the study will administer a survey and a knowledge assessment. The survey will include validated constructs of teachers’ self-efficacy (RQ2a) and instructional practice (RQ2c). The knowledge assessment focuses on teachers’ understanding of teaching fractions. The process for developing and validating items for the knowledge assessment involved (1) drafting items at meetings of various stakeholders, (2) vetting the items by the research team, and (3) review by mathematicians. Successful items were then piloted by hundreds of teachers, and resulting correlations and distributions were reviewed by the research team. Successful items were then placed on forms, balancing for content, difficulty, and shape of information curve. The survey and knowledge assessment will be administered by the study team before and after the PD course is implemented – in summer of 2024 and spring of 2025, respectively.

To answer the study’s implementation questions, the study team will use data from intervention group teachers’ instructional logs, which they will complete after each of the first four PD modules; the teacher survey; archival data from the online components of the PD course; and observational data. These descriptive data will be used to determine the extent to which the PD course was implemented with fidelity (RQ3a) and describe potential challenge teachers experienced with implementing the PD course (RQ3b), to inform potential future improvements. The implementation data, along with data from the teacher survey, will also be used to describe the service contrast between the PD experienced by intervention and comparison group teachers during the 2024/25 school year.

The WestEd study team will provide an online training and written guidance to support intervention and comparison group teachers in administering the student survey and knowledge assessment in their classrooms before and after the PD course has been implemented. The student survey will be able to be completed online or with pencil and paper. The student knowledge assessment will be administered with pencil and paper.

The WestEd study team will administer the teacher survey online to all treatment and comparison teachers before and after the PD course is implemented. The WestEd study team will administer the teacher knowledge assessment in-person following the same pre-post PD course implementation timeline. The data collection timeframe is presented in exhibit 4, and copies of all student and teacher measures are included in attachment A.

Exhibit 4. Timeline for Data Collection

Timeframe	Data Collection
Summer 2024	Administer teacher measures (pre-)
Early Fall 2024	Administer student measures (pre-)
Spring 2025	Administer teacher and student measures (post-)

The proposed data collection timeline in exhibit 4 is dependent upon OMB approval. No data collection will occur until and after the OMB package is cleared. As previously described, the study will collect administrative data that DESE provides as part of the impact study design. These student characteristics include race, ethnicity, special education eligibility, multilingual learner status, economic disadvantage status, and prior achievement on the state’s standardized assessment. Collecting these administrative records does not require OMB clearance. 

2. How, by Whom, and for What Purpose Information Is to Be Used

The findings of this study will be used by state and district decision-makers to inform decisions regarding the types of professional development opportunities to provide to improve teaching and learning for elementary students who struggle with mathematics. The study will result in a report intended for district and state leaders who are responsible for supporting high quality teaching and learning for elementary students, and particularly those who struggle with mathematics. Study findings will be primarily relevant to educators in Massachusetts, but also a secondary audience of educators across the country who are also working to support intervention teachers and improve achievement for elementary students who struggle in mathematics. The study’s impact findings can be used to inform the decision-making of state- and district-level policymakers and program developers. For example, if the study produces positive effects on student learning, these leaders will be able to make more informed decisions about the types of professional learning initiatives for math intervention teachers and programs they sponsor. They might shift the focus or intensity of current professional learning offerings to correspond more closely to the model that was evaluated. Similarly, school-based instructional leaders and teachers might also use the impact and implementation findings to inform how they design, implement, and refine their math intervention programs. Finally, and regardless of the study’s impact findings, the study team will collect data from teachers on what they found most useful about the PD course, what was less useful, and what could be improved. Key recommendations will be included in the final report and shared with the PD course development team.

3. Use of Automated, Electronic, Mechanical, or Other Technological Collection Techniques

The contractor’s proposed data collection plan simultaneously addresses issues related to efficiency, accuracy, and burden. For example, it is likely more efficient for teachers to complete surveys online than with pencil and paper as those data are automatically and securely shared with the research team. Teachers may also prefer to use online surveys with students, depending on how they use technology in their classrooms. The study’s electronic data collection tools include:

• An electronic data collection tool that allows teachers to securely transfer informed consent from teachers and parents/guardians.

• An electronic file transfer protocol site that allows DESE, districts, and schools, to securely and efficiently share administrative records to ED’s contractor.

• Online data collection instruments that allow for the secure and efficient transfer of teacher measures and student measures (in schools that do not opt for paper and pencil measures).

• E-mail systems maintained by districts/schools and ED’s contractor that allow the sharing of unique links to electronic data collection forms for teachers and students.

For any paper and pencil measures, ED’s contractor will mail copies to schools, with return shipping provided. All electronic data collection forms will be made 508-compliant and approved by IES prior to use.

4. Efforts to Avoid Duplication of Effort

This project uses extant administrative data for students, teachers, and schools to the greatest extent possible. As previously mentioned, archival records (e.g., student demographic characteristics, school characteristics) will be provided by the state or district to minimize burden on schools and teachers. To execute the study’s rigorous design, ED’s contractor does need to collect new data from teachers and in intervention and comparison schools, including measures of teachers’ and students’ knowledge of fractions. The study’s implementation data is also needed to assess fidelity of students’ knowledge of fractions and treatment and comparison teachers and students.

5. Sensitivity to Burden on Small Entities

This project is designed to support math intervention teachers, who often work with small numbers of students in grades 4 and 5. Given this context, we are offering both electronic and paper and pencil data collection formats for the student measures. For example, it might be more efficient to print out the short student survey to the 8 students in the intervention class and return those instruments in the pre-paid envelope for teachers in schools or classrooms with limited technology access. However, it is more likely that the electronic teacher surveys will be more efficient for teachers to complete since they have the technology and can submit those data automatically. Our collaboration with the state and districts to collect administrative records also supports the work of teachers who work with small numbers of students. Taken together, this OMB package requests the minimum amount of data needed to answer the study’s research questions.

6. Consequences to Federal Program or Policy Activities if the Collection Is Not Conducted or Is Conducted Less Frequently Than Proposed

The Education Science Reform Act of 2002, Part D, Section 174 states that the central mission and primary function of the RELs includes supporting applied research and providing technical assistance to state and local education agencies within their region (20 U.S.C. 9564). Failure to approve the data collections related to the evaluation of the PD course will jeopardize this attempt to study this program and thereby prevent the REL-NEI contractor from fulfilling its mission.

As described under question 2, the findings of this study will also be potentially useful to state and district decision-makers regarding the types of professional development opportunities to provide for teachers of elementary students who struggle with mathematics. Findings will also be relevant to educators in other states who are seeking evidenced-based approaches for math intervention teachers and their students, as well as researchers who are interested in expanding the research base on rigorous studies of professional development. The final PD course will reflect suggested improvements by study participants, which will benefit other educators who use the resources.

7. Special Circumstances

This OMB clearance request full complies with regulations as it does not include any of the stipulated special circumstances.

8. Federal Register Announcement and Consultation

1. Federal Register Announcement

A 60-day notice was published in the Federal Register (88 FR 37522) on June 8, 2023, providing an opportunity for public comments. No public comments were received. A 30-day notice will be published to further solicit comments. ED will respond to both public and OMB questions, if any, and summarize the responses under 8a.

2. Consultations Outside the Agency

The REL-NEI contractor consulted with a Subject Matter Expert (SME) with expertise in research methodology during the proposal for this study. The SME reviewed and provided actionable feedback on the proposal, which the contractor and REL-NEI summarized for ED as the proposal was revised and finalized.

c. Proposal Review

All studies funded by the Regional Education Laboratories are subjected to external peer review and must receive IRB approval before they are launched. The study was approved by the WestEd IRB on May 11, 2023. The IRB determined the research to be exempt because it involves established educational settings and normal educational practices (i.e., professional development) that are not likely to adversely impacts students' opportunities to learn required educational content.

9. Payment or Gift to Respondents

To motivate study participation, the study will provide stipends to the teachers to compensate them for time spent in PD activities outside of the school day, currently estimated at approximately 11 hours and time spent completing data collection requests (approximately 3 hours) at a rate of $60/hour, for a total of $840. The stipends will be distributed after fall data collection and then again in the spring, with half the total amount distributed at each time period. Teachers in both study conditions will receive the stipend for data collection requests during the study implementation year, and teachers in the intervention condition will receive the stipend associated with PD activities outside their standard workday during the study implementation year also. The intervention will be provided to teachers in the comparison condition in the following year, at which point they will also receive the stipend associated with PD activities outside their standard workday.

10. Data Confidentiality

ED’s contractor for REL-NEI will follow the policies and procedures required by ESRA of 2002, Title I, Part E, Section 183. This requires “All collection, maintenance, use, and wide dissemination of data by the Institute” to “conform with the requirements of section 552 of title 5, United States Code, the confidentiality standards of subsection (c) of this section, and sections 444 and 445 of the General Education Provision Act” (20 U.S.C. 1232g, 1232h). These citations refer to the Privacy Act, the Family Educational Rights and Privacy Act, and the Protection of Pupil Rights Amendment.

In addition, for student information, ESRA states: “The Director shall ensure that all individually identifiable information about students, their academic achievements, their families, and information with respect to individual schools, shall remain confidential in accordance with section 552a of title 5, United States Code, the confidentiality standards of subsection (c) of this section, and sections 444 and 445 of the General Education Provision Act.” Subsection (c) of section 183 requires the Director of IES to “develop and enforce standards designed to protect the confidentiality of persons in the collection, reporting, and publication of data.”

Subsection (d) of section 183 prohibits disclosure of individually identifiable information as well as making the publishing or communicating of individually identifiable information by employees or staff a felony. All documents, consent forms, instruments, notification letters, and email reminders will provide the following language to inform research participants of the penalties to researchers for disclosing individually identifiable information.

Per the policies and procedures required by the Education Sciences Reform Act of 2002, Title I, Part E, Section 183, responses to this data collection will be used only for statistical purposes. The reports prepared for this study will summarize findings across the sample and will not associate responses with a specific school, district, or individual. Any willful disclosure of such information for nonstatistical purposes, except as required by law, is a class E felony.

The contractor for REL-NEI will protect the confidentiality of all information collected for the study and will use it for research purposes only. Paper files will be stored in a locked file cabinet and all digital files will be password protected so that only project researchers can access it. To protect confidential data, only the contractor’s data management staff, investigators, and research staff will have access to the data files on a “need-to-know” basis. Any identifiable variables, raw data, or derived variables will be stored in encrypted files on a secure data management site. Access to this site will be limited to staff assigned to the project. Any data obtained for this study will be used only for statistical and descriptive analyses. All identifiers will be destroyed as soon as they are no longer required. Study reports will not identify the name of any specific analysis unit (e.g., students, school staff members, or schools). In no case will information be reported when the total number for a quantity represents fewer than four cases. Moreover, any data that permit identity disclosure, when used in combination with other known data, will not be published or made available in restricted-use files.

All members of the study team have obtained their certification on the protection of human subjects in research and will also have required federal security clearances. The REL study team will submit to the NCEE security officer a list of the names of all people who will have access to respondents and data. All staff members working on the project who have access to the data or to respondents will be required to sign a confidentiality pledge and affidavits of non-disclosure. The project team will track new staff and staff who have left the study and ensure that additional signatures will be obtained, or clearances will be revoked.

Teacher participants and student guardians will be informed that project staff are committed to keeping data confidential, and that participation in the data collection activities is voluntary. The instructions for all online surveys will reiterate these points.

11. Additional Justification for Sensitive Questions

None of the study instruments contain questions that are highly sensitive in nature. All respondents will be informed that their responses are voluntary and may choose to answer any question.

12. Estimates of Hour Burden

The annualized number of responses is 9,630 (see exhibit 5). Annualized reporting burden associated with this data collection is 3,106 hours. This burden estimate includes the time required for completing consent forms and completing all student and teacher-level measures (e.g., reporting). This request does not involve any burden related to record keeping or third-party disclosure. For each data collection, the burden was estimated based on the contractor’s performance of similar collections and confirmed through pilot testing with former educators. To be conservative, the reporting burden estimates assume response rates of 100 percent. The hourly wage rates for parents and school staff are based on mean wage rates in Massachusetts reported by the Bureau of Labor Statistics (2021). For parents, the overall median hourly wage rate in Massachusetts is used ($25.21). To estimate an hourly wage rate for teachers ($58.19), we divided the mean annual wage for an elementary school teacher ($83,790) by 180 8-hour work days. Because students will take the survey and assessment during school hours, it is assumed that no costs will result from students participating in the data collection.

Exhibit 5. Hour Burden Estimates

CONSENT FORMS All teachers will be asked to complete a consent form prior to participating in the study. The parents and guardians of participating 4th and 5th grade students will receive an information sheet that describes the study with instructions to return the form if they do not want their child(ren) to participate in student data collection activities. The burden calculation assumes up to 10 students per teacher (for a total 1,800 students). For the purpose of calculating burden, the initial consent form burden requires an estimate of 5 minutes of time per respondent.
Instrument	Person Incurring Burden	Number of Respondents	Responses per Respondent	Hours per Response	Total Burden (Hours)
Teacher Consent Forms	Teachers	90	1	0.08	7.2
Parent Information Letter	Parents	1,800	1	0.08	144
TEACHER MEASURES All treatment and comparison teachers will complete a survey pre- and post-intervention. The pre-survey includes teacher background characteristics and measures of self-efficacy and instructional practices. The post-survey is the same except it does not include background characteristics. Treatment and comparison teachers will also complete a fraction knowledge assessment pre- and post-intervention. To measure fidelity, the treatment teachers will complete an instructional log after the first four modules of the PD course. The teacher pre-survey is estimated to take no longer than 30 minutes to complete. The teacher post-survey is estimated to take no longer than 20 minutes to complete. The teacher knowledge of fractions assessment is estimated to take no longer than 45 minutes to complete. Each instructional log is estimated to take no longer than 15 minutes for treatment teachers to complete.
Instrument	Person Incurring Burden	Number of Respondents	Responses per Respondent	Hours per Response	Total Burden (Hours)
Teacher survey (pre): background, self-efficacy, and instructional practices survey	Teachers	90	1	0.50	45
Teacher survey (post): self-efficacy and instructional practices	Teachers	90	1	0.33	29.7
Teacher knowledge of fractions assessment (pre)	Teachers	90	1	0.75	67.5
Teacher knowledge of fractions assessment (post)	Teachers	90	1	0.75	67.5
Teacher instructional log	Teachers	45	4	0.25	45
STUDENT MEASURES All 4th and 5th grade students in each participating teacher’s math intervention class(es) will be asked to complete a self-efficacy and instructional practices survey and a knowledge of fractions assessment pre- and post-intervention (fall of 2024 and spring of 2025). The survey of self-efficacy and instructional practices is estimated to take no longer than 15 minutes to complete. The student knowledge of fractions assessment is estimated to take no longer than 45 minutes to complete.
Instrument	Person Incurring Burden	Number of Respondents	Responses per Respondent	Hours per Response	Total Burden (Hours)
Student self-efficacy and instructional practices survey (pre)	Students	1800	1	0.25	450
Student self-efficacy and instructional practices survey (post)	Students	1800	1	0.25	450
Student fraction knowledge assessment (pre)	Students	1800	1	0.50	900
Student fraction knowledge assessment (post)	Students	1800	1	0.50	900
Total		9,495	9,630		3,105.9

Table Notes:
*The total number of respondents in this table is the sum of the number of respondents for each information collection activity. Because some individuals will participate in more than one information collection activity, the total number of responses exceeds the total number of individuals who will respond.
**The hours per response was rounded to the second decimal place for display only. Therefore, the total burden may not equal the product of the displayed hours per response, number of respondents, and number of respondents.

The total cost to respondents for this collection, which includes no start-up costs, for the full study is $18,870.20 and presented in Exhibit 6. Because this is a one-time study, there are no plans (or associated costs) for follow-up studies or other data collections outside of what is described in this package.

Exhibit 6. Estimates of Annualized Costs for Respondents

Tasks	Type of Respondent	Total Burden Hours	Hourly Wage Rate*	Monetary Cost of Burden
Teachers consent form	Teachers	7.2	$58.19	$418.97
Parent information letter with consent form	Parent or guardian	144	$25.21	$3,630.24
Teacher Measures Pre-Survey Post-Survey Pre-Fractions Knowledge Assessment Post-Fractions Knowledge Assessment Instructional Log (Treatment only)	Teachers	254.7	$58.19	$14,820.99
Student Measures Pre-Survey Post-Survey Pre-Fractions Knowledge Assessment Post-Fractions Knowledge Assessment	Students	2,700	$0	$0
Total		3,105.9		$18,870.20

^* The hourly wage rates for parents and school staff are based on mean wage rates in Massachusetts reported by the Bureau of Labor Statistics (2021). For parents, the overall median hourly wage rate in Massachusetts is used ($25.21). To estimate an hourly wage rate for teachers ($58.19), we divided the mean annual wage for an elementary school teacher ($83,790) by 180 8-hour work days. Because students will take the survey and assessment during school hours, it is assumed that no costs will result from students participating in the data collection. Because students will take the survey and assessment during school hours, it is assumed that no costs will result from students participating in the data collection.

Note. The total burden hours and wage rates were rounded for display only. Therefore, the total monetary cost may not equal the product of the displayed burden hours and the wage rate.

13. Estimate of Total Cost Burden to Respondents or Record-Keepers

There are no capital/startup costs to respondents, nor are there any annual costs to respondents associated with operating or maintaining systems or purchasing services.

14. Estimates of Annualized Cost to the Federal Government

The annualized cost to the federal government for all project activities is $336,455. The estimated total cost for the five-year project is $1,682,275.

15. Reasons for Program Changes or Adjustments

This is a new study.

16. Plan for Tabulation and Publication and Schedule for Project

The project will produce a 15-page report, a 1-page summary of the study’s implementation and impact findings, and a Restricted Use File (RUF). The RUF will include all of the variables to measure the impact of the PD course on teacher and student outcomes, providing external researchers with the opportunity to replicate findings from REL researchers and potentially answer additional research questions. Administrative data will not be included in the RUF, but instructions on how to obtain those data and how they were used in the analyses will be provided. Furthermore, all RUFs are required to be reviewed by IES’ Disclosure Review Board. The Disclosure Review Board (DRB) is comprised of members from each NCES Division, representatives from IES’ Statistical Standards Program, and a member from each of the Institute of Education Sciences (IES) Centers. The DRB will review disclosure risk analyses conducted by the REL contractor to ensure that data released do not disclose the identity of any individual respondent. The DRB approves the procedures used to remove direct identifiers from RUFs.

The project will also produce several dissemination supporting products that will briefly describe the program implemented in the study and summarize the impacts on teachers and students. The actual vehicle of dissemination will depend on the timing of report submission. The products will likely include a brief that highlights the key findings from the project, a blog post, and a joint webinar hosted by the efficacy study team together with the PD course development team for state- and district-level policymakers, curriculum and professional development directors, and school-based instructional leaders and teachers.

The project schedule is presented below:

Activity/Milestone

2022

01

02

03

04

05

06

07

08

09

10

11

12

Resubmit proposal

D

F

IRB for development

F

Development: PD; DM

X

X

X

X

X

X

X

Initial COR Review (PD, DM)a

X

X

X

X

Initial meeting with RWR

X

RCT: Research proposal, data management plan

D

2023

Development: PD; DM

X

X

X

X

X

X

X

Initial COR Review (PD, DM)

X

X

X

X

X

X

X

X

Usability testing: PD; DM

X

X

X

X

X

X

X

X

X

RCT: Research proposal, data management plan

F

RCT: IRB for evaluation

F

RCT: OMB PRA package

D

F

2024

RCT: Recruitment/data-sharing agreements

X

X

X

X

X

X

X

Submit PD, DM

D

Submit summary of revisions (PD, DM)

F

Development IS

X

X

Submit for COR review: IS

X

Usability Testing: IS

X

X

PD, DM through Phase I review before evaluation

X

Submit IS

D

Submit summary of revisions (IS)

F

RCT: Implement PD course in intervention schools

X

X

X

2025

PD, DM approved

F

IS approved

F

RCT: Implement PD course

X

X

X

X

RCT: Data analysis & report writing

X

X

X

X

X

X

X

X

X

RCT: Submit report

D

Provide PD course to comparison schools

X

X

X

2026

Provide PD course to comparison schools

X

X

X

RCT: Report published

F

Restricted Use File

X

X

X

X

X

F

Note: COR = Contracting Officer’s Representative; D = Draft; F = Final; DM = diagnostic & monitoring instruments; IRB = Institutional Review Board; IS = institutionalizing supports; OMB = Office of Management and Budget; PD = professional development; PRA = Paperwork Reduction Act; RCT = randomized controlled trial; X = Months in which activity will occur.

17. Approval Not to Display the Expiration Date for OMB Approval

The study team does not request approval to display the expiration date for OMB approval.

18. Exception to the Certification Statement

There are no exceptions to the certification statement being sought by the study team.

References

Fuchs, L. S., Bucka, N., Clarke, B., Dougherty, B., Jordan, N. C., Karp, K. S., Newman-Gonchar, R., Schumaker, R. F., Woodward, J., Monroe Gersten, R., Jayanthi, M., Keating, B., & Morgan, S. (2021). Assisting students struggling with mathematics: Intervention in the elementary grades: Educator's Practice Guide. WWC 2021006. National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://files.eric.ed.gov/fulltext/ED611018.pdf

Garet, M. S., Heppen, J. B., Walters, K., Smith, T. M., & Yang, R. (2016). Does content-focused teacher professional development work? (NCEE 2017-4010). National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncee/pubs/20174010/pdf/20174010.pdf

Goodman, J. (2019). The Labor of Division: Returns to Compulsory High School Math Coursework. Journal of Labor Economics, 37(4), 1141–1182. https://doi.org/10.1086/703135

NCES [National Center for Education Statistics]. (2019). The nation’s report card. National Assessment of Educational Progress.

National Mathematics Advisory Panel. (2008). Foundations for Success: The final report of the national mathematics advisory panel, U.S. Department of Education: Washington, DC.

Perry, R., & Lewis, C. (2011, September 1). Improving the mathematics content based of lesson study summary of results. https://www.researchgate.net/publication/228440009_Improving_the_Mathematical_Content_Base_of_Lesson_Study_Summary_of_Resultsi

WWC [What Works Clearinghouse]. (2020). What Works Clearinghouse standards handbook, version 4.1. National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. https://ies.ed.gov/ncee/wwc/handbooks

Yoon, K. S., Duncan, T., Lee, S. W. Y., Scarloss, B., & Shapley, K. (2007). Reviewing the evidence on how teacher professional development affects student achievement (Issues & Answers Report, REL 2007–033). Regional Educational Laboratory Southwest, National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. http://eric.ed.gov/?id=ED498548