Instructional Leader Implementation Log

Appendix E.

Instructional Leader Implementation Logs

Instructional Leader Implementation Log: Module 1

Module 1 of the Toolkit centered on the first of three recommendations for instructional practices. The first recommendation focused on the use of solved problems to engage students in analyzing algebraic reasoning and strategy. Now that you and the teachers you support have completed the Plan-Do-Study-Act (PDSA) cycle for Module 1, we’d like to learn about what went well and what was challenging with supporting teachers in implementing this first recommendation and using the related Toolkit materials. We would also like to ask you a few questions about the “setting the stage” activities you completed before commencing the Module 1 activities.

• We expect it to take you about 15 - 20 minutes to complete this implementation log. Please complete this log by [DATE].

• As an instructional leader who is receiving access to the Toolkit and supporting teachers using the Toolkit, your insights are valuable, and we hope you will take the time to complete this log. You may skip any questions you are not comfortable answering; however, we hope that you answer as many as you can.

• The information you provide in this implementation log will help us understand how instructional leaders used the resources and supported teachers implementing the recommendations in their classrooms. Your insights will also inform future improvements to the Toolkit. Any reporting prepared for this study will summarize findings across participating districts and will not associate responses with a specific district, school or individual.

• All data collected for this study will be kept confidential, except as required by law. The researchers conducting this study follow the confidentiality and data protection requirements of the U.S. Department of Education’s Institute of Education Sciences (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183).

• If you complete an implementation log for each of the four Toolkit modules, you will receive a $60 gift card to thank you for your time and effort.

q START IMPLEMENTATION LOG

Section I: Setting the stage activities and resources

In this first section, we’d like to learn about your experience conducting the “Setting the Stage” activities with the teachers you support and your use of the related resources. As a reminder, the “Setting the Stage” activities took place before Module 1, providing teachers with an overview of the recommendations and the PDSA cycle.

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over Setting the Stage: “The session in which the facilitator introduces teachers to the Toolkit and its resources.”]

I.1. How useful was the “Setting the Stage” facilitator guide for preparing you to conduct the “Setting the Stage” PLC session? Mark one only

0. I did not receive the Setting the Stage facilitator guide

1. I received the Setting the Stage facilitator guide but did not use it

2. Not useful

3. Somewhat useful

4. Useful

5. Very useful

[ALL]

I.2. How useful were the “Setting the Stage” activities in helping you understand the PDSA cycle so that you can guide teachers through the process?

1. Not useful

2. Somewhat useful

3. Useful

4. Very useful

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over “norms”: “Norms are agreed-upon behaviors that guide interactions within the PLC. They could include behaviors such as active listening, engagement, etc.”

I.3. Did you and the teachers you support develop norms for the PLC sessions? Mark one only

0. No

1. Yes

[ALL]

I.4. Excluding time spent in “Setting the Stage” PLC sessions, how many hours did you spend using Toolkit resources and supporting teachers’ understanding of the Toolkit resources outside of your usual working hours? Please write your response below

__________________________________

[ALL]

I.5 Please describe any improvements you suggest for the Setting the Stage materials and activities (e.g., the facilitator guide, PLC session activities).

______________________________________________________________________

Section II: Module 1 implementation and usefulness of the resources

In this section, we’d like to learn about your experience facilitating the PLC sessions, supporting teachers as they worked through the PDSA cycle and used the various Toolkit materials during Module 1, and how useful these materials were for implementing the first recommendation in their classrooms. As a reminder, the first recommendation focused on using solved problems to engage students in analyzing algebraic reasoning and strategy.

Module 1 PLC sessions and PDSA Cycle

[ALL]

II.1 In total, how many teachers did you support during Module 1 implementation? Enter the number ______________________

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over first recommendation: “Using solved problems to engage students in analyzing algebraic reasoning and strategy.”]

II.2. Overall, how easy or difficult was it to prepare and plan for Module 1’s PLC sessions using the materials provided (such as the facilitator’s guide, calendars, and agendas)? Mark one only

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

II.3. Overall, how easy or difficult was it to facilitate Module 1’s four PLC sessions for the teachers you support? Mark one only

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

II.4. Thinking back through Module 1, how would you rate teacher adherence to the PLC norms that were established during the “setting the stage” session at the beginning of the study? Mark one only

1. Not good

2. Okay

3. Good

4. Very good

5. We did not set norms

6. I do not remember the norms

[ALL]

[PROGRAMMER: Allow the following examples to display when hovered over support: “Talking through the tools with the teachers; helping teachers complete them; possibly helping teachers with data collection and summarizing findings in regards to the assessment tools/student surveys/visitation tools.”]

[PROGRAMMER: Allow the following definitions to display when hovered over the below terms:

• Plan phase: “Teachers creating an individual plan for teachers to implement the recommendation into their instruction and defining what successful implementation looks like.”

• Do phase: “Teachers implementing the recommendation according to the plan and collect data using the selected data tools to capture how implementation is going and the impact on students’ learning.”

• Study phase: “Teachers reviewing the data, both independently and with the instructional leader, and reflect on how the results compare to how they defined successful implementation.”

• Act phase: “Teachers use insights from the study phase to determine how they might revise and strengthen their implementation of the recommendation and implement again in the future.”]

II.5. For the following questions, please think through how easy or difficult it was to support teachers through each phase of the Module 1 PDSA cycle. Mark one for each item

PDSA Cycle Phase	Level of ease or difficulty supporting teachers through each phase of the Module 1 PDSA cycle
II.5a. Plan phase. This phase took place during and between the Module’s second and third PLC sessions.	1q Very difficult	2q Difficult	3q Easy	4q Very easy
II.5b. Do phase. This phase took place between the Module’s third and fourth PLC sessions.	1q Very difficult	2q Difficult	3q Easy	4q Very easy
II.5c. Study phase. This phase took place before and during the Module’s fourth PLC session.	1q Very difficult	2q Difficult	3q Easy	4q Very easy
II.5d. Act phase. This phase took place after the Module’s fourth PLC session.	1q Very difficult	2q Difficult	3q Easy	4q Very easy

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over classroom walkthrough: “A visit to a classroom to observe teacher implementation of the recommendation.”]

II.6. Did you complete a classroom walkthrough for each teacher you support during Module 1?

0. No

1. Yes

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over classroom walkthrough: “A visit to a classroom to observe teacher implementation of the recommendation.”]

II.7. In total across all the teachers you support, how many classroom walkthroughs did you complete during Module 1? Enter the number

________________

[ALL]

II.8. Please describe ways the PLC sessions or the PDSA cycle could be improved to make them more useful in supporting teachers with implementing the first recommendation in their classrooms.

__________________________________________________________________________

Module 1 Resources

[ALL]

II.9. How useful was the Module 1 facilitator guide for preparing you to guide teachers through implementing the first recommendation? Mark one only

0. I did not use the Module 1 facilitator guide

1. Not useful

2. Somewhat useful

3. Useful

4. Very useful

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over implementation liaison: “The member of the team who developed the Toolkit. This individual is available to provide you with individualized support while you support teachers in implementing the Toolkit this school year.”]

II.10. How often did you meet with your district’s implementation liaison during the implementation of Module 1? Mark one only

0. Not at all

1. Once

2. Twice

3. Three times

4. Four or more times

[ALL]

[PROGRAMMER: Allow the following definitions to display when hovered over the below terms:

• PDSA tool: “Helps teachers implement Toolkit practices in their classroom and study the effect on their instructional practice and student learning.”

• Teacher self-reflection tool: Helps teachers reflect on their use of the recommendation’s instructional strategies.”

• Visitation tool: “Gathers feedback from a colleague on teacher implementation of the recommendation’s instructional practices.”

• Student survey: “Helps teachers assess students’ engagement and mindset in math class and whether the recommendation’s instructional strategies have improved their math ability and learning.”

• Student knowledge assessment tool: “Helps teachers assess student mastery of algebra.”

• Module 1 teacher participant workbook: “Provides teachers with an overview of the Toolkit and resources they will use as they work through the learning modules and activities for the first recommendation.”]

II.11. Thinking back through Module 1, how easy or difficult was it to support teachers’ use of the following resources for implementing the first recommendation? Mark one for each item

Resource	Level of ease supporting teacher use of the resources
II.11a. PDSA tool (Appendix A)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
II.11b. Teacher self-reflection tool (Appendix B1)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
II.11.c. Visitation tool (Appendix B2)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
II.11.d. Student survey (Appendix B3)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
II.11.e. Student knowledge assessment tool (Appendix B4)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use

[ALL]

II.12. How easy or difficult was it to provide support and guidance to teachers as they collected and analyzed data to assess the effects of the practices on their students’ learning? Mark one only

0. I did not provide this type of support or guidance to teachers during Module 1.

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

II.13. Excluding time spent in Module 1 PLC sessions, how many hours did you spend using Toolkit resources and supporting teachers’ implementation of the first recommendation outside of your usual working hours? Please write your response below

__________________________________

Section III. Supporting teacher implementation of the first recommendation and its related instructional strategies

There are three instructional strategies that the Toolkit suggests teachers use to carry out the first recommendation. In this section, we’d like to learn what your experience was like supporting teachers as they implemented the first recommendation and used these instructional strategies in their classrooms.

[ALL]

III.1. Thinking about the first recommendation, using solved problems to engage students in analyzing algebraic reasoning and strategy, please answer the following questions.

III.1a. How well do you understand the first recommendation? Mark one only	Not at all Not very well Well Very well
III.1b. How prepared did you feel to help teachers integrate this first recommendation into their lesson plans?	Very unprepared Unprepared Prepared Very prepared
III.1c. How prepared did you feel to help teachers implement this first recommendation in their classroom?	Very unprepared Unprepared Prepared Very prepared

[ALL]

III.2. We recognize the teachers you support may have needed varying levels of support during Module 1. Thinking across the teachers you helped, overall how easy or difficult was it to support these teachers in using each instructional strategy for the first recommendation in their classrooms?

Instructional strategies for implementing the first recommendation	How easy or difficult was it to support teachers in using each instructional strategy in their classrooms? Mark one for each row
III.2a. Have students discuss solved problem structures and solutions to make connections among strategies and reasoning.	0=Teachers did not use strategy	1=Very difficult	2=Difficult	3=Easy	4=Very easy
III.2b. Select solved problems that reflect the lesson’s instructional aim, including problems that illustrate common errors.	0=Teachers did not use strategy	1=Very difficult	2=Difficult	3=Easy	4=Very easy
III.2c. Use whole-class discussions, small-group work, and independent practice activities to introduce, elaborate on, and practice working with solved problems.	0=Teachers did not use strategy	1=Very difficult	2=Difficult	3=Easy	4=Very easy

[ALL]

III.3. To what extent do you feel your ability to support teachers in their implementation of the first recommendation and related instructional strategies increased since starting Module 1? Mark one only

1. Not at all

2. Very little

3. Somewhat

4. A lot

[ALL]

III.4. Overall, what went well with supporting teachers as they implemented the first recommendation and related instructional strategies in their classrooms? Please describe.

________________________________________________________________________

[ALL]

III.5. Overall, what was challenging with supporting teachers as they implemented the first recommendation and related instructional strategies in their classrooms? Please describe.

_______________________________________________________________________

[ALL]

III.6. How did you attempt to overcome the challenges you encountered when supporting teachers as they implemented the first recommendation and related instructional strategies in their classrooms? Please describe.

______________________________________________________________________

[ALL]

III.7. Do you have any suggested improvements to any of the materials you used during Module 1 (e.g., the facilitator guide, the teacher resources, the meeting agendas or calendars)? Please describe.

______________________________________________________________________

Thank you for completing the Module 1 Implementation Log!

END OF IMPLEMENTATION LOG

Instructional Leader Implementation Log: Module 2

Module 2 of the Toolkit centered on the second of three recommendations for instructional practices. The second recommendation focused on teaching students to utilize the structure of algebraic representations. Now that you and the teachers you support have completed the Plan-Do-Study-Act (PDSA) cycle for Module 2, we’d like to learn about what went well and what was challenging with supporting teachers in implementing this second recommendation and using the related Toolkit materials.

• We expect it to take you about 15-20 minutes to complete this implementation log. Please complete this log by [DATE].

• As an instructional leader who is receiving access to the Toolkit and supporting teachers using the Toolkit, your insights are valuable, and we hope you will take the time to complete this log. You may skip any questions you are not comfortable answering; however, we hope that you answer as many as you can.

• The information you provide in this implementation log will help us understand how instructional leaders used the resources and supported teachers implementing the recommendations in their classrooms. Your insights will also inform future improvements to the Toolkit. Any reporting prepared for this study will summarize findings across participating districts and will not associate responses with a specific district, school or individual.

• All data collected for this study will be kept confidential, except as required by law. The researchers conducting this study follow the confidentiality and data protection requirements of the U.S. Department of Education’s Institute of Education Sciences (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183).

• If you complete an implementation log for each of the four Toolkit modules, you will receive a $60 gift card to thank you for your time and effort.

q START IMPLEMENTATION LOG

Section I: Module 2 implementation and usefulness of the resources

In this section, we’d like to learn about your experience facilitating the PLC sessions, supporting teachers as they worked through the PDSA cycle and used the various Toolkit materials during Module 2, and how useful these materials were for implementing the second recommendation in their classrooms. As a reminder, the second recommendation focused on teaching students to utilize the structure of algebraic representations.

Module 2 PLC sessions and PDSA Cycle

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over second recommendation: “Teaching students to utilize the structure of algebraic representations.”]

I.1. Overall, how easy or difficult was it to prepare and plan for Module 2’s PLC sessions using the materials provided (such as the facilitator’s guide, calendars and agendas)? Mark one only

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

I.2. Overall, how easy or difficult was it to facilitate Module 2’s four PLC sessions for the teachers you support? Mark one only

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over Setting the Stage: “The session in which the facilitator introduces teachers to the Toolkit and its resources.”]

I.3. Thinking back through Module 2, how would you rate teacher adherence to the PLC norms that were established during the “setting the stage” session at the beginning of the study? Mark one only

1. Not good

2. Okay

3. Good

4. Very good

5. We did not set norms

6. I do not remember the norms

[ALL]

[PROGRAMMER: Allow the following examples to display when hovered over support: “Talking through the tools with the teachers; helping teachers complete them; possibly helping teachers with data collection and summarizing findings in regards to the assessment tools/student surveys/visitation tools.”]

[PROGRAMMER: Allow the following definitions to display when hovered over the below terms:

• Plan phase: “Teachers creating an individual plan for teachers to implement the recommendation into their instruction and defining what successful implementation looks like.”

• Do phase: “Teachers implementing the recommendation according to the plan and collect data using the selected data tools to capture how implementation is going and the impact on students’ learning.”

• Study phase: “Teachers reviewing the data, both independently and with the instructional leader, and reflect on how the results compare to how they defined successful implementation.”

• Act phase: “Teachers use insights from the study phase to determine how they might revise and strengthen their implementation of the recommendation and implement again in the future.”]

I.4. For the following questions, please think through how easy or difficult it was to support teachers through each phase of the Module 2 PDSA cycle. Mark one for each item

PDSA Cycle Phase	Level of ease or difficulty supporting teachers through each phase of the Module 2 PDSA cycle
I.4a. Plan phase. This phase took place during and between the Module’s second and third PLC sessions.	1q Very difficult	2q Difficult	3q Easy	4q Very easy
I.4b. Do phase. This phase took place between the Module’s third and fourth PLC sessions.	1q Very difficult	2q Difficult	3q Easy	4q Very easy
I.4c. Study phase. This phase took place before and during the Module’s fourth PLC session.	1q Very difficult	2q Difficult	3q Easy	4q Very easy
I.4d. Act phase. This phase took place after the Module’s fourth PLC session.	1q Very difficult	2q Difficult	3q Easy	4q Very easy

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over classroom walkthrough: “A visit to a classroom to observe teacher implementation of the recommendation.”]

I.5. Did you complete a classroom walkthrough for each teacher you support during Module 2?

0. No

1. Yes

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over classroom walkthrough: “A visit to a classroom to observe teacher implementation of the recommendation.”]

I.6. In total across all the teachers you support, how many classroom walkthroughs did you complete during Module 2? Enter the number

______________

[ALL]

I.7. Please describe ways the PLC sessions or the PDSA cycle could be improved to make them more useful in supporting teachers with implementing the second recommendation in their classrooms.

__________________________________________________________________________

Module 2 Resources

[ALL]

I.8. How useful was the Module 2 facilitator guide for preparing you to guide teachers through implementing the second recommendation? Mark one only

0. I did not use the Module 2 facilitator guide

1. Not useful

2. Somewhat useful

3. Useful

4. Very useful

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over implementation liaison: “The member of the team who developed the Toolkit. This individual is available to provide you with individualized support while you support teachers in implementing the Toolkit this school year.”]

I.9. How often did you meet with your district’s implementation liaison during the implementation of Module 2? Mark one only

0. Not at all

1. Once

2. Twice

3. Three times

4. Four or more times

[ALL]

[PROGRAMMER: Allow the following definitions to display when hovered over the below terms:

• PDSA tool: “Helps teachers implement Toolkit practices in their classroom and study the effect on their instructional practice and student learning.”

• Teacher self-reflection tool: Helps teachers reflect on their use of the recommendation’s instructional strategies.”

• Visitation tool: “Gathers feedback from a colleague on teacher implementation of the recommendation’s instructional practices.”

• Student survey: “Helps teachers assess students’ engagement and mindset in math class and whether the recommendation’s instructional strategies have improved their math ability and learning.”

• Student knowledge assessment tool: “Helps teachers assess student mastery of algebra.”

• Module 2 teacher participant workbook: “Provides teachers with an overview of the Toolkit and resources they will use as they work through the learning modules and activities.”]

I.10. Thinking back through Module 2, how easy or difficult was it to support teachers as they used the resources to implement the second recommendation? Mark one for each item

Resource	Level of ease supporting teacher use of the resources
I.10a. PDSA tool (Appendix A)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
I.10b. Teacher self-reflection tool (Appendix C1)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
I.10c. Visitation tool (Appendix C2)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
I.10d. Student survey (Appendix C3)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
I.10e. Student knowledge assessment tool (Appendix C4)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use

[ALL]

I.11. How easy or difficult was it to provide support and guidance to teachers as they collected and analyzed data to assess the effects of the practices on their students’ learning? Mark one only

0. I did not provide this type of support or guidance to teachers during Module 2.

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

I.12. Excluding time spent in the Module 2 PLC sessions, how many hours did you spend using Toolkit resources and supporting teachers’ implementation of the second recommendation outside of your usual working hours? Please write your response below

__________________________

Section II. Supporting teacher implementation of the second recommendation and its related instructional strategies

There are three instructional strategies that the Toolkit suggests teachers use to carry out the second recommendation. In this section, we’d like to learn what your experience was like supporting teachers as they implemented the second recommendation and used these instructional strategies in their classrooms.

[ALL]

II.1. Thinking about the second recommendation, teaching students to utilize the structure of algebraic representations, please answer the following questions.

II.1a. How well do you understand second recommendation? Mark one only	Not at all Not very well Well Very well
II.1b. How prepared did you feel to help teachers integrate this second recommendation into their lesson plans?	Very unprepared Unprepared Prepared Very prepared
II.1c. How prepared did you feel to help teachers implement this second recommendation in their classroom?	Very unprepared Unprepared Prepared Very prepared

[ALL]

II.2. We recognize the teachers you support may have needed varying levels of support during Module 2. Thinking across the teachers you helped, overall how easy or difficult was it to support these teachers in using each instructional strategy for the second recommendation in their classrooms?

Instructional strategies for implementing the second recommendation	How easy or difficult was it to support teachers in using each instructional strategy in their classrooms? Mark one for each row
II.2a. Promote the use of language that reflects mathematical structure.	0=Teachers did not use strategy	1=Very difficult	2=Difficult	3=Easy	4=Very easy
II.2b. Encourage students to use reflective questioning to notice structure as they solve problems.	0=Teachers did not use strategy	1=Very difficult	2=Difficult	3=Easy	4=Very easy
II.2c. Teach students that different algebraic representations can convey different information about an algebra problem.	0=Teachers did not use strategy	1=Very difficult	2=Difficult	3=Easy	4=Very easy

[ALL]

II.3. To what extent do you feel your ability to support teachers in their implementation of the second recommendation and related instructional strategies increased since starting Module 2? Mark one only

1. Not at all

2. Very little

3. Somewhat

4. A lot

[ALL]

II.4. Overall, what went well with supporting teachers as they implemented the second recommendation and related instructional strategies in their classrooms? Please describe.

________________________________________________________________________

[ALL]

II.5. Overall, what was challenging with supporting teachers as they implemented the second recommendation and related instructional strategies in their classrooms? Please describe.

_______________________________________________________________________

[ALL]

II.6. How did you attempt to overcome the challenges you encountered when supporting teachers as they implemented the second recommendation and related instructional strategies in their classrooms? Please describe.

______________________________________________________________________

[ALL]

II.7. Do you have any suggested improvements to any of the materials you used during Module 2 (e.g., the facilitator guide, the teacher resources, the meeting agendas or calendars)? Please describe.

______________________________________________________________________

Thank you for completing the Module 2 Implementation Log!

END OF IMPLEMENTATION LOG

Instructional Leader Implementation Log: Module 3

Module 3 of the Toolkit centered on the third of three recommendations for instructional practices. The third recommendation focused on teaching students to intentionally choose from alternative algebraic strategies when solving problems. Now that you and the teachers you support have completed the Plan-Do-Study-Act (PDSA) cycle for Module 3, we’d like to learn about what went well and what was challenging with supporting teachers in implementing this third recommendation and using the related Toolkit materials.

• We expect it to take you about 15 - 20 minutes to complete this implementation log. Please complete this log by [DATE].

• As an instructional leader who is receiving access to the Toolkit and supporting teachers using the Toolkit, your insights are valuable, and we hope you will take the time to complete this log. You may skip any questions you are not comfortable answering; however, we hope that you answer as many as you can.

• The information you provide in this implementation log will help us understand how instructional leaders used the resources and supported teachers implementing the recommendations in their classrooms. Your insights will also inform future improvements to the Toolkit. Any reporting prepared for this study will summarize findings across participating districts and will not associate responses with a specific district, school or individual.

• All data collected for this study will be kept confidential, except as required by law. The researchers conducting this study follow the confidentiality and data protection requirements of the U.S. Department of Education’s Institute of Education Sciences (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183).

• If you complete an implementation log for each of the four Toolkit modules, you will receive a $60 gift card to thank you for your time and effort.

q START IMPLEMENTATION LOG

Section I: Module 3 implementation and usefulness of the resources

In this section, we’d like to learn about your experience facilitating the PLC sessions, supporting teachers as they worked through the PDSA cycle and used the various Toolkit materials during Module 3, and how useful these materials were for implementing the third recommendation in their classrooms. As a reminder, the third recommendation focused on teaching students to intentionally choose from alternative algebraic strategies when solving problems

Module 3 PLC sessions and PDSA Cycle

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over third recommendation: “Teaching students to intentionally choose from alternative algebraic strategies when solving problems.”]

I.1. Overall, how easy or difficult was it to prepare and plan for Module 3’s PLC sessions using the materials provided (such as the facilitator’s guide, calendars and agendas)? Mark one only

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

I.2. Overall, how easy or difficult was it to facilitate Module 3’s four PLC sessions for the teachers you support? Mark one only

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over Setting the Stage: “The session in which the facilitator introduces teachers to the Toolkit and its resources.”]

I.3. Thinking back through Module 3, how would you rate teacher adherence to the PLC norms that were established during the “setting the stage” session at the beginning of the study? Mark one only

1. Not good

2. Okay

3. Good

4. Very good

5. We did not set norms

6. I do not remember the norms

[ALL]

[PROGRAMMER: Allow the following examples to display when hovered over support: “Talking through the tools with the teachers; helping teachers complete them; possibly helping teachers with data collection and summarizing findings in regards to the assessment tools/student surveys/visitation tools.”]

[PROGRAMMER: Allow the following definitions to display when hovered over the below terms:

• Plan phase: “Teachers creating an individual plan for teachers to implement the recommendation into their instruction and defining what successful implementation looks like.”

• Do phase: “Teachers implementing the recommendation according to the plan and collect data using the selected data tools to capture how implementation is going and the impact on students’ learning.”

• Study phase: “Teachers reviewing the data, both independently and with the instructional leader, and reflect on how the results compare to how they defined successful implementation.”

• Act phase: “Teachers use insights from the study phase to determine how they might revise and strengthen their implementation of the recommendation and implement again in the future.”]

I.4. For the following questions, please think through how easy or difficult it was to support teachers through each phase of the Module 3 PDSA cycle. Mark one for each item

PDSA Cycle Phase	Level of ease or difficulty supporting teachers through each phase of the Module 3 PDSA cycle
I.4a. Plan phase. This phase took place during and between the Module’s second and third PLC sessions.	1q Very difficult	2q Difficult	3q Easy	4q Very easy
I.4b. Do phase. This phase took place between the Module’s third and fourth PLC sessions.	1q Very difficult	2q Difficult	3q Easy	4q Very easy
I.4c. Study phase. This phase took place before and during the Module’s fourth PLC session.	1q Very difficult	2q Difficult	3q Easy	4q Very easy
I.4d. Act phase. This phase took place after the Module’s fourth PLC session.	1q Very difficult	2q Difficult	3q Easy	4q Very easy

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over classroom walkthrough: “A visit to a classroom to observe teacher implementation of the recommendation.”]

I.5. Did you complete a classroom walkthrough for each teacher you support during Module 3?

0. No

1. Yes

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over classroom walkthrough: “A visit to a classroom to observe teacher implementation of the recommendation.”]

I.6. In total across all the teachers you support, how many classroom walkthroughs did you complete during Module 3? Enter the number

______________

[ALL]

I.7. Please describe ways the PLC sessions or the PDSA cycle could be improved to make them more useful in supporting teachers with implementing the third recommendation in their classrooms.

__________________________________________________________________________

Module 3 Resources

[ALL]

I.8. How useful was the Module 3 facilitator guide for preparing you to guide teachers through implementing the third recommendation? Mark one only

0. I did not use the Module 3 facilitator guide

1. Not useful

2. Somewhat useful

3. Useful

4. Very useful

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over implementation liaison: “The member of the team who developed the Toolkit. This individual is available to provide you with individualized support while you support teachers in implementing the Toolkit this school year.”]

I.9. How often did you meet with your district’s implementation liaison during the implementation of Module 3? Mark one only

0. Not at all

1. Once

2. Twice

3. Three times

4. Four or more times

[ALL]

[PROGRAMMER: Allow the following definitions to display when hovered over the below terms:

• PDSA tool: “Helps teachers implement Toolkit practices in their classroom and study the effect on their instructional practice and student learning.”

• Teacher self-reflection tool: Helps teachers reflect on their use of the recommendation’s instructional strategies.”

• Visitation tool: “Gathers feedback from a colleague on teacher implementation of the recommendation’s instructional practices.”

• Student survey: “Helps teachers assess students’ engagement and mindset in math class and whether the recommendation’s instructional strategies have improved their math ability and learning.”

• Student knowledge assessment tool: “Helps teachers assess student mastery of algebra.”

• Module 3 teacher participant workbook: “Provides teachers with an overview of the Toolkit and resources they will use as they work through the learning modules and activities.”]

I.10. Thinking back through Module 3, how easy or difficult was it to support teachers as they used the resources to implement the third recommendation? Mark one for each item

Resource	Level of ease supporting teacher use of the resources
I.9a. PDSA tool (Appendix A)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
I.9b. Teacher self-reflection tool (Appendix D1)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
I.9c. Visitation tool (Appendix D2)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
I.9d. Student survey (Appendix D3)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
I.9e. Student knowledge assessment tool (Appendix D4)	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use

[ALL]

I.11. How easy or difficult was it to provide support and guidance to teachers as they collected and analyzed data to assess the effects of the practices on their students’ learning? Mark one only

0. I did not provide this type of support or guidance to teachers during Module 3.

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

I.12. Excluding time spent in the Module 3 PLC sessions, how many hours did you spend using Toolkit resources and supporting teachers’ implementation of the third recommendation outside of your usual working hours? Please write your response below

___________________________________

Section II. Supporting teacher implementation of the third recommendation and its related instructional strategies

There are three instructional strategies that the Toolkit suggests teachers use to carry out the third recommendation. In this section, we’d like to learn what your experience was like supporting teachers as they implemented the third recommendation and used these instructional strategies in their classrooms.

[ALL]

II.1. Thinking about the third recommendation, teaching students to intentionally choose from alternative algebraic strategies when solving problems, please answer the following questions.

II.1a. How well do you understand third recommendation? Mark one only	Not at all Not very well Well Very well
II.1b. How prepared did you feel to help teachers integrate this third recommendation into their lesson plans?	Very unprepared Unprepared Prepared Very prepared
II.1c. How prepared did you feel to help teachers implement this third recommendation in their classroom?	Very unprepared Unprepared Prepared Very prepared

[ALL]

II.2. We recognize the teachers you support may have needed varying levels of support during Module 3. Thinking across the teachers you helped, overall how easy or difficult was it to support these teachers in using each instructional strategy for the third recommendation in their classrooms?

Instructional strategies for implementing the third recommendation	How easy or difficult was it to support teachers in using each instructional strategy in their classrooms? Mark one for each row
II.2a. Teach students to recognize and generate multiple strategies for solving problems.	0=Teachers did not use strategy	1=Very difficult	2=Difficult	3=Easy	4=Very easy
II.2b. Encourage students to articulate the reasoning behind their choice of strategy and the mathematical validity of their strategy when solving problems.	0=Teachers did not use strategy	1=Very difficult	2=Difficult	3=Easy	4=Very easy
II.2c. Have students evaluate and compare different strategies for solving problems.	0=Teachers did not use strategy	1=Very difficult	2=Difficult	3=Easy	4=Very easy

[ALL]

II.3. To what extent do you feel your ability to support teachers in their implementation of the third recommendation and related instructional strategies increased since starting Module 3? Mark one only

1. Not at all

2. Very little

3. Somewhat

4. A lot

[ALL]

II.4. Overall, what went well with supporting teachers as they implemented the third recommendation and related instructional strategies in their classrooms? Please describe.

________________________________________________________________________

[ALL]

II.5. Overall, what was challenging with supporting teachers as they implemented the third recommendation and related instructional strategies in their classrooms? Please describe.

_______________________________________________________________________

[ALL]

II.6. How did you attempt to overcome the challenges you encountered when supporting teachers as they implemented the third recommendation and related instructional strategies in their classrooms? Please describe.

______________________________________________________________________

[ALL]

II.7. Do you have any suggested improvements to any of the materials you used during Module 3 (e.g., the facilitator guide, the teacher resources, the meeting agendas or calendars)? Please describe.

______________________________________________________________________

Thank you for completing the Module 3 Implementation Log!

END OF IMPLEMENTATION LOG

Instructional Leader Implementation Log: Module 4

Module 4 of the Toolkit centered on addressing implementation roadblocks and sustaining long-term use of the three recommended practices. Now that you and the teachers you support have completed Module 4, we’d like to learn about what went well and what was challenging with supporting teachers in addressing roadblocks, thinking through sustained long-term use of the recommended practices, and using the related Toolkit materials.

• We expect it to take you about 15 - 20 minutes to complete this implementation log. Please complete this log [DATE].

• As an instructional leader who is receiving access to the Toolkit and supporting teachers using the Toolkit, your insights are valuable, and we hope you will take the time to complete this log. You may skip any questions you are not comfortable answering; however, we hope that you answer as many as you can.

• The information you provide in this implementation log will help us understand how instructional leaders used the resources and supported teachers implementing the recommendations in their classrooms. Your insights will also inform future improvements to the Toolkit. Any reporting prepared for this study will summarize findings across participating districts and will not associate responses with a specific district, school or individual.

• All data collected for this study will be kept confidential, except as required by law. The researchers conducting this study follow the confidentiality and data protection requirements of the U.S. Department of Education’s Institute of Education Sciences (The Education Sciences Reform Act of 2002, Title I, Part E, Section 183).

• If you complete an implementation log for each of the four Toolkit modules, you will receive a $60 gift card to thank you for your time and effort.

q START IMPLEMENTATION LOG

Section I: Module 4 implementation and usefulness of the resources

In this section, we’d like to learn about your experience facilitating the PLC sessions, supporting teachers as they used the various Toolkit materials during Module 4, and how useful these materials were for addressing implementation roadblocks and sustaining long-term use of the three recommended practices.

Module 4 PLC sessions and activities

[ALL]

I.1. Overall, how easy or difficult was it to prepare and plan for the Module 4 PLC sessions using the materials provided (such as the facilitator’s guide, calendars, and agendas)? Mark one only

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

I.2. Overall, how easy or difficult was it to facilitate the two PLC sessions for Module 4 for the teachers you support? Mark one only

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over Setting the Stage: “The session in which the facilitator introduces teachers to the Toolkit and its resources.”]

I.3. Thinking back through Module 4, how would you rate teacher adherence to the PLC norms that were established during the “Setting the Stage” session at the beginning of the study? Mark one only

1. Not good

2. Okay

3. Good

4. Very good

5. We did not set norms

6. I do not remember the norms

[ALL]

I.4. Overall, how easy or difficult was it to support teachers through the self-reflection exercises that are part of the Module 4 activities? Mark one only

1. Very difficult

2. Difficult

3. Easy

4. Very easy

[ALL]

I.5. Overall, how easy or difficult was it to support teachers through the goal setting exercises that are part of the Module 4 activities? Mark one only

1. Very difficult

2. Difficult

3. Easy

4. Very easy

Module 4 Resources

[ALL]

I.6. How useful was the Module 4 facilitator guide for preparing you to guide teachers in thinking through addressing implementation roadblocks? Mark one only

0. I did not use the Module 4 facilitator guide

1. Not useful

2. Somewhat useful

3. Useful

4. Very useful

[IF I.6=0, GO TO I.8]

[ALL]

I.7. How useful was the Module 4 facilitator guide for preparing you to guide teachers in thinking through sustaining long-term use of the three recommendations in their classrooms? Mark one only

0. I did not use the Module 4 facilitator guide

1. Not useful

2. Somewhat useful

3. Useful

4. Very useful

[ALL]

[PROGRAMMER: Allow the following definition to display when hovered over implementation liaison: “The member of the team who developed the Toolkit. This individual is available to provide you with individualized support while you support teachers in implementing the Toolkit this school year.”]

I.8. How often did you meet with your district’s implementation liaison during the implementation of Module 4? Mark one only

0. Not at all

1. Once

2. Twice

3. Three times

4. Four or more times

[ALL]

[PROGRAMMER: Allow the following examples to display when hovered over support: “Talking through the tools with the teachers; helping teachers complete them; possibly helping teachers with data collection and summarizing findings in regards to the assessment tools/student surveys/visitation tools.”]

[PROGRAMMER: Allow the following definitions to display when hovered over the below terms:

• Cumulative self-reflection resource: “Helps teachers reflect on implementing the Toolkit practices in their classroom.”

• Algebra teaching practices card sort activity: “Helps teachers collect their students’ perspectives on the implementation of the teaching practices in the classroom.”

• Action plan resource: “Helps teachers think through upcoming goals, the resources needed to accomplish these goals, and how they will measure success.”

• Module 4 participant workbook: “Provides teachers with an overview of the Toolkit and resources they will use as they work through the learning modules and activities to address implementation roadblocks and sustain long-term use of the recommended practices.”]

I.9. Thinking back through Module 4, how easy or difficult was it to support teachers’ use of the following resources for addressing implementation roadblocks and sustaining long-term use of the three recommendations in their classroom? Mark one for each item

Resource	Level of ease supporting teacher use of the resources
I.9a. Cumulative self-reflection resource	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
I.9b. Algebra teaching practices card sort activity	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use
I.9c. Action plan resource	1=Very difficult	2=Difficult	3=Easy	4=Very easy	0=Teachers did not use

[ALL]

[PROGRAMMER: Allow the following definitions to display when hovered over the below terms:

• PDSA tool: “Helps teachers implement Toolkit practices in their classroom and study the effect on their instructional practice and student learning.”

• Teacher self-reflection tool: Helps teachers reflect on their use of the recommendation’s instructional strategies.”

• Visitation tool: “Gathers feedback from a colleague on teacher implementation of the recommendation’s instructional practices.”

• Student survey: “Helps teachers assess students’ engagement and mindset in math class and whether the recommendation’s instructional strategies have improved their math ability and learning.”

• Student knowledge assessment tool: “Helps teachers assess student mastery of algebra.”

• Module 1 teacher participant workbook: “Provides teachers with an overview of the Toolkit and resources they will use as they work through the learning modules and activities for the first recommendation.”

• Module 2 teacher participant workbook: “Provides teachers with an overview of the Toolkit and resources they will use as they work through the learning modules and activities for the second recommendation.”

• Module 3 teacher participant workbook: “Provides teachers with an overview of the Toolkit and resources they will use as they work through the learning modules and activities for the third recommendation.”

• Module 4 teacher participant workbook: “Provides teachers with an overview of the Toolkit and resources they will use as they work through the learning modules and activities to address implementation roadblocks and sustain long-term use of the recommended practices.”

• Cumulative self-reflection resource: “Helps teachers reflect on implementing the Toolkit practices in their classroom.”

• Algebra teaching practices card sort activity: “Helps teachers collect their students’ perspectives on the implementation of the teaching practices in the classroom.”

• Action plan resource: “Helps teachers think through upcoming goals, the resources needed to accomplish these goals, and how they will measure success.”]

I.10. Looking ahead to the rest of the school year and beyond, after the Toolkit professional development activities have concluded, do you plan to encourage teachers to use any of the following Toolkit resources in their classrooms? Mark one for each item

Resources	Yes	No	Unsure
I.10a. PDSA tool	1q	0q	2q
I.10b. Teacher self-reflection tool	1q	0q	2q
I.10c. Visitation tool	1q	0q	2q
I.10d. Student survey	1q	0q	2q
I.10e. Student knowledge assessment tool	1q	0q	2q
I.10f. Module 1 participant workbook	1q	0q	2q
I.10g. Module 2 participant workbook	1q	0q	2q
I.10h. Module 3 participant workbook	1q	0q	2q
I.10i. Module 4 participant workbook	1q	0q	2q
I.10j. Cumulative self-reflection resource	1q	0q	2q
I.10k. Algebra teaching practices card sort activity	1q	0q	2q
I.10l. Action plan resource	1q	0q	2q

[ALL]

I.11. Excluding time spent in the Module 4 PLC sessions, how many hours did you spend using Toolkit resources and supporting teachers’ as they thought through implementation roadblocks and sustaining long-term use of the three recommendations outside of your usual working hours? Please write your response below

___________________________

Section II. Supporting ongoing teacher implementation of the three recommendations

As you recall, the Toolkit is comprised of three recommendations for teachers to use in their Algebra I classrooms. Now that you and the teachers you support have learned about all three recommendations, we’d like to learn to what extent you plan to encourage and support teachers to use these recommendations in the classroom once the Toolkit professional development activities conclude.

[ALL]

II.1. Looking ahead to the rest of the school year and beyond, how often do you anticipate you will encourage and support teacher use of each of the recommendations in their classrooms? Mark one for each item

Recommendation	Planned frequency of support going forward
II.1a. Recommendation 1: Using solved problems to engage students in analyzing algebraic reasoning and strategy.	0=Never	1=Less than monthly	2=Monthly	3=A few times a month	4=Once a week	5=More than once a week
II.1b. Recommendation 2: Teaching students to utilize the structure of algebraic representations.	0=Never	1=Less than monthly	2=Monthly	3=A few times a month	4=Once a week	5=More than once a week
II.1c. Recommendation 3: Teaching students to intentionally choose from alternative algebraic strategies when solving problems.	0=Never	1=Less than monthly	2=Monthly	3=A few times a month	4=Once a week	5=More than once a week

[ALL]

II.2. To what extent do you feel your ability to support teachers in addressing implementation roadblocks and thinking through sustained long-term use of the three recommendations in their classrooms increased since starting Module 4? Mark one only

1. Not at all

2. Very little

3. Somewhat

4. A lot

[ALL]

II.3. Overall, what went well with supporting teachers as they addressed implementation roadblocks and thought through sustained long-term use of the three recommendations in their classrooms? Please describe.

________________________________________________________________________

[ALL]

II.4. Overall, what was challenging with supporting teachers as they addressed implementation roadblocks and thought through sustained long-term use of the three recommendations in their classrooms? Please describe.

_______________________________________________________________________

[ALL]

II.5. How did you attempt to overcome the challenges you encountered when supporting teachers as they addressed implementation roadblocks and thought through sustained long-term use of the three recommendations in their classrooms? Please describe.

______________________________________________________________________

[ALL]

II.6. Please describe improvements to the Module 4 PLC sessions or resources (e.g., the facilitator guide, the teacher resources, the meeting agendas or calendars) that would help you better support teachers as they address implementation roadblocks and sustain long-term use of the three recommendations in their classroom. Please describe.

______________________________________________________________________

Thank you for completing the Module 4 Implementation Log!

END OF IMPLEMENTATION LOG