Recruitment Materials

Appendix A: Recruitment Materials

• Appendix A1 – School Division Recruitment Flyer and Followup Phone Call Talking Points

• Appendix A2 – School Leader Recruitment Email, Followup Phone Call Talking Points, and Agenda for School Staff Informational Webinar

• Appendix A3 – Teacher Recruitment Email and Consent Form

Appendix A1 – School Division Recruitment Flyer and Follow-up Phone Call Talking Points

Information for School Divisions

Who are we?

Regional Educational Laboratory Appalachia (REL AP) partners with state and local education agencies, school boards, institutes of higher education, and student, family, and community organizations in Kentucky, Tennessee, Virginia, and West Virginia. REL AP partners with these organizations on applied research and development; training, coaching and technical supports; and dissemination to develop evidence that can inform consequential decisions about policy, programs, and practice with the goal of improving educational outcomes for students in the region. REL AP is operated by the Education Division of SRI International, an independent non-profit research institute. REL AP is testing a toolkit to support professional learning around teaching math to preschool students. What is the Early Math Toolkit?

The Early Math Toolkit is a four-month professional development program designed to increase preschool teachers’ ability to plan for and apply evidence-based, core math teaching practices to increase the quantity and quality of math instruction in their classroom. Mathematical knowledge and skills acquired in early childhood provide a critical foundation for long-term student success in math as well as reading. The toolkit was developed using the recommendations in the What Works Clearinghouse (WWC) Teaching Math to Young Children practice guide. The WWC, which is run by the U.S. Department of Education, develops practice guides that feature specific recommendations for educators to implement evidence-based practices, and these recommendations are supported by a comprehensive review of findings from rigorous research studies.

The toolkit includes three components: (1) self-diagnostics and monitoring tools for teachers and instructional leaders to evaluate their instructional practices, (2) professional development resources built on the foundation of a developmental understanding of early math, and (3) a concise guide to help school and district leaders build institutional supports to scale and sustain changes in teaching practice. As part of the toolkit’s professional development experience, teachers will complete 5 instructional modules over the course of the school year and reflect on their teaching practices. Each module is designed to increase teachers’ knowledge about how to plan for and apply evidence-based math practices to increase the quantity and quality of math instruction.

We invite your teachers to participate in an efficacy study of the toolkit. We are hoping to recruit between 2 and 20 preschool teachers from your division to participate in the study. The study hopes to learn how the toolkit supports preschool teachers in implementing evidence-based teaching practices and activities that promote math learning in young children. Schools randomly assigned to implement the toolkit will receive professional development and resources for improving preschool math learning. Schools not randomly assigned to implement the toolkit (the “business-as-usual” group) can continue any existing professional development on preschool math instruction and will be able to access the toolkit resources after the completion of the study. The findings will be used to refine and disseminate the toolkit to other schools and teachers around the country.

What will school divisions be asked to do?

• Allow REL AP researchers to contact schools to recruit preschool teachers. Half the schools will be randomly assigned to participate in the Early Math Toolkit professional development, and the other half will engage in business-as-usual professional development but will be given access to the toolkit materials after completion of the study.

• Provide REL AP researchers access to existing administrative demographic data on students in the participating teachers’ classrooms, including special education status, receipt of services for economic disadvantage, and English learner status.

• Provide REL AP researchers access to Virginia Kindergarten Readiness Program Early Mathematics Assessment System and Phonological Awareness Literacy Screening scores for students of participating teachers.

Divisions will receive a $400 stipend for assistance with data exports from administrative data systems.

What will teachers and instructional leaders be asked to do?

• Teachers will complete brief surveys at the beginning and end of the school year.

• Teachers will complete an instructional log once a day for three weeks (one week at the beginning, middle, and end of the school year).

• Teachers will be observed by the research team twice during the school year.

• Teachers assigned to implement the Early Math Toolkit will also:

  • Complete five professional development modules over the course of the school year.

  • Complete a self-assessment of their math teaching at the beginning, middle, and end of the school year and a satisfaction survey at the end of the school year.

  • Receive up to $800 in gift cards for completing the professional development modules (if completed outside of work hours, otherwise the study team will provide funds for substitute teachers), surveys, and instructional log as well as providing information about implementation.

• Teachers assigned to the business-as-usual group will also receive up to $150 in gift cards for completing the surveys and instructional log.

• Instructional leaders will be asked to review and implement resources that support adoption of the toolkit by teachers and the ability to scale and maintain use of the toolkit when the study ends.

Confidentiality of participants

• SRI is committed to maintaining the confidentiality of participant information and the protection of human subjects.

• All student and teacher data will be used only for research purposes.

• SRI will work with schools and divisions to ensure we comply with school and division policies related to external research.

If you are interested in inviting your preschools to participate in the study, or if you would like to learn more about the Early Math Toolkit and the study, please contact us:

Dr. Erika Gaylor

SRI International

(650)580-5933

[email protected]

Sarah Nixon Gerard

SRI International

(703)247-8545

[email protected]

Participation in this research is voluntary; school staff will not be penalized in any way if they decide not to participate. For information about participant rights, please contact the SRI Institutional Review Board at 650-859-4022 or by email: [email protected]

Talking Points for Division Follow-up Call

• Introduction

  • Detailed description of toolkit

  • Clarify SRI’s role in toolkit evaluation

• Review intervention and study

  • Length of study

  • Breakdown of benefits for school, teachers, and students

• Questions?

• Request to participate

  • Overview of data collection

  • Incentives

• Next steps

  • Ask for help in reaching out to principals

  • Division research application if necessary

Memo of understanding if necessary (e.g., agreement to share division data) Data management plan

Appendix A2 – School Leader Recruitment Email, Talking Points for Follow-up Phone Call, and Agenda for School Staff Informational Webinar

School Leader Recruitment Email

Subject: Invitation to participate in REL AP toolkit evaluation study

Dear [School Leader],

The U.S. Department of Education’s Regional Educational Laboratories (RELs) partner with educators and policymakers nationwide to improve educational outcomes. The Regional Educational Laboratory Appalachia (REL AP), which is led by SRI International, has developed a toolkit to support teaching math to young children in the region. The toolkit is rooted in evidence-based practices in preschool mathematics instruction identified by the Department's What Works Clearinghouse (WWC), and highlighted in its Teaching Math to Young Children Practice Guide. The toolkit includes diagnostics and monitoring tools for teachers and instructional leaders, professional development resources built on the foundation of a developmental understanding of early math, and institutional supports for systematic and maintainable implementation of the practice guide recommendations. Through implementing the toolkit, teachers will complete 5 instructional modules over the course of the school year and reflect on their teaching practices. By completing these modules, teachers will increase their knowledge about how to plan for and apply evidence-based math practices to increase the quantity and quality of math instruction. As a next step in its work, REL AP is going to partner with a select group of schools in Virginia to study the effectiveness of this new toolkit compared to any existing professional development in preschool mathematics instruction. Specifically, the study will examine whether using the toolkit improves preschool students’ math achievement, whether the toolkit improves teachers’ ability to plan for and apply evidence-based math practices to increase the quantity and quality of math instruction in their classroom, and how the toolkit is implemented in schools. The study hopes to learn how the toolkit supports preschool teachers in implementing evidence-based teaching practices and activities that promote math learning in young children. The findings will be used to refine and disseminate the toolkit to other schools and teachers around the country. Would you consider being part of this important work?

We are inviting preschools across several districts in Virginia to participate in the study. Participating schools will be randomly assigned to either receive the toolkit supports or to engage in business-as-usual professional development. Both groups of schools will be asked to participate in study data collection, including teacher instructional logs, teacher surveys, and teacher classroom observations. As part of the project, members of our team will conduct two observations in [Name] preschool classrooms. Teachers participating in data collection will receive gift cards.

We know that the time and energy of [Name] early educators are precious. SRI staff have extensive experience collaborating with early learning programs and making sure that studies provide educators with substantial benefits, such as access to evidence-based resources and training for improving student learning outcomes, and minimal burden. Below are some questions we often hear from our partners.

• Will this be disruptive to teaching? Study staff will enter your classrooms only at times that are convenient for teachers and school staff, and observations will take place during normal classroom activities. Observers work regularly with early learning classrooms and are trained to be as unobtrusive as possible, that is, to be a “fly on the wall.” Observers will not interact with students.

• Is it safe for study staff to come to the program? Study staff will complete any required background checks and will comply with all health and safety requirements at the district or program level.

• Is student privacy at risk? Throughout the study, we will provide programs with detailed information about our data security procedures. Study staff receive extensive training on protecting student and teacher data and complying with SRI’s extensive security procedures.

• What will school leaders need to do? Leaders of schools that are assigned to receive the toolkit supports will be provided resources to help them support adoption of the toolkit by teachers and enhance their ability to scale and maintain use of the toolkit when the study ends.

If you are interested, could we set up a time to discuss the toolkit further and answer any other questions you may have? Please email us back with your preferred date, time, and whether you would like to talk over the phone or Zoom.

Thank you so much for your time and consideration! We look forward to hearing from you.

Best,

Erika Gaylor

Study Principal Investigator

Talking Points for School Leader Follow-up Call

• Introduction

  • Detailed description of toolkit

  • Clarify SRI’s role in toolkit evaluation

• Review intervention and study

  • Length of study

  • Breakdown of benefits for school, teachers, and students

• Questions?

• Request to participate

  • Overview of data collection

  • Incentives

• Next steps

  • Follow-up calls if needed

  • Ask for help in reaching out to teachers

  • District research application if necessary

  • Other

School Staff Webinar Agenda

• Overview of toolkit

• Overview of study

• Benefits of and responsibilities of participation

• Timeline

• Q&A

• Next steps

Appendix A3 – Teacher Recruitment Email and Consent Form

Teacher Recruitment Email

Subject: Invitation to participate in REL AP toolkit evaluation study

Dear [Teacher],

We recently held a webinar for [Name of School] describing a new professional development toolkit that supports teaching math to young children that has been developed by the U.S. Department of Education’s Regional Educational Laboratory Appalachia (REL AP). This toolkit was developed to support teachers and school leaders in implementing evidence-based practices to improve early math learning through professional learning and instructional resources. The toolkit features five PD modules designed to increase teachers’ knowledge about how to plan for and apply evidence-based math teaching practices to increase the quantity and quality of math instruction for young children. The PD modules feature a set of self-guided content and demonstration videos, a self-assessment checklist for teachers to assess their math instruction, and guides for participating in Learning Community meetings with other teachers in their school or division.

The REL research team is working with your school to study the effectiveness of this toolkit. Specifically, the study will examine whether using the toolkit improves preschool students’ math achievement, whether the toolkit improves teachers’ ability to plan for and apply evidence-based math practices to increase the quantity and quality of math instruction in their classroom, and how the toolkit is implemented in schools. The study hopes to learn how the toolkit supports preschool teachers in implementing evidence-based teaching practices and activities that promote math learning in young children. The findings will be used to refine and disseminate the toolkit to other schools and teachers around the country. Would you consider being part of this important work?

Participating schools will be randomly assigned to either receive the toolkit supports or to engage in business-as-usual professional development. Both groups of schools will be asked to participate in study data collection, including teacher instructional logs, teacher surveys, and teacher classroom observations. The consent form attached to this email provides more information about these data-collection activities. As part of the project, a member of our team would conduct two observations in [Name] preschool classrooms. The observer will sit quietly at the side or back of the classroom and will not interact with the students. If you choose to participate, you will receive $50 in gift cards for each wave of data collection completed. [Insert if the district allows payments to teachers:] If your school is in the treatment group you will also receive up to $800 in gift cards for completing the toolkit’s professional development modules (if completed outside of work hours, otherwise the study team will provide funds for substitute teachers).  

For more information, click [Link to consent form] to see the consent form. If you would like to participate, please sign the consent form. If you have any additional questions, please don’t hesitate to reach out.

Best,

Erika Gaylor

Study Principal Investigator

(650)580-5933

[email protected]

Sarah Nixon Gerard

Study Co-Principal Investigator

(703)247-8545

[email protected]

Teacher Consent Form

CONSENT TO PARTICIPATE IN RESEARCH 

Principal Investigator: Dr. Erika Gaylor 

Study Title: Teaching Math to Young Children Toolkit Efficacy Evaluation 

REL Appalachia is conducting a research study to learn how the Early Math Toolkit (Toolkit) can help teachers support their preschool students to learn math and the ways in which the Toolkit can be improved. The Toolkit is designed to support teaching practices essential to promoting early math skills and knowledge in preschool children. The results of this study will help inform future use of the toolkit to support preschool students in learning foundational math skills. 

Procedures: Participating schools will be randomly assigned to either a business-as-usual group or to the Toolkit group. You have a 50 percent chance of being in the Toolkit group. If you choose to be involved, you will be asked to participate in the research activities listed below based on which group you are in. 

• All teachers:

  • Demographics and Experiences Survey: A 30-minute online survey about your background, education, training, and PD experiences, completed once at the beginning of the study

  • Attitudes, Beliefs, and Confidence Preschool Math Survey: A 20-minute online survey about your attitudes and confidence relating to mathematics, completed at the beginning and end of the study

  • Instructional Log: A 5-minute daily online survey about math classroom activities completed for three weeks during the study (25 minutes per week, for a total of 1 hour and 15 minutes)  

  • Observation: Permit researchers to observe your math teaching once at the beginning of the school year and once near the end of the year

  • Classroom Assessment Scoring System (CLASS) observation scores: We will ask for permission to access your CLASS observation scores, collected by the Virginia Department of Education

• Teachers in the Toolkit Group:

  • Professional Development: Complete five 4-hour professional development modules, for a total of 20 hours over the course of the school year.  

  • Self-Assessment: Complete a self-assessment of your math teaching at the beginning, middle, and end of the school year.

  • Implementation Checklist: A 10-minute online checklist tracking completion of the module resources and recommended classroom activities.

  • Toolkit Satisfaction Survey: A 30-minute survey about your perceptions of the Toolkit’s resources and usability, your satisfaction with the support and professional development, and the effectiveness of the professional development format 

Possible Risks: The risks to participating in this research are minimal. We do not expect you to be hurt or upset by being in this study. You may skip or decline to answer survey questions you are asked. One risk is that information about you could accidentally be shared with people without permission to see it. However, SRI is experienced at securing data, and we will handle all information securely to reduce the risk of anyone’s seeing it. 

Possible Benefits: Based on the opportunity to increase math teaching abilities as well as confidence and comfort in teaching math to young children, teachers in the Toolkit group may benefit immediately. We do not expect that teachers in the business-as-usual group will directly benefit from participating in this study. However, this is an opportunity to contribute to an important study of a new, evidence-based professional development toolkit that that will be made available to preschool and kindergarten teachers nationwide.

Confidentiality: We will take precautions to safeguard each participant’s information. All data will be kept in secured files; your name will not be used by or shared with anyone other than the SRI researchers working on the study. Any data we collect will be reported as a summary without using your name, your school’s name or your division’s name.  

Payments: To thank you for your time, you will receive $50 for each wave of completed data collection. [Insert if the district allows payments to teachers:] If your school is in the treatment group you will also receive $40 per hour (up to $800) in gift cards for completing the toolkit’s professional development modules (if completed outside of work hours, otherwise the study team will provide funds for substitute teachers).     

Participation in Research Is Voluntary: You do not have to participate in this study if you do not want to, and if you decide to be in the study and change your mind, we will not use your information. You can stop being in the study at any time by contacting the Principal Investigator listed below. 

Questions: If you have any questions about the study, you may contact the Principal Investigator Erika Gaylor at [email protected] or 650-580-5933. 

Independent Contact: If you have any concerns, complaints, or questions about the research or your rights as a participant, please contact the Institutional Review Board at SRI International by calling 650-859-4022 or e-mailing [email protected].  

[online consent in Qualtrics] 

Consent to Participate: Signing your name means you agree to participate in the activities described above. If you do give your consent, click on the “Yes” button. If you do not agree to participate as described above, click on the “No” button below.

□ No, I do not give my consent to participate 

□ Yes, I give my consent to participate 

Name:  

[If no:] 

Thank you for considering this request. Please click “Submit” to complete your response. 

SUBMIT 

[If yes:] 

Name: 

Please print this page for your records and then click "Next" to submit your response. 

NEXT 

Thank you for considering this request. Please click “Submit” to complete your response. 

Regional Educational Laboratory Appalachia Early Math Toolkit Evaluation

13