Teacher Interview Questions

Appendix F: PRISMS Toolkit Evaluation Teacher Interview Questions

Thank you so much for taking the time to talk with me today about your experience with the toolkit. The research team will protect the confidentiality of all information collected for the study and will use it for research purposes only. Only the evaluation team members with training in how to deal with sensitive and confidential data will be allowed access. None of your responses will be individually attributed to you or your school or district and will be used for statistical purposes only. You may opt out from responding to a question or the entire measure at any time with no consequences.

Virtual meeting/Conference recording notice

The American Institutes for Research^® (AIR^®) allows for the recording of audio, visuals, participants, and other information sent, verbalized, or utilized during business-related meetings. By joining a meeting, you automatically consent to such recordings. Any participant who prefers to participate via audio only should disable their video camera so that only their audio will be captured. Video and/or audio recordings of any AIR session shall not be transmitted to an external third party without the permission of AIR.

First, I’d like to get to know a little bit about your background.

1. Please tell me your name and a brief description of your role at the school.

   1. How long have you been in this role at your school?

   2. What was your job before moving into your current role?

The PRISMS Toolkit was designed to support implementation of the recommendations from the What Works Clearinghouse (WWC) practice guide, Providing Reading Interventions for Students in Grades 4–9. Prior to this interview, we shared a one-page overview of the toolkit components with you, so you can reference it during this interview. [Note: We will provide the toolkit resource roundup infographic to participants. This infographic will be part of component 4 of the toolkit. To answer this interview question, participants will state which components of the toolkit that they used, as depicted on the infographic. We will ask participants to name the component aloud to capture their responses.]

2. Over the course of the year, which toolkit components did you use or engage with?

1. Teacher’s fidelity of implementation tool?

2. Two online professional development modules?

3. Visual scaffolds to use in your classroom with students?

4. Instructional routines aligned to the WWC recommendations?

5. Professional learning communities (PLCs)?

6. Toolkit resource roundup infographic?

7. Toolkit manuals?

8. Other toolkit resources? [If participant indicates that they used “other toolkit resources,” ask to please elaborate or describe.]

Now, we are going to discuss the components of the toolkit that you used or engaged with so that I can better understand your perceptions of their utility and quality as well as any barriers you had in using them.

3. How would you describe your experience using the fidelity of implementation tool?

1. How did you use this tool? To what extent were you able to assess how your current practices aligned with the WWC practice guide recommendations?

2. How often and in what context did you use the tool (for example, on your own time, during the PLC only)?

3. Why did you use the tool (for example, PLC requirement, required by school/district leadership, supported self-reflection and/or professional growth)?

4. To what extent did you find the fidelity of implementation tool to be clear and easy to use?

5. How long did it take you to complete the tool?

6. Was the amount of time it took to use the tool appropriate?

7. Was there anything that you would have changed to improve the tool?

4. How would you describe your experience using the professional learning online modules?

1. Which sections of the online modules did you complete? [Probe for module 1, chapters 1–2; module 2, chapters 1–4.]

2. When did you complete these modules?

3. In what context did you complete the modules (for example, on your own time, during predesignated time provided by your school leader or district)?

4. What was most informative about the online modules?

5. Approximately how long did it take you to complete the chapters from module 1?

   1. Was this amount of time appropriate?

6. Approximately how long did it take you to complete the chapters from module 2?

   1. Was this amount of time appropriate?

7. To what extent did you find the online modules and associated activities to be clear and easy to understand?

8. Was the content appropriate and applicable to your context?

9. To what extent did the online modules and associated activities inform your instruction? In what ways?

10. To what extent did you find them accessible and easy to use?

    1. Did you find yourself returning to the modules to revisit any of the information or resources?

       1. If so, which ones? How often?

11. Was there anything that you would have changed to improve the professional learning modules?

5. What did you think about the resources included in the toolkit, such as the visual scaffolds and instructional routines?

1. Did you use the visual scaffolds with students? If so, how and how often?

1. Can you provide a specific example of how you used the visual scaffolds?

2. If you did not use the visual scaffolds, can you share why you did not?

2. Did you use the instructional routines with students? If so, which ones and how often?

1. Can you provide a specific example of how you used the instructional routines?

2. If you did not use the instructional routines, can you share why you did not?

3. To what extent did you find the resources to be clear and easy to understand?

4. To what extent did you find them accessible and easy to use?

5. Was there anything that you would have changed to improve the resources included in the toolkit?

6. How would you describe your experience participating in the PLCs?

1. How many PLC sessions were you able to attend? (Note: There were nine PLC sessions total.)

2. Were PLC sessions held in person, virtually, or a combination of meeting types?

   1. Were they held during the school day, before school, or after school?

3. Was the duration of the PLCs appropriate?

4. What was most informative about the PLCs?

5. Was the content of the PLCs appropriate?

6. To what extent did the PLCs and associated activities inform your instruction? In what ways?

7. To what extent did you find the PLCs and associated activities to be clear and easy to understand?

8. To what extent did you find the PLCs and associated activities accessible and easy to use?

9. Was there anything that you would have changed to improve the PLCs?

7. How would you describe the overall quality of the toolkit?

1. To what extent did you find the toolkit materials to be clear and easy to understand?

2. To what extent did you find them accessible and easy to use?

3. Was there anything that you would have changed to improve the toolkit?

Next, I would like to discuss supports or barriers that you encountered as you implemented the toolkit and the instructional recommendations described in the toolkit.

8. What supported your use of the fidelity of implementation tool, online modules, and PLCs? [If needed, use probes listed in 8a–8h.]

1. School leadership support?

2. District-level support?

3. Alignment with other school or district professional learning priorities?

4. Dedicated time to engage with the tools?

5. Trusting relationships with other teachers and coaches?

6. Use of student data?

7. Consistent scheduling of PLCs?

8. Systems and structures to guide PLCs (for example, use of an agenda, PLC roles, colleagues following through on next steps, strong facilitation)?

9. Did you or your colleagues face any challenges as you engaged with the fidelity of implementation tool, online modules, and PLCs? [If needed, use probes listed in 9a–9k.]

1. Time constraints?

2. Schedule challenges?

3. Role of team members who participated in implementation?

4. Communication issues?

5. Lack of training?

6. Lack of buy-in from your colleagues, leaders, and/or other staff in your building?

7. Too many students in need of intervention?

8. Toolkit did not adequately address the needs of certain student groups (for example, multilingual learners, students with significant disabilities)?

9. Information conflicted with previous understandings of or beliefs about literacy intervention?

10. Lack of alignment between WWC recommendations and the school’s curriculum?

11. Lack of systems and structures to guide PLCs?

10. [Skip if respondent does not mention challenges.] How, if at all, did you address or overcome these challenges?

    1. What supports or resources would you need to overcome these challenges?

We are approaching the end of our time together today. To wrap up:

11. Is there anything else you think we should know about the toolkit that we have not covered?

Thank you for your time today.

Regional Educational Laboratory Southwest Toolkit Evaluation 5