Impact Evaluation of Training in Multi-Tiered Systems of Support for Reading in Early Elementary School

Impact Evaluation of Training in Multi-Tiered Systems of Support for Reading in Early Elementary School

Appendix B MTSS-R_District Cost Interview with Consent Script_06_25_clean

Impact Evaluation of Training in Multi-Tiered Systems of Support for Reading in Early Elementary School

OMB: 1850-0953

Document [docx]
Download: docx | pdf

Fall 2021 and Spring 2022, 2023, and 2024 District Cost Interviews



Opening Script and Consent

Interviewer Name: _____________________

District ID: _____________________

Date of Interview: _____________________

Staff ID: _____________________

Opening Script

Thank you for taking the time to speak with us today. My name is [name], and I’m joined by my colleague [name]. We are conducting interviews with each participating district in order to learn about the work your district did to prepare for implementing ‘Approach A’ and ‘Approach B,’ and what resources and support your district is providing to support each approach. In each interview, we’ll focus on activities led by, and conducted by, the central district office staff. We will use this information to help document the costs and challenges, which we will summarize in the study report.

At times, we’ll ask about work that applies to both Approach A and B (e.g., if you updated our screening or progress monitoring tools). At other times, we’ll ask you to discuss work first for Approach A, and then for Approach B (e.g., supporting implementation). We anticipate the interview will take approximately 40 minutes, but have scheduled an hour to be safe.

If you don’t know the answer to a question, please just say so, and we’ll figure out who is in a position to answer the question. It’s important that you answer only the questions to which you know the answer.

Before we ask you to participate, we want you to know that participation in this interview is voluntary. All your responses will be kept private and will only be used for this study and related research. Information collected for this study falls under the confidentiality and data protection requirements of the Institute of Education Sciences. All information from this study will be kept confidential as required by the Education Sciences Reform Act of 2002 (Title I, Part E, Section 183). Responses to this interview will be used only for statistical purposes. Personally identifiable information about individual respondents will not be reported. The study team will not provide information that identifies you, schools in your district, or your district to anyone outside the study team, except as required by law.



Do you agree to participate in this interview?

  • Interviewee agrees to participate in the interview.

  • Interviewee does not agree to participate in the interview.



If interviewee agrees to participate, say: I’d like to record the rest of this call, as a backup for my notes. You can ask me to stop recording at any time. Do I have your permission to record the interview?

  • Yes

  • No













































According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number. The valid OMB control number for this information collection is #1850-NEW. The time to complete this interview is estimated to average 60 minutes, including time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. The obligation to respond to this collection is voluntary. If you have comments or concerns regarding the status of your individual submission of this form, application or survey, please write directly to Lauren Angelo, U.S. Department of Education, Institute of Education Sciences, 550 12th Street, SW, Room 4104 Washington, DC 20004 or email [email protected].

Note: All interviews will involve one interviewer and one notetaker. The notetaker will take notes within the Word document, and notes will be transferred to a database for analysis. This document provides the questions for the interview, but not specified sections (e.g., boxes) to take notes in.

Fall 2021 Interview Protocol

We’ll start by asking about activities conducted during the spring and summer of 2021, after your district knew it would be participating in the study. In this section, we’ll discuss the work your district engaged in to identify the Multi-Tiered System of Supports for Reading (MTSS-R) Coach, ensure that your district has screening and progress monitoring data systems in place, and plan or prepare for implementation.

Hiring or Selecting MTSS-R Coaches

  1. First, we’d like to know how you selected the MTSS-R Coach to work in the Approach A and B schools. When joining the study, did you already know who the coaches would be for Approach A and B, or did you need to search for candidates?

  • We already knew who each coach would be. Skip to 13

  • We knew who would coach for Approach A, but not for Approach B.

  • We knew who would coach for Approach B, but not for Approach A.

  • We needed to search for candidates for both Approach A and B.

  1. Did anyone in your district’s central office meet with ECRI or UMN/CORE to discuss the hiring process (e.g., to learn about coach qualifications, schedule times for the coach candidates to meet with the provider)?

  • Yes, we met with both ECRI and UMN/CORE.

  • Yes, we met with ECRI, but not UMN/CORE.

  • Yes, we met with UMN/CORE, but not ECRI.

  • No Skip to 3

    1. We would like to learn about the number of meetings held, how long they lasted, and who participated. Approximately how many meetings with ECRI/UMN/CORE did you have related to the hiring process?


      Meetings

    2. For each meeting, we’d like to know approximately how long the meeting lasted, and who from your district participated in the meeting; specifically, we’re interested in their title. We understand that these meetings occurred many months ago, so please answer to the best of your ability. [Note: The study team will record the meeting times and district participants for any meetings the study team is a part of. The team will share this information with district staff prior to these interviews.]


      How long was the meeting?

      Who participated from [the district] in the meeting (i.e., position)?

      Which provider did you meet with?

      1st Meeting



      • ECRI

      • UMN/CORE

      2nd Meeting



      • ECRI

      • UMN/CORE




  1. Outside of meeting with ECRI or UMN/CORE, did staff from your district’s central office meet to plan the hiring process?

  • Yes

  • No Skip to 4

    1. Approximately how many meetings without ECRI or UMN/CORE did you have related to the hiring process?


      Meetings

    2. For each meeting, we’d like to know approximately how long the meeting lasted, and who from your district participated in the meeting. If the meetings were specific to ECRI or UMN/CORE, please also let us know. We understand that these meetings occurred many months ago, so please answer to the best of your ability.


      How long was the meeting?

      Who participated from [the district] in the meeting (i.e., position)?

      Which provider/approach did the meeting focus on? Select all that apply.

      1st Meeting



      • ECRI

      • UMN/CORE

      2nd Meeting



      • ECRI

      • UMN/CORE




  1. Beyond meetings, did staff from the central office conduct activities related to the hiring process, such as posting the job opportunity(s)?

  • Yes

  • No Skip to 5

    1. Could you say what each activity was, who worked on it (i.e., position), roughly how much time each person spent on this activity, and which provider/approach the activity focused on?

      Activity

      Who worked on this activity from [the district] (i.e., position)?

      Approximately how much time did each person spend on this activity?

      Which provider/approach did the activity focus on? Select all that apply.




      • ECRI

      • UMN/CORE




      • ECRI

      • UMN/CORE





  1. Did someone from your district screen the applications, or did you hire another group to do the screening?

  • Someone in our district screened the applications. Skip to 5b

  • We hired an outside group. Go to 5a

    1. How much did it cost for the outside group to screen the applications?

Cost:


Skip to question 6.

    1. Who in your district screened the applications, and approximately how much time was spent on this work?

      Who screened the applications [the district] (i.e., position)?

      Approximately how much time did each person spend on this activity?



  1. [If the district searched for candidates for both ECRI and UMN/CORE, ask] When candidates were brought in to be interviewed, was the interview about just one position (e.g., just the ECRI position, but not the UMN/CORE position), or were the interviews for either (i.e., for ECRI or UMN/CORE)?

  • Candidates were interviewed for just one position.

  • Candidates were interviewed for either position.

  1. [Ask separately for ECRI and UMN/CORE whether candidates were interviewed for just one position.] How many candidates did your district interview?


    Candidates

  2. For each candidate your district interviewed, who interviewed the candidate (i.e., position), and how long did each interview last, on average?


    Who interviewed the candidates
    (i.e., position)?

    Approximately how long did each interview last, on average?

    1



    2





  3. Did the staff who interviewed the candidates meet to debrief or discuss which candidates to make an offer to?

  • Yes

  • No Skip to 10

    1. Approximately how many meetings took place?


      Meetings

    2. For each meeting, approximately how long did the meeting last, and who from your district participated in the meeting? If the meetings were specific to ECRI or UMN/CORE, please also let us know. We understand that these occurred many months ago, so please answer to the best of your ability.


      How long was the meeting?

      Who participated from [the district] in the meeting (i.e., position)?

      Which provider/approach did the meeting focus on? Select all that apply.

      1st Meeting



      • ECRI

      • UMN/CORE

      2nd Meeting



      • ECRI

      • UMN/CORE




  1. Did the coach selection or hiring process involve any work by central office staff that we did not cover in our questions so far?

  • Yes

  • No Skip to 11

    1. Could you say what each activity was, who worked on it (i.e., position), roughly how much time each person spent on this activity, and which provider/approach the activity focused on?

      Activity

      Who from [the district] worked on this activity (i.e., position)?

      Approximately how much time did each person spend on this activity?

      Which provider/approach did the meeting focus on? Select all that apply.




      • ECRI

      • UMN/CORE




      • ECRI

      • UMN/CORE




  1. Did your district need to purchase any equipment (e.g., computers) to support the hiring or selection process?

  • Yes

  • No Skip to 12

    1. What equipment did your district purchase?

For each type or piece of equipment mentioned…

Purchased equipment

i.
When did your district purchase the equipment?

ii.
How much did the equipment cost?

iii.
Is this equipment used for purposes other than the hiring or selection process?

iv.
How much time during a typical week was this equipment being used for the hiring or selection process?




  • Yes Go to iv

  • No





  • Yes Go to iv

  • No






  1. Were any other resources used by your district for the hiring or selection process? [Probes: In-kind donations, transportation costs, incentives, consultants]

  • Yes

  • No Skip to 14

    1. Please say what the resource was, when your district purchased it, and how much it cost.

Resource

i. When did your district purchase this?

ii. How much did this cost?











Screening and Progress Monitoring

[Note: This section will be included only for districts that replaced their screening or progress monitoring systems. The cost of the new systems will be covered by the provider, and trainings for school staff will be covered through study funds. Therefore, this section asks only about additional work in which the central office staff may have engaged to prepare for the changes in the tool, for districts that replaced their screening or progress monitoring tools.]

Let’s turn now to your screening and progress monitoring system. We understand that your district now uses [Tool Name] for screening and progress monitoring/[Tool Name] for screening and [Tool Name] for progress monitoring in the schools implementing ECRI and UMN/CORE.

  1. Did you have internal meetings among district staff about the transition to the new Screening or Progress Monitoring tools?

  • Yes

  • No Skip to 14

    1. Approximately how many meetings took place?


      Meetings

    2. For each meeting, approximately how long did the meeting last, and who from your district participated? We understand that these meetings occurred many months ago, so please answer to the best of your ability.


      How long was the meeting?

      Who participated from [the district] in the meeting (i.e., position)?

      1st Meeting



      2nd Meeting





  1. Once the new Screening or Progress Monitoring tools were installed, did you or anyone from your district test the tools (e.g., make sure that they were set up correctly and school staff could access the systems online, that data entry and analysis functions worked as intended)?

  • Yes

  • No Skip to 15

    1. Could you say what each activity was, who worked on it (i.e., position), and roughly how much time each person spent on this activity?

      Activity

      Who from [the district] worked on this activity (i.e., position)?

      Approximately how much time did each person spend on this activity?









  1. Did your district purchase any equipment (e.g., computers, iPads) to test the tools?

  • Yes

  • No Skip to 16

    1. What equipment did your district purchase?

For each type or piece of equipment mentioned…

Purchased equipment

i.
When did your district purchase the equipment?

ii.
How much did the equipment cost?

iii.
Is this equipment used for purposes other than the screening or progress monitoring?

iv.
How much time during a typical week was this equipment being used to test the tools?




  • Yes Go to iv

  • No





  • Yes Go to iv

  • No






  1. Were any other resources used by your district to set up or test the screening and progress monitoring tools? [Probes: In-kind donations, transportation costs, incentives, consultants]

  • Yes

  • No Skip to 17

    1. Please say what the resource was, when your district purchased it, and how much it cost.

Resource

i. When did your district purchase this?

ii. How much did this cost?









Supporting Implementation

We’d now like to ask a few questions about the steps that your district took to plan for the implementation of ECRI and UMN/CORE for this school year (2021–22), again focusing on the work that took place prior to the 2021–22 school year. We’ll ask the same questions for both ECRI and UMN/CORE, but we’d like to focus on just one at a time.

Let’s start with ECRI.

  1. Think first about the time before the summer trainings and readiness activities. Did district staff have meetings with school leadership in the ECRI schools to discuss the program’s timeline, implementation, and/or schedule summer trainings and readiness activities?

  • Yes

  • No Skip to 18

    1. Approximately how many meetings took place?


      Meetings

    2. For each meeting, approximately how long did the meeting last, and who from your district participated? We understand that these meetings occurred many months ago, so please answer to the best of your ability.


      Who participated from [the district] in the meeting (i.e., position)?

      How long was the meeting?

      1st Meeting



      2nd Meeting





    3. Did district staff undertake any additional activities to help schools prepare for the summer trainings and readiness activities (e.g., scheduling and confirming teacher and staff participation by sending emails)?

      Activity

      Who from [the district] worked on this activity (i.e., position)?

      Approximately how much time did each person spend on this activity?









    4. After the summer trainings and readiness activities, but before the beginning of this school year, did you or your district undertake any activities to support schools that began implementing ECRI this year (e.g., meeting with school leadership/coaches, visiting schools, reviewing progress monitoring or screening data)?

  • Yes

  • No Skip to 18

      1. Please describe these activities, tell us who worked on each activity, and indicate approximately how much time you think each person spent on the activity this year.

        Activity

        Who from [the district] worked on this activity (i.e., position)?

        Approximately how much time did each person spend on this activity?









  1. Did your district purchase any equipment that staff at the central office used to support ECRI schools (e.g., computers, videoconferencing software) prior to the 2021–22 school year?

  • Yes

  • No Skip to 19

    1. What equipment did your district purchase?

For each type or piece of equipment mentioned…

Purchased equipment

i.
When did your district purchase the equipment?

ii.
How much did the equipment cost?

iii.
Is this equipment used for purposes other than supporting the ECRI schools?

iv.
How much time during a typical week was this equipment being used to support the ECRI schools?




  • Yes Go to iv

  • No





  • Yes Go to iv

  • No






  1. Were any other resources used by your district to prepare for summer training and readiness activities prior to the 2021–22 school year? [Probes: In-kind donations, transportation costs, incentives, consultants]

  • Yes

  • No Skip to 20

    1. Please say what the resource was, when your district purchased it, and how much it cost.

Resource

i. When did your district purchase this?

ii. How much did this cost?









Now let’s turn to UMN/CORE.

  1. Think first about the time before the summer trainings and readiness activities. Did district staff have meetings with school leadership in the UMN/CORE schools to discuss the program’s timeline and implementation and/or schedule summer trainings and readiness activities?

  • Yes

  • No Skip to 21

    1. Approximately how many meetings took place?


      Meetings

    2. For each meeting, approximately how long did the meeting last, and who from your district participated? We understand that these meetings occurred many months ago, so please answer to the best of your ability.


      Who participated from [the district] in the meeting (i.e., position)?

      How long was the meeting?

      1st Meeting



      2nd Meeting





    3. Did district staff undertake any additional activities to help schools prepare for the summer trainings and readiness activities (e.g., scheduling and confirming teacher and staff participation by sending emails)?

  • Yes

  • No Skip to part d.

      1. Please describe these activities, tell us who worked on each activity, and indicate approximately how much time you think each person spent on the activity this year.

        Activity

        Who from [the district] worked on this activity (i.e., position)?

        Approximately how much time did each person spend on this activity?









    1. After the summer trainings and readiness activities, but before the beginning of this school year, did you or your district undertake any activities to support schools that began implementing UMN/CORE this year (e.g., meeting with school leadership/coaches, visiting schools, reviewing progress monitoring or screening data)?

  • Yes

  • No Skip to 21

      1. Please describe these activities, tell us who worked on each activity, and indicate approximately how much time you think each person spent on the activity this year.

        Activity

        Who from [the district] worked on this activity (i.e., position)?

        Approximately how much time did each person spend on this activity?









  1. Did your district purchase any equipment (e.g., computers, videoconferencing software) that staff at the central office used to support UMN/CORE schools prior to the 2021–22 school year?

  • Yes

  • No Skip to 22

    1. What equipment did your district purchase?

For each type or piece of equipment mentioned…

Purchased equipment

i.
When did your district purchase the equipment?

ii.
How much did the equipment cost?

iii.
Is this equipment used for purposes other than supporting the UMN/CORE schools?

iv.
How much time during a typical week was this equipment being used to support the UMN/CORE schools?




  • Yes Go to iv

  • No





  • Yes Go to iv

  • No






  1. Were any other resources used by your district to prepare for summer training and readiness activities prior to the 2021–22 school year? [Probes: In-kind donations, transportation costs, incentives, consultants]

  • Yes

  • No End of Interview

    1. Please say what the resource was, when your district purchased it, and how much it cost.

Resource

i. When did your district purchase this?

ii. How much did this cost?











Spring 2022, 2023, and 2024 Interview Protocol

Note: This form refers to the 2021–22 school year, though the years will be updated for each round, and the questions will be the same across years.

During this interview, we’ll ask questions about costs related to your district’s implementation of the Multi-Tiered System of Supports for Reading (MTSS-R) since we last spoke about the Enhanced Core Reading Instruction (ECRI) program, and the University of Minnesota/Consortium of Reading Excellence in Education (UMN/CORE) program. Our focus is on activities led by the central district office, as well as any supports that the central office provided to the schools implementing the program. In other words, the focus will be on activities conducted centrally, not in individual schools. These activities and resources include any support, monitoring, or resources that you or your district provided to schools implementing ECRI or UMN/CORE during the 2021–22 school year, but exclude any work you may have done to prepare for any plans for the subsequent school year. We’ll ask the same questions for both ECRI and UMN/CORE, but we’d like to focus on just one at a time.

Let’s start with ECRI.

  1. Over the course of the 2021–22 school year, did you or others in your district meet with coach(es) or school staff to discuss the implementation of ECRI?

  • Yes

  • No Skip to 2

    1. What types of meetings did you have? (Probes: check-in meetings with the coaches, meetings with school administrators to discuss ECRI, meetings with other school staff to discuss implementation, meetings with central office staff)

For each type of meeting…

Meeting type

Approximately how often did these meetings occur?

For a typical meeting, who from [the district] would participate (i.e., position)?

Approximately how long did each meeting last, on average?












  1. Have you or your district conducted other activities to monitor or support implementation of ECRI during the 2021–22 school year?

  • Yes

  • No Skip to 3

    1. Please describe these activities, tell us who worked on each activity, and indicate approximately how much time you think each person spent on the activity this year.

      Activity

      Who from [the district] worked on this activity (i.e., position)?

      Approximately how much time did each person spend on this activity?









  1. Did your district purchase equipment for staff at the central office to support the ECRI schools during the 2021–22 school year?

  • Yes

  • No Skip to 4

    1. What equipment did your district purchase?

For each type or piece of equipment mentioned…

Purchased equipment

i.
When did your district purchase the equipment?

ii.
How much did the equipment cost?

iii.
Is this equipment used for purposes other than supporting the ECRI schools?

iv.
How much time during a typical week was this equipment being used to support the ECRI schools?




  • Yes Go to iv

  • No





  • Yes Go to iv

  • No






  1. Were any other resources used by your district to support the ECRI schools during the 2021–22 school year? [Probes: In-kind donations, transportation costs, incentives, consultants]

  • Yes

  • No Skip to 5

    1. Please say what the resource was, when your district purchased it, and how much it cost.

Resource

i. When did your district purchase this?

ii. How much did this cost?









Now let’s turn to UMN/CORE.

  1. Over the course of the 2021–22 school year, did you or others in your district meet with coach(es) or school staff to discuss the implementation of UMN/CORE?

  • Yes

  • No Skip to 6

    1. What types of meetings did you have? (Probes: check-in meetings with the coaches, meetings with school administrators to discuss UMN/CORE, meetings with other school staff to discuss implementation, meetings with central office staff)

For each type of meeting…

Meeting type

Approximately how often did these meetings occur?

For a typical meeting, who from [the district] would participate
(i.e., position)?

Approximately how long did each meeting last, on average?
















  1. Have you or your district conducted other activities to monitor or support implementation of UMN/CORE during the 2021–22 school year?

  • Yes

  • No Skip to 7

    1. Please describe these activities, tell us who worked on each activity, and indicate approximately how much time you think each person spent on the activity this year.

      Activity

      Who from [the district] worked on this activity (i.e., position)?

      Approximately how much time did each person spend on this activity?









  1. Did your district purchase equipment for staff at the central office to support the UMN/CORE schools during the 2021–22 school year?

  • Yes

  • No Skip to 8

    1. What equipment did your district purchase?

For each type or piece of equipment mentioned…

Purchased equipment

i.
When did your district purchase the equipment?

ii.
How much did the equipment cost?

iii.
Is this equipment used for purposes other than supporting the UMN/CORE schools?

iv.
How much time during a typical week was this equipment being used to support the UMN/CORE schools?




  • Yes Go to iv

  • No





  • Yes Go to iv

  • No






  1. Were any other resources used by your district to support the UMN/CORE schools during the 2021–22 school year? [Probes: In-kind donations, transportation costs, incentives, consultants]

  • Yes

  • No End of interview

    1. Please say what the resource was, when your district purchased it, and how much it cost.

Resource

i. When did your district purchase this?

ii. How much did this cost?











American Institutes for Research OMB Clearance Request: District Cost Interviews—18

File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitleThe Impact Evaluation of Parent Messaging Strategies
SubjectOMB Clearance Request
AuthorAIR
File Modified0000-00-00
File Created2021-01-14

© 2024 OMB.report | Privacy Policy