An Impact Evaluation of Training in Multi-Tiered Systems of mSupport for Behavior (MTSS-B)

Appendix B: Site Visit Interview of Behavior Team Leader Protocol

Behavior Team Leader (Fall 2015; Spring 2017)

These questions are from the School-wide Evaluation Tool (SET)¹ and Individual Students Systems Evaluation Tool (ISSET) ²and stay the same at each wave.

Date:

Interviewer_ID: District_ID: School_ID: STAFF_ID:

Introduction (Introduction will be repeated each wave)

Good morning/good afternoon. My name is [name] and I work with the Multi-Tiered Systems of Support for Behavior (MTSS-B) study team.

We are conducting interviews with school administrators and team leaders to learn their thoughts about behavior support practices at their school. We anticipate the interview will take no more than 30 minutes. Your participation in the interview is completely voluntary. We hope you will participate because your perspective is essential to understanding how to improve behavior support practices in schools across the country.

All the information we collect is confidential, and you, your school, or anyone working at or attending your school will not be identified by name. I have a document here describing the study and this interview for your reference.

Do you have any questions before we begin?

National Study of Multi-Tiered Systems of Support for Behavior (MTSS-B)

Site Visit Interview

What is the purpose of the study?

This study is evaluating Multi-Tiered Systems of Support for Behavior (MTSS-B), an approach to teaching and reinforcing appropriate behavior for all students and providing additional supports for students with greater needs. Some schools in the study have been randomly selected to receive training and support in MTSS-B and other schools in the study were randomly selected to continue with their existing behavior support practices. Data is being collected to understand how this program works across schools and districts; it will not be used to evaluate individual schools, teachers, or staff members.

The U.S. Department of Education has picked MDRC, a research group, to carry out the study. MDRC is working with partner organizations to collect data for this research study—American Institutes of Research (AIR), Decisions Information Resources (DIR) and Harvard Graduate School of Education. These four organizations make up the MTSS-B study team.

What am I being asked to do?

You are invited to complete an interview about behavior support practices. It will take approximately 30 minutes.

What are the benefits from taking part in this interview?

We hope that you will feel satisfied knowing that you are helping to improve teaching and learning in your school and throughout the country.

What are the possible risks of this interview?

There are no anticipated risks of participating in this interview. Your responses and personal information will be stored securely and will not be released to any person outside of the research team. Information collected will not be used to evaluate you or other individuals in your school.

Who will know about my participation in this interview?

Information collected for this study comes under the confidentiality and data protection requirements of the Institute of Education Sciences. All information from this study will be kept confidential as required by the Education Sciences Reform Act of 2002 (Title I, Part E, Section

183). Responses to this data collection will be used only for statistical purposes. Personally identifiable information about individual respondents will not be reported. We will not provide information that identifies you, your school, or your district to anyone outside the study team, except as required by law.

Is my participation in this interview voluntary?

Your participation in the interview is completely voluntary and your decision will not affect your current or future status in your school. We hope you will participate because your views and experiences are very important to us. If you begin the interview and change your mind, you can withdraw at any time.

If I have additional questions, how can I get them answered?

If you want more information about this study, please contact Pamela Wells at Decision Information Resources (DIR), at [email protected] or by calling, 832-485-3720. The MTSS-B Project Director, Fred Doolittle, can be reached at [email protected] or by calling 212-340-8638.

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless such collection displays a valid OMB control number.  The valid OMB control number for this information collection is #1850-NEW.  The time to complete this interview is estimated to average 45 minutes, including time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. The obligation to respond to this collection is voluntary.  If you have comments or concerns regarding the status of your individual submission of this form, application or survey, please write directly to Lauren Angelo, U.S. Department of Education, Institute of Education Sciences, 555 new Jersey Avenue, NW, Suite 502h, Washington, D.C. 20208 or email [email protected].

Do you have any questions before we begin?

Student Support Team Leader INTERVIEW

1. How often does your __(name of SST/CST)_______ team meet? (once a week, twice per month, monthly, once a semester, etc.)

2. Are (written) procedures in place for referring students to the _____________ team? (Collect SST referral form at this time.)

1. Please describe process.

3. Is there a documented (written) process for determining if a student (not with an IEP) needs access to Targeted or Intensive interventions?

4. What is the amount of time that it typically takes the SST (or someone on the SST) to respond to a teacher regarding a request for assistance? (in days)

5. Are all staff members who are directly involved notified about students receiving targeted or intensive support services? Do they receive ongoing updates about student progress?

6. Is there a form or document that is used to monitor individual student progress on a monthly basis? (Please note what material you have seen; e.g., progress report form, intervention form contains place for data monitoring)

7. Is there a documented (written) process for selecting evidence-based interventions for individual students (i.e., targeted interventions)? (Please note what material you have seen or is referenced; e.g., flowchart, table, computerized system)

Now I need to collect some information about interventions that you may implement at your school. I am looking for interventions in the following areas (different interventions for each area):

1. Classroom Management Strategies – any intervention/program that teachers in the school can access and use in their classrooms to promote classroom management skills. Please not the name of the program in the table below.

2. Other Social-emotional learning program (school-wide intervention; e.g. Character Counts) – any intervention/program that ALL the students in the school can access. It may be social skills lessons, like character counts.

8. Do you use any school-wide interventions at this school for each area?

List names of interventions in the table

Critical Features for School-Wide Interventions	Classroom Management Strategy _________________ (specify) 0 = No 2 = Yes	Social-emotional Learning _______________ (specify) 0 = No 2 = Yes	Total Points
Intervention is linked directly to school-wide expectations and/or academic goals.	0 2	0 2
Intervention is implemented continuously throughout the school year.	0 2	0 2
Intervention is implemented during class time or homeroom.	0 2	0 2
Uses data to monitor impact of intervention. (0=no/verbal, 2=see written documentation)	0 2	0 2
Intervention results in students receiving positive feedback from staff.	0 2	0 2
Intervention results in students receiving direct instruction in the skill development.	0 2	0 2
Intervention has a process for monitoring whether it is being implemented as designed.	0 2	0 2
All staff implementing the intervention have received initial training in the intervention.	0 2	0 2
Trained staff receive ongoing coaching on how to implement the intervention.	0 2	0 2
Written instructions exist for how to implement intervention. (0=no/verbal, 2=see written documentation)	0 2	0 2

Finally, I need to collect some information about the interventions used at your school.

9. Do you use any interventions at this school for each area?

• Student engagement and truancy (e.g. Check In, Check out) - any student engagement or truancy intervention/program that is only available to a subset of student population who have an identified need. Check in/Check out and Check & Connect are examples.

• Academic Intervention - any academically focused intervention that is only available to a subset of the student population who have an identified need.

Critical Features for Targeted and Intensive Individualized Interventions	Check In / Check Out _______________ (specify) 0 = No 2 = Yes	Academic Intervention _____________ (specify) 0 = No 2 = Yes	Total Points
Intervention is linked directly to school-wide expectations and/or academic goals.	0 2	0 2
Once the intervention is in place, intervention is continuously available for student participation.	0 2	0 2
Intervention is implemented within 3 school days of determination that the student should receive the intervention.	0 2	0 2
Uses data to monitor impact of intervention. (0=no/verbal, 2=written documentation)	0 2	0 2
Intervention results in student receiving positive feedback from staff.	0 2	0 2
Intervention requires no more than 10 min. per day from any instructional/supervisory staff to monitor (other than people who coordinate, implement, or manage the program).	0 2	0 2
Intervention has a process for monitoring whether it is being implemented as designed.	0 2	0 2
All staff implementing the intervention have received initial training in the intervention.	0 2	0 2
Trained staff receive ongoing coaching on how to implement the intervention.	0 2	0 2
Written instructions exist for how to implement intervention. (0=no/verbal, 2=see written documentation)	0 2	0 2
Written description of intervention is provided to student’s classroom teacher or parent. (0=no/verbal, 2=see written documentation)	0 2	0 2

10. When a team meets to complete an FBA, does the team include:

1. an administrator?

2. one of the student’s teachers?

3. an individual with knowledge about FBAs?

4. use of a written protocol to guide the process? (Please collect the FBA form at this time.)

10. Is there documented evidence that the FBA team has received staff development in Tier 3 procedures this school year? (Please note what material you have seen.)

1 Sugai et al., 2005

2 Anderson et al., 2014. ; Debnam et al., 2012