School Personnel case studies (including revised protocals)

Appendix D

REVISED SITE VISIT PROTOCOLS

National Study of Education Data Systems and Decision Making

School Leader Interview Protocol update

Introduction

The following is a suggested introduction. When you begin the interview, make these points, but use your own words for a more personal introduction:

Remind them that the National Study of Educational Data Systems and Decision Making is a large-scale project sponsored by the US Department of Education to look at the ways that districts and schools are using data in their educational decision making. The project studies the prevalence and quality of both the data systems that are in use nationwide and the supports in place for educational decision makers to use them. In particular for this round of site visits, we are interested to learn about any changes from our last visit in how teachers are using data from these systems to make decisions about their instruction, and what types of professional development and support teachers are receiving to learn about the systems and their applications.

Again, that this project does not evaluate you or your school; we’re looking to learn from your experiences to get a better understanding of how teachers and schools across the country use data systems for teaching and learning.

Before we start, I need to have you review and sign this consent form.

AFTER THEY REVIEW CONSENT FORM: Do you have any questions before we get started?

School Context and Data Use

The purpose of this section is to update information on data-informed decision making at the school level.

1. Have your goals as school leader for using data for instructional decision making in your school changed from last year (2006-07)? IF YES: For what reason?

2. Are the system or systems you use support data-driven decision making at this school changed from last year? [IF MENTIONS SYSTEM OTHER THAN CURRENT ONE] When did you get involved in using [system name]? Please describe the contents of the school system and the reason for its development. [Probe on whether or not this is a system formally used by the district or a supplement to the current district system, what type of data is provided, who has access and how the data are used.]

3. Has your ability to access data increased since last year? IF YES: probe on how access has increased and how the additional data are being used.

1. What proportion of the teachers in this school would you say look at data at least three times a year to assist in planning, conducting, or evaluating instruction?

5. Are teachers in your school able to access more data or access data more frequently than last year? IF YES: probe on what data are now available to teaches and how it is used.

Support for Data-Informed Decision Making

6. IN SCHOOLS WHERE DATA DISCUSSIONS OCCUR IN GROUPS: Who is leading discussions with teachers about data analysis and interpretation? Is this an assigned role or does it vary by topic?

• IF ASSIGNED ROLE: Was knowledge about how to interpret/use data a required skill or were other skills/expertise required?

• From your perspective, what differences do you see when teachers work with data collaboratively vs. on their own?

• Have there been any changes in how data discussions are structured?

7. To what extent have you placed an emphasis on the following strategies over the past 2 years to promote data-driven decision making: [also probe on district efforts]

• Establishing a process for continuous improvement (e.g., developing measurable goals, measuring progress, reflection, making refinements)?

• Establishing a culture of data use at the school (e.g., explicit norms and expectations regarding data use, mutual accountability among staff, creating a safe climate for data use)?

• Designing/acquiring professional development and capacity-building activities for your staff (e.g., school-based professional development, providing tools to interpret and act on data, modeling data use, providing time to discuss data)?

• Building your own capacity for data-driven decision making (e.g., data analysis and interpretation)?

8. Do you feel that teaches in your school still have any unmet needs with respect to professional development in using data for instructional purposes? IF YES: what are they?

9. In your view, are the barriers to system use by teachers the same as last year? IF DIFFERENT: probe on what changes occurred and how access and usability can be improved given the new barriers.

Closing

10. Are there any other things you would like us to know about your work with data-driven decision making that we have not already covered?

Before you leave, be sure to ask for copies of any documents referred to in the interview (e.g., teacher professional development materials or data templates to guide discussions).

National Study of Education Data Systems and Decision Making

Teacher Interview Protocol update

Introduction

The following is a suggested introduction. When you begin the interview, make these points, but use your own words for a more personal introduction:

Remind them that the National Study of Educational Data Systems and Decision Making is a large-scale project sponsored by the US Department of Education to look at the ways that districts and schools are using data in their educational decision making. The project studies the prevalence and quality of both the data systems that are in use nationwide and the supports in place for educational decision makers to use them. In particular for this round of site visits, we are interested to learn about any changes from our last visit in how teachers are using data from these systems to make decisions about their instruction, and what types of professional development and support teachers are receiving to learn about the systems and their applications.

Again, that this project does not evaluate you or your school; we’re looking to learn from your experiences to get a better understanding of how teachers and schools across the country use data systems for teaching and learning.

Before we start, I need to have you review and sign this consent form.

AFTER THEY REVIEW CONSENT FORM: Do you have any questions before we get started?

Data Use and support for Instructional Decision Making

1. Are you able to access more data or access data more frequently than last year? IF YES: probe on what data are now available to teachers and how it is used.

2. Has the usefulness of data changed since last year?

3. Has your ability to make sense of data changed?

4. IN SCHOOLS WHERE TEACHERS ARE INVOLVED IN GROUP DATA USE: In your view, what are the differences in working with data with your colleagues vs. working with data on your own? Probe on differences in: the type of data reviewed, understanding of the data, inferences made from the data, and actions taken.

5. When you have trouble making sense of data, who do you go to?

6. To what extent do you feel that school leadership has placed an emphasis on the following strategies over the past 2 years to promote data-driven decision making in your school [e.g., no emphasis, a little emphasis, a big emphasis—probe on evidence for making judgment]:

• Building staff capacity for data analysis and interpretation—designing/acquiring professional development and capacity-building activities for school staff (e.g., school-based professional development, providing tools to interpret and act on data, modeling data use, providing time to discuss data)

• Establishing a process for continuous improvement (e.g., developing measurable goals, measuring progress, reflection, making refinements)

• Establishing a culture of data sue at the school (e.g., explicit norms and expectations regarding data use, mutual accountability among staff, creating a safe climate for data use)

• What about district efforts in these areas?

7. What do you see as the current barriers to teachers using data that is made available to them?

8. Do you feel that teachers in your school still have unmet needs with respect to professional development in using data for instructional purposes? If so, what are they?

Closing

9. Are there any other things you would like us to know about changes in the school since last year that we have not already covered?

D-7