Study on Sustaining the Positive Effects of Preschool

Appendix E. Principal Interview Protocol

Study on Sustaining the

Positive Effects of Preschool

Principal Interview Protocol

Interviewer_____________________________________ Interviewee ID #_________________

Site __________________________________________ Date/Time_____________________

Introduction

My name is ______, and I work for American Institutes for Research (AIR). We’ve been contracted by the U.S. Department of Education to conduct case studies on sites that are implementing particular programs to support early elementary students.

As you may know, the purpose of the study is to document characteristics of programs and policies in early elementary school that support student growth and development, such as PK–3 alignment, differentiated instruction, and other programs designed to sustain the advantages of preschool. We are interested in learning more about the theoretical or practical background of these programs, how they are implemented, how they are sustained and funded, and their outcomes.

I want to reassure you that your responses to this interview will be used only for research purposes. Though your site’s name will appear in the final report, we will not provide your name in any reporting or to anyone outside the study team, except as required by law. Because we are producing case studies of only a small number of named sites, it may be possible for a reader to identify you based on the data that we report, even though your name will not be used. If there is information that you do not want shared directly in any reporting, please let me know. I will now give you a consent form that provides more details about the study and your rights as a participant.

Do you have any questions about the study?

I plan to audio record this session, solely for our note-taking purposes. Research staff will only use the audio recording, and we’ll destroy it when the project is done. Is that okay with you?

Role and Previous Education-Related Experience

1. I’d like to start by asking you to tell me a bit about yourself. Briefly, how did you come to be principal of [SCHOOL NAME]?

School Context

1. In one or two sentences, how would you describe any unique features of your school to a colleague from another district?

2. You mentioned on the pre-interview survey that X% of your kindergartners attended preschool programs. Do you have any information about the quality, nature, or focus of those programs?

Approach to Program or Strategy and Implementation

My next questions focus very specifically on [PROGRAM]. One purpose of this study is to help other districts and schools think about whether a similar program could work in their context.

4. I understand this program involves [BRIEFLY DESCRIBE OUR UNDERSTANDING OF THE PROGRAM]. Can you tell me more about all the activities involved?

   1. Are you targeting one particular subject matter or outcome?

   2. What are the most important aspects of this program?

5. Can you tell me about the ways you are involved with [PROGRAM] day to day?

   1. How much of your time does this initiative use, compared with your other roles and duties?

   2. Besides yourself, who else at the school is involved with the program and in what ways?

6. To what degree were you involved in the design of the program before the district or school first implemented it?

If involved:

1. What research, theory, or experiences informed the development of [PROGRAM]?

2. Who was involved in the design of the program?

3. Did the initiative involve any approval processes, such as a grant application, new funding streams, or formal adoption by a school board?

4. Has the program changed or evolved since its inception? If so, how?

1. What are the explicit goals of [PROGRAM]?

   1. Is your school doing anything different from or in addition to the district’s program?

   2. If so: What is different and why?

2. In what areas of student performance would you most like to see improvement as a result of your program?

If any of these are mentioned, be sure to get details (such as specific goals and rationale):

• Higher achievement in certain content area

• Better attendance

• Reduced placement in special education

• Reduced grade-level retention

• Better social or behavioral outcomes

9. How do you communicate with parents about the program elements, goals, or vision of [PROGRAM]?

   1. What messages do you emphasize?

   2. What methods of communication do you use?

   3. What questions about [PROGRAM] have you gotten from parents, if any?

   4. Do parents play an active role in the [PROGRAM]? How? Is there training for parents on their role?

10. Let’s talk about when the program was first rolled out here at the school. When was [PROGRAM] first implemented?

    1. If you were at the school then, can you describe what was needed to initially start up and roll out [PROGRAM]?

If not mentioned, ask:

1. What did your school stop doing, or how did you make time, to begin this initiative?

1. What staff training was needed?

2. What resources were required?

3. What had to be done to ensure staff buy-in to the program? What was done to maintain staff buy-in of the program over time?

11. In what ways do you or the district monitor fidelity of the program’s implementation? In other words, how do you know if the program is being rolled out as intended?

ASK QUESTIONS IN THE TWO FOLLOWING SECTIONS, DEPENDING ON WHICH TOPIC(S) IS/ARE A FOCUS FOR THE SITE VISIT.

TOPIC 1: PK–3 Alignment

I’d like to talk a little more about PK–3 alignment efforts at your school or in your district.

12. Are assessments aligned, connected, or coordinated across grades? If so, how?

If not mentioned, ask:

1. What assessments are used? How often are they given?

2. How is the information from assessments used?

3. Does your district use a Kindergarten Entry Assessment? If so, which one?

4. Do you receive the results of assessments for children that attended preschool, from their preschool programs? If so, how is this information used?

1. How are standards in different grades and in preschool related or aligned? How are curricula and/or other materials and resources related or aligned?

   1. What changes were made to standards and curricula, if any, when [PROGRAM] was implemented?

   2. How do you assure children are learning new material and not repeating something they learned the prior year?

2. Can you describe the training or other professional development that teachers receive in order to align instructional practices across Grades PK–3?

   1. Can you describe any ongoing training?

   2. How is data used and shared across grades?

   3. How, if at all, are preschool teachers included in this training and support?

3. Can you describe any training or support that you, as principal, received about PK–3 alignment?

   1. What is helpful about the support for you and what would you change?

4. How do teachers from preschool through Grade 3 know what other teachers are doing in their classrooms?

   1. Do they have time to meet together? When? Who participates? Is this time paid?

5. If pre-interview survey indicates that student or teacher mobility is a problem:

   1. I see that student mobility is a challenge in your school. How do you account for student mobility when implementing [PROGRAM]] if some students miss a portion of the PreK–grade 3 span and enter the program later? Is it difficult to catch them up?

   2. I see that teacher mobility is a challenge in your school. How do you train new teachers to implement [program], given that they did not receive the original training or onboarding for the PreK–grade 3 program?

6. Can you describe any collaborative efforts your school might have with preschool programs?

   1. If this collaboration exists: What types of activities does this collaboration involve?

TOPIC 2: Differentiated Instruction

Let’s talk now about how differentiated instruction looks in classrooms.

19. How do teachers in the classroom work with children at different skill levels?

If not mentioned, ask:

1. Can you describe how and whether students are placed in groups or assigned to specific classrooms or teachers? [If applicable:] How long do these grouping assignments last?

2. Are there specific strategies or practices used for personalizing instruction in kindergarten and first grade that are different from other grades? How are these different than strategies that teachers in later grades (Grade 2 and beyond) might be using?

3. How do teachers differentiate instruction for students with higher skills or competencies?

4. What resources and tools help teachers effectively differentiate instruction with students?

1. How does the school identify the individual needs of students? How is student progress monitored?

If not mentioned, ask:

1. What assessments are used?

2. How is the information from assessments used?

3. How often are children assessed?

If not mentioned, ask:

4. How, if at all, are curricula or other materials and resources used?

5. How, if at all, is technology used?

6. How, if at all, are small groups used?

1. How are teachers trained and supported on differentiation strategies?

   1. Can you describe any ongoing training?

   2. In what other ways does the school support teachers in differentiating instruction?

2. Can you describe any training or support that you, as principal, received about differentiation strategies?

   1. What is helpful for you about that support, and what would you change?

ASK QUESTIONS IN FOLLOWING SECTIONS OF ALL INTERVIEWEES.

Other Specific Interventions or Supports

23. Are there any other programs or supports for early elementary students, designed to sustain the effects of preschool or otherwise support their learning and development, that we have not yet talked about?

If yes, ask questions 24-27:

24. What activities take place at the school as part of this program?

25. Who does the program serve?

If the following are mentioned, make sure to get details:

• Parents

• Particular groups of students (e.g., those who attended preschool, English language learners, students with disabilities)

26. How are activities or approaches decided upon? What guidance do program staff members receive to carry out the program?

27. Tell me about the staff who implements [PROGRAM]. Who are they, and what roles do they play?

If not mentioned, ask:

1. Do you, as principal select them? If so, how did you select them?

2. How were staff roles determined?

3. Have these staff received any special training or professional development? If so, what did that training focus on?

Outcomes

28. You previously stated that your goals were [RESTATE GOALS FROM QUESTION ABOVE]. How do you measure progress against those goals?

29. What effect, if any, do you think [PROGRAM] has had on student outcomes in kindergarten through third grade?

If interviewee reports effect:

1. For which outcomes have you seen gains?

2. How are these gains measured or documented?

3. Have you seen gains for any particular student subgroups?

4. Have these gains been consistent over time?

5. Can you share or point us to the data demonstrating these patterns?

If no effect:

1. What do you think are the barriers to seeing more gains?

30. Has your school documented changes in teacher practices? Teacher turnover?

If yes:

1. Have you seen any patterns?

2. Can you share any data with us?

If no:

1. What do you think are the barriers to changes in teacher practice?

Success, Challenges, and Sustainability

31. Overall, what is working well with [PROGRAM] at your school?

32. What aspect of [PROGRAM] needs strengthening? Why?

    1. What plans do you have to try to strengthen it?

33. What challenges have you experienced in implementing this program, and what steps did you take or may still be needed to address them?

34. Is [PROGRAM] an approach that will or should continue for the long term, in your opinion?

35. Are there resources you need to continue [PROGRAM]?

36. One purpose of this study is to help other schools potentially begin a similar program. What advice would you give to such schools? Are there certain resources, structures, or capabilities that need to be in place for a program like this to succeed?

37. Those are all the questions I have. Are there any questions that I should ask about [PROGRAM] that I didn’t? Is there anything else you’d like to share with me today in regard to [PROGRAM]?

American Institutes for Research Task Order 18: Study on Sustaining the Positive Effects of Preschool—E-1

1875_06/15