Profiles of Selected Practices of Charter Schools, Charter Management Organizations, and Charter School Authorizers

Appendix B

Interview Protocol

Overview

For the Profiles of Selected Practices of Charter Schools, Charter Management Organizations, and Charter School Authorizers project, study team members will interview respondents about their experiences developing and implementing innovative programs/initiatives in selected profile topic areas (Exhibit 1). For each topic area, we will include three sites that could be a single charter school, network, or authorizer or a combination of two or more of these organizations for highlighting in a profile. In each of the sites, we will interview respondents at various levels of the organization (e.g., school staff, school administration, network or authorizer staff) who can provide perspective on the topic we are investigating.

Exhibit 1. Profile Topics

Human Capital Management

1. How are charter schools and networks allocating human capital to address the dual challenges of efficiency and performance?

2. How are charter schools and networks addressing the need to quickly and effectively develop and induct new teachers?

3. How are charter networks addressing the challenge of developing and attracting high-capacity school leaders?

School Improvement

4. How are charter schools, charter networks, and school districts working together to improve chronically low-performing schools?

Serving Students with Disabilities

5. What charter authorizer practices and state policies have been developed to ensure proportionate enrollment of students with disabilities?

Community/Family Engagement

6. How are charter schools, charter networks, and school districts working together to provide family-friendly school enrollment systems to streamline applications and access to schools of choice?

7. How are charter schools, networks, and school districts working together to provide information tools and support systems to help families access choice?

Quality and Accountability

8. How are charter authorizers addressing the need for measurement systems that capture a broad spectrum of program goals?

9. How are charter schools and networks addressing the need to better understand students’ postsecondary outcomes?

Student Supports

10. How are charter schools and networks addressing social emotional learning and mental health needs?

11. How are charter schools and networks using CTE programs to prepare students for postsecondary?

Sustaining and Replication of Effective Practices

12. How are independent charter schools sustaining and scaling effective practices?

Note: We anticipate narrowing this list of profile topics to about 11 of these topics.

This appendix provides draft interview questions, starting with introductory text about the project, a general protocol of key questions and probes that can be used across profile topics and are organized by study constructs aligned to the study questions (Exhibit 2), followed by tailored questions that are relevant to the specific profile topics (Exhibit 3).

Interview Protocol

Prior to the start of each interview, the interviewer will share the following information about the project and participation in the project:

The U.S. Department of Education (Department) has commissioned this project, Profiles of Selected Practices of Charter Schools, Charter Management Organizations, and Charter School Authorizers, to create profiles of selected innovative practices of charter schools, charter management organizations (CMOs) or networks, and authorizers. Each profile will examine one innovative practice and feature charter schools, networks, and/or charter authorizers that are implementing that practice with some indication of favorable student outcomes.

As part of this project, we have asked your organization to participate in [a site visit/phone or video interviews] to learn more about [specify practice]. SRI Education, the Center on Reinventing Public Education, and Policy Studies Associates researchers will be conducting interviews with a sample of respondents from your organization. These interviews will be between 60 and 90 minutes in length. Interview questions will ask about the key features of the practice, implementation challenges and successes, and lessons learned. Your participation in this project will enable the broader education field to learn from your experiences and insights.

Your participation in the interview is voluntary. You will not be penalized in any way for not participating. If you decide to participate, you may discontinue your participation in the interview at any time without penalty of any type. Interviews will only be audio recorded with your consent.

We will protect the confidentiality of the information you provide, to the extent provided by law. After we collect your responses, your name will be disassociated from the data. Your responses will be used to summarize findings in an aggregate manner or will be used to provide examples of program implementation in a manner that does not associate responses with a specific individual.

The type of information we plan to ask respondents will be similar across topics. Thus, we developed one general protocol with a bank of questions that will allow us to tailor questions to the profile topic and respondent role in its implementation (Exhibit 2).

Exhibit 2. General Interview Protocol/Item Bank

Background

1. We understand you are [job title]. Please briefly describe your role and responsibilities.

2. How long have you been involved in the charter school/network/authorizer?

3. What factors led to the creation of the charter school/network/authorizer?

4. What is the mission of your organization?

5. What are the characteristics (e.g., race/ethnicity, socio-economic status, English learners, special education, achievement, age/grade level) of the student population you serve? [For network and authorizer staff] How, if at all, does the population vary by school?

Key Features

6. What are the goals of the [program/initiative]? Probe for the theory behind the program/practice

7. Do all or a subset of [students/teachers/schools] participate in the [program/initiative]? If not all, which subset of [students/teachers/schools] participates?

8. What are the characteristics of the target [students/teachers/schools] for the [program/initiative]? Probe for specific characteristics (e.g., demographics, achievement, motivation). How did you decide who would participate?

9. How many [students/teachers/schools] participate in the [program/initiative]? Probe for percentage and/or number in a year.

Exhibit 2. General Interview Protocol/Item Bank (concluded)

Design and Startup

10. Why did your school [or network/authorizer] decide to implement [program/initiative]? Probe for the problem/challenge the school/network/authorizer was hoping to solve; data (e.g. student achievement results, graduation rates, educator/parent/community input, etc.) that informed your school’s/network’s/authorizer’s decision.

11. What materials and resources (i.e., partnerships, staff, funding, evidence-based models) did your school/network/authorizer enlist to design the [program/initiative]?

12. What was the timeline for design? For example, did you start by piloting on a small-scale and then scale-up?

13. Were there initial, one-time startup costs? If yes, what were they? What ongoing costs are involved?

Implementation of Practice

14. What does the [program/initiative] look like in practice? [Note: For each topic area, we will develop practice-specific sub-questions. Please refer to Exhibit 3 for these topic-specific questions.]

15. What professional supports and/or training do staff involved with [program/initiative] receive?

16. What contextual factors (e.g., local community; authorization practices; state charter law; federal, state or local policies, such as ESSA; student population; leadership; network structure) have informed your school’s/network’s/authorizer’s implementation of the [program/initiative]? Have any contextual factors hindered implementation of the [program/initiative]?

17. What data do you collect about this [program/initiative]? What purpose do these data serve? Probe for how data are used to inform programmatic adjustments

1. In an ideal world, what data would you like to have?

2. What challenges, if any, have you faced obtaining and/or using data? Probe for challenges with record keeping; trouble with analyzing/running reports on data; staff’s difficulty with using data to inform their practice

18. In what ways, if any, has the [program/initiative] changed over time? What led to these changes?

Challenges and Solutions

19. What challenges, if any, has your school/network/authorizer faced in implementing the [program/initiative]? How has your organization overcome these challenges? Probe for challenges to four areas of capacity: policy, organizational, human, resources

20. What factors have contributed to the sustainability of the [program/initiative]?

Signs of Successful Implementation

21. Has implementation of the [program/initiative] met expectations in terms of its desired effects? What impact have you seen? How do you assess its impact? Probe for evidence of effectiveness; types of outcomes collected and used by the charter school/network/authorizer.

22. Were there any unexpected consequences from implementing the [program/initiative]? If so, please explain.

Lessons Learned

23. What lessons have you learned from your organization’s experience implementing the [program/initiative]?

24. What advice would you give to others seeking to implement this type of [program/initiative]?

25. What do you wish you had known before you started undertaking implementation of [program/initiative]?

In addition to the general protocol questions that apply across topics, we will ask questions specific to each of the profile practices. Exhibit 3 outlines the topic-specific questions that we will ask as part of Question #14 in the general interview protocol. For purposes of this information collection request, we include topic-specific implementation questions for all possible topic areas.

Exhibit 3. Topic-Specific Interview Questions

Topic	Who?	What?	Where?/When?	How?/Why?
1. Charter schools and networks implementing flexible staffing strategies (e.g., creating new roles, repurposing existing roles) to increase efficiency and improve performance.	Who developed your organization's approach to staffing? Whose roles are affected by your innovative staffing strategies? Probe for teachers, leaders, paraprofessionals.	Describe how your school/network organizes educators for instruction.	When and where do educators work with students? Probe for specifics (e.g., schedules). When and where do they work with colleagues? Probe for specifics (e.g., schedules). Do they have other responsibilities?	How does staffing this way improve efficiency and student learning?
2. Charter schools and networks quickly and effectively developing and inducting new teachers (e.g., teacher residencies, co-teaching models).	Who leads your organization's effort to develop and induct new teachers? Who does your organization recruit as new teachers? Probe for specific characteristics. Who does your organization rely on to prepare and support new teachers? Probe for specific characteristics.	Describe your school's /network’s model for preparing/inducting new teachers.	When and where does your school/network develop/induct new teachers? When and where do new teachers work with more experienced educators? When and where do they work with students?	How does this model prepare teachers more quickly and effectively?
3. Charter networks developing and attracting high-capacity school leaders (e.g., pipelines for charter school leaders).	Who leads your organization's effort to attract, develop, and retain school leaders? Who do you recruit into leadership positions? Who supports and develops leaders and prospective leaders?	Describe your network’s model for recruiting, developing, and retaining school leaders.	When and where does your network recruit/develop new leaders?	How does this model ensure high-capacity school leadership?

Exhibit 3. Topic-Specific Interview Questions (continued)

Topic	Who?	What?	Where?/When?	How?/Why?
4. Charter schools, charter networks, and school districts working together to improve chronically low-performing schools (e.g., partnerships for school improvement).	Who leads your organization's effort to improve chronically low-performing schools? Who at the [charter school/network/school district] do you partner with on this effort?	Describe your organization’s model for improving chronically low-performing schools.	What criteria does your organization use to determine when to intervene to improve a chronically low-performing school?	What resources does your organization leverage to improve low-performing schools? How does your organization access these resources? Why are they important? How does this cross-sector partnership improve schools?
5. Charter authorizer practices and state policies designed to ensure proportionate enrollment of students with disabilities.	Who leads your organization’s school enrollment system? Who, if anyone, at the state level provides guidance (or oversight) to your organization?	What is the process for enrolling in charter schools? How are families informed of their choices? What, if any, state policies inform enrollment/retention practices for students with disabilities?	When and where do families apply to enroll their child in a charter school? When and where does your organization get involved in the enrollment process? Probe for involvement in examining student attrition.	How do your practices ensure opportunities for students with disabilities?
6. How are charter schools, charter networks, and school districts working together to provide family-friendly school enrollment systems to streamline applications and access to schools of choice?	Which organizations work together to implement the school enrollment system? Who leads this work for your organization?	What information is available to parents/guardians/families to help them make enrollment decisions? What processes facilitate the sharing of enrollment information between charter schools and the traditional school district?	Describe the process for parents/guardians to enroll a child in school. When does enrollment open and close? Where do parents and families enroll their child? When do families learn which school their child was admitted to?	How has this enrollment system expanded choice? access to high-quality education options?

Exhibit 3. Topic-Specific Interview Questions (continued)

Topic	Who?	What?	Where?/When?	How?/Why?
7. How are charter schools, networks, and school districts working together to provide information tools and support systems to help families access choice?	Which organizations work together to implement the school enrollment system? Who leads this work for your organization?	What information is available to parents/guardians/families to help them make enrollment decisions? What processes facilitate the sharing of enrollment information between charter schools and the traditional school district?	Describe the process for parents/guardians to enroll a child in school. When does enrollment open and close? Where do parents and families enroll their child? When do families learn which school their child was admitted to?	How is information about school choice shared with parents/guardians/families? How has this enrollment system expanded choice? access to high-quality education options?
8. Charter authorizers addressing the need for measurement systems that capture a broad spectrum of program goals (e.g., collecting rigorous performance information beyond test scores).	Who designed your organization’s school accountability system? Who leads the work now? Who do you work with at the school level to collect data about school performance?	Describe your school accountability system. What range of program goals does it take into account? What data do you collect to assess school performance? What systems/processes do you use to collect these data?	When do you collect performance data from schools? How frequently do you collect performance data? Why?	How do you use alternative performance data to assess school performance? How have schools reacted to collecting this alternative performance data?
9. Charter schools and networks tracking students’ post-secondary outcomes.	Who in your school/network is responsible for tracking students’ post-secondary outcomes? Are you able to track post-secondary data for all students? If not, which students’ data are missing? Why?	What data do you collect about students’ post-secondary experiences? What systems/processes do you use to track students’ post-secondary outcomes?	• How long, in years, do you track students after high school?	How do you gather data about students’ post-secondary experiences? How do you use data about students' post-secondary outcomes to inform your school/network's program?

Exhibit 3. Topic-Specific Interview Questions (continued)

Topic	Who?	What?	Where?/When?	How?/Why?
10. Charter schools and networks addressing social emotional learning (SEL) and mental health needs.	Who in your school/network supports students’ SEL and mental health needs? Probe for teaching staff, counselors, leaders, outside organizations. Who receives services? Probe for tiered services.	What practices does your school/network use to support students’ SEL and mental health needs? Why? Probe for supportive policies, cultivating a safe and caring environment, providing instruction to students on skills and strategies, coordinating with parents and community. What SEL skills are most important for students to learn so they can be active and successful learners? Probe for conflict resolution, calming themselves when angry, ability to make friends.	Where and when do students receive SEL and mental health support? How often and for how long do students receive SEL and mental health support?	How do students learn SEL skills at your school/network? Probe for curriculum, lessons, activities.
11. How are charter schools and networks using CTE programs to prepare students for postsecondary?	Which staff members are involved in managing and continuing to develop the CTE program?? Probe for existing teaching staff, support staff, outside organizations. How are they selected? Which students participate in CTE courses? How is participation determined?	What resources (curriculum, online learning platforms) do CTE teachers leverage to support their work with students? How is this determined? Please describe the structure and components of the CTE program (e.g., Do students participate in work-based learning? Are there opportunities for dual or concurrent enrollment? Is the CTE theme integrated into all coursework?) What resources are available to students to support them in making decisions about their postsecondary plans?	How many CTE courses do students typically take in each grade?	How are students gaining the skills needed for college and/or career? How does the CTE curriculum build from year to year? Why is it structured in this way?

Exhibit 3. Topic-Specific Interview Questions (concluded)

Topic	Who?	What?	Where?/When?	How?/Why?
12. Independent charter schools sustaining and scaling effective practices.	Who determines which practices will be replicated? Who are the key stakeholders from whom you need to gain buy-in and support to ensure the replication of practices? the sustainability of practices?	What practices have you successfully scaled up? Probe for specific examples.	When is a practice ready for replication? How do you determine where to replicate a new practice?	When scaling practices, how important is fidelity vs local adaptation? How do you monitor implementation of new practices?