| Grantee Interview (Pre and Post) | |||||
| General Changes: For ease of review, questions were divided in the master document between what will be asked in the pre-interview and what will be asked in the post-interview. Questions were also re-ordered to be grouped by topic when possible. The pre interview is divided between "Programmatic Details" and "Partnerships with Schools and Districts." | |||||
| Instrument Name | Original Number | Approved Instrument | New Number | Final Instrument | Explanation/Notes |
| Grantee Pre-Interview | 6 | Were you working in the same schools last year and if not, what changes were made and why? | 1 | Are you working in the same schools this year as last year? If not, what changes were made and why? | Re-ordered to group by topic and improve question flow. Slight wording changes to enhance clarity. |
| N/A | N/A | 2 | What is the process for selecting and placing members in partner schools? How do you work with districts to determine which and how many members will be placed in certain schools? | New question - added based on lessons from Y1 (wanted question aimed at getting a better understanding of selection and placement processes). | |
| 7 & 4 | Can you describe the activities that will be/were implemented by your organization’s School Turnaround AmeriCorps members this school year? For each of the following, elaborate on how your School Turnaround AmeriCorps members are providing services to students and/or families and how do those services differ across schools: Whole School Intervention, Grade Specific Intervention, Student Specific Intervention |
3 | What are the key activities your School Turnaround AmeriCorps members are engaging in at schools this school year? Of these activities, which focus on specific students, which focus on specific grades, and which focus on the entire school? Have these activities changed over the course of the program, and if so, how and why? | Re-ordered to group by topic and improve question flow. Grouped questions 7 and 4 so the respondent explains the breakdown while describing interventions. Also made slight wording changes to enhance clarity. |
|
| 5 | One central objective of School Turnaround AmeriCorps is to increase the school’s capacity to implement school turnaround goals. In what ways did your programming directly or indirectly address that objective? How could programming be changed to better address school capacity building? | 4 | How does your program relate to your host schools’ improvement goals? As applicable, explain how members’ activities fit into the broader scope of the turnaround initiatives of the schools where they serve. | Rephrased because the original wording was too narrow, reworded more broadly to have respondent describe the connection between program and turnaround goals. | |
| 2 | What worked well last year and why? | 5 | What worked well last year and why? Which strategies were more helpful in supporting schools’ turnaround efforts? (As needed) Why do you say that? | Added follow-up questions to obtain additional information from respondent. | |
| 4 | What did not work so well last year and why? What changes have you made to your program for this school year? | 6 | What was one specific issue you overcame last year related to members’ work in schools (e.g., not a grant administration problem)? Follow up: Why did you make these changes? How did you communicate with the school(s) about any changes? |
Rephrased question to ask for specific issue. Aim is to learn about a range of problem-solving strategies, as well as get more insight into specific working relationships with schools and districts and any use of (or absence of) partnership agreements as a resolution tool. | |
| N/A | N/A | 7 | Have any of your schools successfully exited SIG status since the start of the School Turnaround AmeriCorps Program? If any, why do you think the school(s) was effective in addressing turnaround goals? To what extent did AmeriCorps members contribute to that success? | Added question specifically for grantee with SIG exiter, which is a new research question in Y2 that emerged from the findings in Y1. | |
| 11 | Can you please describe your collaboration with the schools and school districts taking part in the School Turnaround AmeriCorps implementation? | 8 | Can you please describe your current relationship with the schools and school districts taking part in the School Turnaround AmeriCorps implementation? Have there been any changes in the partnership over time? | Slight wording changes to enhance clarity. Added follow-up probing about relationship building over time to address updated research question based on findings in Y1. |
|
| 16 | What can your organization do to improve your relationship with your school partners? What can the school(s) do to create a better relationship between school staff and your organization? | 9 | How would you characterize the relationships between AmeriCorps members and school staff members last year? Has your organization or the school(s) done anything to help members fit into the school community? | To build on findings from Y1, slightly modified question to focus more on relationship between AmeriCorps members and staff and how grantee facilitates this relationship. | |
| 14 | What challenges arose in forming and implementing the partnership with the school(s) you are working with? How did you address them? | 10 | What challenges have you faced partnering with school(s)? How have you addressed these challenges? | Slight wording changes to enhance clarity. | |
| N/A | N/A | 11 | Can you describe any challenges or barriers you faced with successfully partnering with schools or districts at the outset of the grant? How did you overcome these challenges, if at all? | Added question to address updated research questions on partnership building based on findings from Y1. | |
| N/A | N/A | 12 | What lessons (both positive and negative) did you learn from the start-up stage of the grant in terms of your school partnership(s) that might be helpful to other grantees and schools? | Added question to address updated research questions on partnership building based on findings from Y1. | |
| N/A | N/A | 13 | Please describe for me the process of coming up with your partnership agreement with your partner school(s). • How did you determine the primary roles and responsibilities of your organization and the school(s) as outlined in your partnership agreement? Did you discuss what these roles and responsibilities would be with the school(s) before drafting the agreement? Who wrote, read, and approved of the partnership agreement(s)? |
Added question to address updated research questions on partnership agreements based on findings from Y1. | |
| N/A | N/A | 14 | How effective do you think the partnership agreements are overall? • Has the partnership agreement been a helpful tool when discussing and implementing the program in schools? Have you revised the agreement since the beginning of the School Turnaround AmeriCorps program and if so, how and why? • Have you experienced any issues with lack of buy-in into program by school leaders despite having partnership agreement? If so, how did you address these issues? |
Added question to address updated research questions on partnership agreements based on findings from Y1. | |
| Post Interview | |||||
| Grantee Post-Interview | N/A | N/A | 15 | Were there any changes in the key activities members engaged in at your School Turnaround AmeriCorps school(s) during the year? Why did you make these changes? To what extent were these changes data informed by data? | Added follow-up to see if/how grantees are tailoring services in response to school needs or requests, or in response to data, to address the focus on time comparisons in Year 2. |
| 2 | What worked well last year and why? | 16 | What worked well this year? Were there any activities that were effective this year that were not as effective last year? | Added follow-up and slightly changed language to make it more applicable for post interview. | |
| 4 | What did not work so well last year and why? What changes have you made to your program for this school year? | 17 | What did not work as well this year? Were there any activities that were challenging this year that were not challenging last year (e.g., new challenges)? If possible, what changes would you make to the design of your program going forward? | Added follow-up and slightly changed language to make it more applicable for post interview. | |
| 5 | One central objective of School Turnaround AmeriCorps is to increase the school’s capacity to implement school turnaround goals. In what ways did your programming directly or indirectly address that objective? How could programming be changed to better address school capacity building? | 18 | We talked in the fall about your partner schools’ turnaround goals. How effective do you feel the program was in helping schools build the capacity to address those goals this year? Are there goals in particular you think the program helped address and if so, why? | Rephrased question to make it clearer for respondents (based on the team's experience in administering this question in Y1). | |
| 10 | Who serves as the primary point of contact between the school(s) and your organization? | 19 | Who served as the primary supervisor of the program this year? Was that person a school staff member or from your organization? What were the supervisor’s responsibilities? What about this role worked well and what did not work as well? | Changed focus of question from "point of contact" to "primary supervisor" to build on findings from Y1. | |
| N/A | N/A | 20 | Were there any [other] major personnel or other changes in the schools you are working with? If so, did this have any impact on the program and how did you address these changes the year? | Added question to reflect updated Y2 research question based on findings from Y1. | |
| 11 | Can you please describe your collaboration with the schools and school districts taking part in the School Turnaround AmeriCorps implementation? | 21 | Can you please describe your relationship with leadership in the schools and school districts taking part in the School Turnaround AmeriCorps implementation this year? Did anything change about your relationship from last year to this year? | Changed wording slightly for clarity and added follow-up to capture change over time. | |
| 14 | What challenges arose in forming and implementing the partnership with the school(s) you are working with? How did you address them? | 22 | Did any new challenges arise in maintaining your partnership with the school(s) you are working with? How did you address them? | Changed wording slightly for clarity and added follow-up to capture change over time. | |
| 16 | What can your organization do to improve your relationship with your school partners? What can the school(s) do to create a better relationship between school staff and your organization? | 23 | What can your organization do to improve your relationship with your school partners? What can the school(s) do to create a better relationship between school staff and your organization? | Re-ordered only, same wording as Y1. | |
| N/A | N/A | 24 | Do you know if there are other school partners or volunteers at your School Turnaround AmeriCorps Schools? Does your organization interact at all with these other school partners? | Adapted from questions approved in comparison principal and teacher interview protocols in Y1. Added to obtain additional perspective on role of other school partners. | |
| N/A | N/A | 25 | If there are additional external partners, how are their activities similar to those of School Turnaround AmeriCorps members? How are they different? | Adapted from questions approved in comparison principal and teacher interview protocols in Y1. Added to obtain additional perspective on role of other school partners. | |
| 21 | What else should we discuss that is important for understanding how your organization interacts with school(s) engaged in this turnaround effort? | 26 | What else should we discuss that is important for understanding how your organization interacts with school(s) engaged in this turnaround effort? | Re-ordered only, same wording as Y1. | |
| 20 | N/A | 27 | What lessons have you learned from observing and working with the school that might be helpful to other organizations? | Adapted from questions approved in teacher and principal focus group protocol in Y1. Added to obtain additional perspective on lessons learned. | |
| Principal Interview - Treatment Schools | |||||
| General Changes: Questions were re-ordered by topic. The topics for this interview are: School Context, Program Activities, School-Grantee Relationships, and Perceived Effectiveness. Since some of the principals will have been interviewed in Y1, questions that are just for principals we have not interviewed yet are marked "Y2-only principals." | |||||
| Instrument Name | Original Number | Approved Instrument | New Number | Final Instrument | Explanation/Notes |
| Principal Interview - Treatment Schools | 1 | Please describe the community and student population your school serves (e.g., demographic composition)? | 1 | [Y2-only principals] Please describe the community and student population your school serves. | No change. Only asking Y2-only principals as have information for Y1 principals. |
| N/A | N/A | 2 | [Y1 principals version] Last year we talked about your school’s population [review stats]. Is there any additional information or context that you think is important for understanding your school and community? [Y2 principals version] Is there any additional information or context that you think is important for understanding your school and community? |
New question added to obtain more information on local context. | |
| 2 | Please describe your school’s turnaround plan. Also, state the change model (turnaround, transformation, school closure, restart) that your school follows. | 3 | [Y2-only principals] Please describe the major components, goals, and objectives of your school’s turnaround plan. Also, state the change model (turnaround, transformation, school closure, restart) that your school follows. Has anything about your turnaround plan changed since last year? | For Y2-only principals, as we have this information from Y1. Added follow-up to capture changes over time. | |
| N/A | N/A | 4 | Please describe any structural or policy changes you have made at your school to meet your turnaround goals. What other resources have you devoted to addressing your turnaround plan? Have these changes impacted the design or implementation of the program? | Added question to get more context on strategies/resources for addressing turnaround goals and address updated research questions revised based on findings from Y1. | |
| N/A | N/A | 5 | In addition to your School Turnaround AmeriCorps members, are there other school partners, volunteers, or external support staff with whom you work or are familiar with who support your school’s turnaround efforts? If so, who are they and what do they do? • How are the activities of external partners similar to those of School Turnaround AmeriCorps members? How are they different? |
Adapted from questions approved in comparison principal and teacher interview protocols in Y1. Added to obtain additional perspective on role of other school partners and provide context for comparative analysis. | |
| 5 | What roles do School Turnaround AmeriCorps members play at your school? | 6 | What are the key activities School Turnaround AmeriCorps members are engaging in at your school this year? Have these activities changed over the course of the School Turnaround AmeriCorps program? If so, how and why? |
Wording changes to enhance clarity. Added follow-up to capture changes over time. | |
| N/A | N/A | 7 | What is the process for selecting students to receive services from School Turnaround AmeriCorps members? | Adapted from approved question in Y1 teacher protocol. Added as school leaders likely know more about selection process. | |
| 4 | Who serves as the primary point of contact between your school and the School Turnaround AmeriCorps grantee organization? | 8 | Do your School Turnaround AmeriCorps members have an on-site supervisor, or some other kind of supervision? If so, is that person from the AmeriCorps organization or your school? What are the supervisor’s roles and responsibilities? If have on-site supervisor, How effective is the on-site supervisor? How have they been helpful in managing and implementing the program? Is there anything that could be done to better manage the program? |
Changed focus from "primary point of contact" to on-site supervision to build on findings from Y1. Added follow-up questions to better understand the structure of the role. | |
| N/A | N/A | 9 | What lessons (both positive and negative) did you learn from the start-up stage of the grant in terms of your school partnership(s) that might be helpful to other grantees and schools? | Adapted from approved question in Y1 teacher and principal focus group protocol. Added to obtain more perspective on lessons learned and revised to focus on improvements to partnerships to address an updated research question, revised based on Y1 findings. | |
| N/A | N/A | 10 | How effective do you think the partnership agreements are overall? Did you have a role in drafting the agreement? • How did you determine the primary roles and responsibilities of your school and the AmeriCorps organization as outlined in your partnership agreement? Did you discuss what these roles and responsibilities would be with the AmeriCorps organization before drafting the agreement? Who wrote, read, and approved the partnership agreement(s)? • How do you use and apply the partnership agreement, if at all, when discussing and implementing the program in schools? • Have you revised the agreement since the beginning of the School Turnaround AmeriCorps program and if so, how and why? • [As needed] How similar or different is the AmeriCorps partnership agreement to how you work with other partners? |
Added question to address updated research questions on partnership agreement, revised based on findings from Y1. | |
| N/A | N/A | 11 | What is the relationship like between School Turnaround AmeriCorps members and school staff this year? How well do you feel staff understand the role of the AmeriCorps members? How has their relationship and understanding changed over time? | Adapted from approved questions in Y1 teacher protocol. Added to better understand the relationship between members and school staff, to build on findings from Y1. | |
| N/A | N/A | 12 | Do you feel that members were well-prepared and had the skills necessary to be effective in your school this year? If not, how do you think School Turnaround AmeriCorps members could be better prepared? | Adapted from approved questions in Y1 teacher protocol. Added to better understand principals' perceptions of member preparedness to contribute at their school, to build on findings from Y1. | |
| 12 | In which of the following ways did School Turnaround AmeriCorps members contribute to your school’s capacity to implement its school turnaround plan? (list of ways) | 13 | One central objective of School Turnaround AmeriCorps is to help schools meet their turnaround goals. How, if at all, have School Turnaround AmeriCorps members helped build your school’s capacity to address those goals? | Re-phrased to enhance clarity and be more open-ended as to whether program is overall helping school address goals. | |
| N/A | N/A | 14 | Which of your turnaround goals has the School Turnaorund AmeriCorps program been more helpful in addressing either directly or indirectly? Which of your turnaround goals was the program less helpful in addressing? How so? • What specific member strategies or activities were most successful in addressing turnaround goals? What were less successful? |
Adapted from question approved in Y1 principal and teacher surveys. Added question to obtain more information and context about the specific turnaround goals the program is impacting, and to address updated research questions revised based on findings from Y1. | |
| 18 | What other supports do you believe are missing? | 15 | What additional supports from AmeriCorps members or other external partners would benefit your school and its turnaround efforts, if any? | Added additional information to clarify question. | |
| Principal Interview - Comparison Schools | |||||
| General Changes: In Y1, the comparison protocol was almost entirely focused on other external partners. The Y2 research questions require a broader comparative focus so the protocol has been modified to address the updated research questions. Many of the questions are modified versions of those in the treatment schools' principal interview protocol in order to allow for clear comparisons. | |||||
| Instrument Name | Original Number | Approved Instrument | New Number | Final Instrument | Explanation/Notes |
| Principal Interview - Comparison Schools | N/A | N/A | 1 | Please describe the community and student population your school serves. | Same as treatment protocol. |
| N/A | N/A | 2 | Is there any additional information or context that you think is important for understanding your school and community? | Same as treatment protocol. |
|
| N/A | N/A | 3 | Please describe the major components, goals, and objectives of your school’s turnaround plan. Also, state the change model (turnaround, transformation, school closure, restart) that your school follows. Has anything about your turnaround plan changed since last year? | Same as treatment protocol. |
|
| N/A | N/A | 4 | Have there been any major changes at your school since the beginning of your turnaround plan ? Have these changes impacted your ability to meet turnaround goals? | Same as treatment protocol. |
|
| N/A | N/A | 5 | Please describe any structural or policy changes you have made at your school to meet your turnaround goals. What other resources have you devoted to addressing your turnaround plan? | Same as treatment protocol. |
|
| N/A | N/A | 6 | Can you give me more detail on what strategies your school has utilized to address your turnaround goals? Which strategies have involved changes or additional efforts by your school staff? Probes: this may include more specific description of professional development activities, tutoring, extended learning time, data monitoring, etc. |
Same as treatment protocol. |
|
| 1 | Does your school have volunteers, support staff, or external partners whose role is to support school-wide implementation of your school improvement initiative? If so, please describe their roles and organizations (if they represent an external partner). What do those volunteers, support staff, or external partners do at your school? Did the activities change throughout the year? If so, then why? | 7 | Are there other school partners, volunteers, or external support staff with whom you work or are familiar with who support your school’s turnaround efforts? If so, who are they and what do they do? • [If have partners] To what extent have external partners been successful in helping your school meet turnaround goals? • [If do nothave partners] To what extent could external partners be a useful resource in helping your school address turnaround goals? |
Slight wording revisions for clarity, and added follow-up questions to better understand external partners' impact on turnaround goals. | |
| 11 | What challenges have you experienced in implementing interventions led by volunteers, support staff, or external partners in your classroom or in student counseling/tutoring programs? How were they resolved? | 8 | [If have partners] Can you please describe your collaboration with these partner organizations? How has that relationship changed over time? What has gone well? What challenges have you encountered? How have you overcome any challenges? | Revised to get broader sense of relationship before addressing challenges. | |
| N/A | N/A | 9 | What is the relationship like between school staff and the external partners? How well do you feel staff understand the role of the partners? How has their relationship and understanding changed over time? | Same as treatment protocol but focused on all external partners. | |
| N/A | N/A | 10 | What sort of training or preparation would external partners need to effectively address turnaround goals and the needs of students at your school? | Same as treatment protocol but focused on all external partners. | |
| N/A | N/A | 11 | Overall, how successful has your school been in addressing turnaround goals? Which strategies (described in #6) have helped build your school's capacity to address those goals, and why? | Added to get more general idea of challenges/success in addressing turnaround goals and address updated research questions, revised based on Y1 findings. | |
| N/A | N/A | 12 | What are the biggest challenges your school has faced in addressing turnaround goals? | Same as treatment protocol but more general than just School Turnaround AmeriCorps. | |
| N/A | N/A | 13 | What type of additional supports would benefit your school and its turnaround efforts, if any? | Same as treatment protocol. |
|
| N/A | N/A | 14 | What lessons have you learned from your school turnaround strategies that might be helpful to other sites/schools? | Same as treatment protocol. |
|
| N/A | N/A | 15 | Is there anything else you’d like to share with me to help us understand your school’s experience with its turnaround goals? | Same as treatment protocol. |
|
| Teacher/Counselor Interview - Treatment Schools | |||||
| General Changes: Questions were re-ordered by topic. The topics for this interview are: School Context, Program Design/Activities, Staff-Member Relationships, and Perceived Effectiveness. | |||||
| Instrument Name | Original Number | Approved Instrument | New Number | Final Instrument | Explanation/Notes |
| N/A | N/A | 1 | Please describe how long you’ve been at this school, your role, and how long you’ve been working in education. | New question added to get context about the respondent's role and tenure at the school. | |
| N/A | N/A | 2 | Is there any additional information or context that you think is important for understanding your school and community? | Adapted from question approved in Y1 principal interview protocol. Added to obtain more information on local context. | |
| N/A | N/A | 3 | How aware are you of your school’s turnaround plan? How does your school leadership team provide you with information on the turnaround plan and school progress towards turnaround goals? | Adapted from question approved in Y1 principal protocol. Added to obtain more information on teachers' understanding of turnaround plan. | |
| N/A | N/A | 4 | In addition to your School Turnaround AmeriCorps members, are there other school partners, volunteers, or external support staff with whom you work or are familiar with who support your school’s turnaround efforts? If so, who are they and what do they do? • How are the activities of external partners similar to those of School Turnaround AmeriCorps members? How are they different? |
Adapted from question approved in Y1 comparison teacher and principal interview protocols. Added to obtain additional perspective on role of other school partners and provide context for comparative analysis. | |
| Teacher/ Counselor Interview - Treatment Schools | 1 | What do School Turnaround AmeriCorps members do at your school? Did the activities change throughout the year? If so, then why? | 5 | What are the key activities do School Turnaround AmeriCorps members do are engaging in at your school this year? • Have these activities changed over the course of the School Turnaround AmeriCorps program? If so, then why? |
Changed wording slightly to enhance clarity and timing of administering the protocol midway through the school year in Y2. |
| 2 | What level of interaction did you have with School Turnaround AmeriCorps members? In what capacity, if any, did you work directly with AmeriCorps members? | 6 | What level of interaction do you have with School Turnaround AmeriCorps members this year? In what capacity, if any, do you work directly with AmeriCorps members? | Changed tense of question for consistency. | |
| 3 | Can you please describe the process for assigning tasks to School Turnaround AmeriCorps members? | 7 | What is the process for selecting students to receive services from School Turnaround AmeriCorps members? | Changed wording slightly to enhance clarity and change focus to role of staff in selecting students to receive services. | |
| 4 | How did you communicate with School Turnaround AmeriCorps members about the progress of your students with whom they were working? | 8 | If School Turnaround AmeriCorps members are working with students you teach or counsel, how do you communicate with them about the progress of these students? | Changed wording to better capture communication between staff and members. | |
| 8 | Who supervised the School Turnaround AmeriCorps members? How adequate is the supervision of AmeriCorps members? By school faculty/staff? By a designated AmeriCorps team leader/supervisor? | 9 | Please describe how School Turnaround AmeriCorps members are supervised at your school. • Do your School Turnaround AmeriCorps members have an on-site supervisor, or some other kind of supervision? If so, is that person from the AmeriCorps organization or your school? What are the supervisor’s roles and responsibilities? i. If have on-site supervisor, How effective is the on-site supervisor? How have they been helpful in managing and implementing the program? Is there anything that could be done to better manage the program? |
Rephrased and added follow-up questions to obtain more specific information on supervision, and build on findings from Y1. | |
| 5 | How prepared were the School Turnaround AmeriCorps members with whom you worked directly to serve at your school? What recommendations do you have about improving the training for AmeriCorps members before or once they begin to work in schools? | 10 | Do you feel that members were well-prepared and had the skills necessary to be effective in your school this year? If not, how do you think School Turnaround AmeriCorps members could be better prepared? | Re-phrased to enhance question flow (based on the team's interview experiences in Y1). | |
| N/A | N/A | 11 | What is the relationship like between School Turnaround AmeriCorps members and school staff this year? How well do you feel staff understand the role of the AmeriCorps members? How has this relationship and understanding changed over time? | Added question to better understand the relationship between members and school staff, to build on findings from Y1. | |
| 11 | What challenges have you experienced in implementing School Turnaround AmeriCorps members’ interventions in your classroom or in student counseling/tutoring programs? How were they resolved? | 12 | What challenges have you experienced in implementing School Turnaround AmeriCorps members’ interventions in your classroom or in student counseling/tutoring programs? How were they resolved? | No changes in wording, just re-ordered. | |
| 9 | In which of the following ways did School Turnaround AmeriCorps members contribute to your school’s capacity to implement its turnaround plan? | 13 | One central objective of School Turnaround AmeriCorps is to help schools meet their turnaround goals. How, if at all, have School Turnaround AmeriCorps members helped build your school’s capacity to address those goals? • Probe: Did they affect relationships between students and staff? Involve families? Improve the school climate? Support the academic needs of specific groups of students? Support students’ behavioral and/or socio-emotional health needs? Provide before school, after school, or extended learning time programming? |
Re-phrased to enhance clarity and be more open-ended as to whether program is overall helping school address goals. Probe included if respondent wants more information. | |
| N/A | N/A | 14 | Which of your turnaround goals has the School Turnaround AmeriCorps program been more helpful in addressing either directly or indirectly? Which of your turnaround goals was the program less helpful in addressing? How so? • What specific member strategies or activities were most successful in addressing turnaround goals? What were less successful? |
Adapted from question approved in Y1 principal and teacher surveys. Added question to obtain more information and context about specific turnaround goals the program is impacting and to address updated research questions (that were revised based on findings from Y1). | |
| N/A | N/A | 15 | What lessons have you learned from observing and working with the program (and members) that might be helpful to other sites/schools? | Adapted from questions approved in teacher and principal focus group protocol in Y1. Added to obtain additional perspective on lessons learned. | |
| Teacher Interview - Comparison Schools | |||||
| General Changes: In Y1, the comparison protocol was almost entirely focused on other external partners. To build on Y1 findings, the Y2 research questions have a broader comparative focus so the protocol has been modified to address the updated research questions. Many of the questions are modified versions of those in the treatment schools' teacher interview protocol in order to allow for clear comparisons. | |||||
| Instrument Name | Original Number | Approved Instrument | New Number | Final Instrument | Explanation/Notes |
| Teacher Interview - Comparison Schools | N/A | N/A | 1 | Please describe how long you’ve been at this school, your role, and how long you’ve been working in education. | Same as treatment protocol. |
| N/A | N/A | 2 | Is there any additional information or context that you think is important for understanding your school and community? | Same as treatment protocol. | |
| N/A | N/A | 3 | How aware are you of your school’s turnaround plan? How does your school leadership team provide you with information on the turnaround plan and school progress towards turnaround goals? | Same as treatment protocol. | |
| N/A | N/A | 4 | How has the school’s turnaround status affected your role, if at all? How does your school’s improvement status and plan affect your responsibilities and focus, if at all? Note: This may include activities such as professional development, instructional strategies, tutoring, data monitoring, etc. | New question, added to obtain more context on the turnaround plan based on findings from Y1. | |
| N/A | N/A | 5 | What strategies has your school utilized to address your turnaround goals? | New question, added to understand the broader strategies being implemented in comparison schools to address updated research questions based on findings from Y1. | |
| N/A | N/A | 6 | How is implementation of particular turnaround strategies going (to the extent that you think of them this way)? What is going well in terms of your school improving and what is more challenging? Note: This includes successes or challenges with classroom strategies, perspective on professional development, relationship with administration, morale among teachers, etc. |
New question, added to understand how well implementation of turnaround strategies is working to address updated research questions based on findings from Y1. | |
| 1 | Does your school have volunteers, support staff, or external partners whose role is to support school-wide implementation of your school improvement initiative? If so, please describe their roles and organizations (if they represent an external partner). What do those volunteers, support staff, or external partners do at your school? Did the activities change throughout the year? If so, then why? | 7 | Are there other school partners, volunteers, or external support staff with whom you work or are familiar with who support your school’s turnaround efforts? If so, who are they and what do they do? | Slight wording changes to enhance clarity. | |
| 2 | What level of interaction did you have with these volunteers, support staff, or external partners? In what capacity, if any, did you work directly with these individuals? | 8 | What level of interaction do you have with these volunteers, support staff, or external partners? In what capacity, if any, do you work directly with these individuals? • Probes: In what capacity, if any, do these partners work with your students? In your classroom? Please explain how your relationship has been with these external organizations. |
Revised to correct tense to refer to the current year and added probe to better understand partner role. | |
| N/A | N/A | 9 | [If do not have partners] Do you think external partners could be a useful resource in helping your school address turnaround goals? If so, how? | Same as treatment protocol but focused on external partners. | |
| N/A | N/A | 10 | [If do not have partners] What sort of training or preparation would external partners need to effectively address turnaround goals and the needs of students at your school? | Same as treatment protocol but focused on external partners. | |
| N/A | N/A | 11 | Overall, to your knowledge, how successful has your school been in addressing turnaround goals? | New question, added to get clearer idea of how comparison school is addressing turnaround goals to enhance comparative analysis based on findings from Y1. | |
| N/A | N/A | 12 | Which turnaround goals have you been more successful in addressing? Which of your turnaround goals have you been less successful in addressing? • What strategies or activities have been most successful in addressing turnaround goals? What have been the least successful? |
Same as treatment protocol but focused on external partners. | |
| 9 | In which of the following ways did these individuals contribute to your school’s capacity to implement its school improvement plan? | 13 | To what extent have external partners or other resources influenced or contributed to your school’s ability to address turnaround goals? | Made question broader to understand the role of partner resources in contributing to addressing turnaround goals, and improve ability to make comparisons to program schools. | |
| N/A | N/A | 14 | What are the biggest challenges your school has faced in addressing turnaround goals? | Same as treatment protocol. | |
| N/A | N/A | 15 | What type of additional supports would benefit your school and its turnaround efforts, if any? | Same as treatment protocol. | |
| N/A | N/A | 16 | What lessons have you learned from your school turnaround initiatives that might be helpful to other sites/schools? Is there anything else you’d like to share with me to help us understand your school’s experience with its turnaround goals? | Same as treatment protocol, and added additional wrap up question. | |
| Teacher Interview - SIG Exiter | |||
| This is a new protocal, designed to address the updated research questions focused on School Turnaround AmeriCorps schools that have exited SIG/Priority status since the beginning of the grant period. Because it is a new protocal all questions are new and we have indicated instead when a question was adapted from a protocol that was approved in Y1. | |||
| Instrument Name | Number | Final Instrument | Instrument Adapted From |
| Teacher Interview - SIG Exiter | 1 | Please describe how long you’ve been at this school, your role, and how long you’ve been working in education. | Teacher interview - program and comparison schools |
| 2 | Is there any additional information or context that you think is important for understanding your school and community? | Teacher interview - program and comparison schools | |
| 3 | Last year (2014-15), how aware were you of your school’s turnaround plan? How did your school leadership team provide you with information on the turnaround plan and school progress towards turnaround goals? | Teacher interview - program schools (although focused on previous year to better understand last year of SIG status). | |
| 4 | What major changes took place at your school over the course of your turnaround status, which started in [YEAR]? How did these changes affect your school’s ability to meet turnaround goals, if at all? | Teacher interview - program and comparison schools | |
| 5 | Why do you think your school was able to exit SIG status? What happened in your district and/or school that enabled your school to meet its turnaround goals (e.g. longer school day/year, academic supports and/or interventions, non-academic supports and/or interventions, partnerships, new leadership, new staff, professional development, curricular changes)? • [If partnerships mentioned] Which partners worked with your school during the turnaround period? Do you think they helped your school exit SIG status? If so, how and why? |
New to SIG-exiter protocol | |
| 6 | Can you describe any specific strategies your district leadership used to help your school meet its goals and exit SIG status? Strategies used by school leadership? And strategies used by teachers and other school staff? • Which strategies were the most effective and why? • Which strategies did not help very much and why? |
Focus new to SIG-exiter protocol, but adapted from questions in program and comparison teacher interview protocols. | |
| 7 | [If school still has members] What are the key activities that School Turnaround AmeriCorps members are engaging in at your school this year? • Have these activities changed over the course of the School Turnaround AmeriCorps program? If so, how and why? • To what extent are members still helpful, even though your school has exited SIG status? |
Teacher interview - program schools. Second follow-up question is new to SIG-exiter protocol. | |
| 8 | In addition to your School Turnaround AmeriCorps members, are there other school partners, volunteers, or external support staff with whom you work or are familiar with who support your school’s turnaround efforts? If so, who are they and what do they do? | Teacher interview - program and comparison schools | |
| 9 | [If school still has members] What is the relationship like between School Turnaround AmeriCorps members and school staff this year? How well do you think staff understand the role of the AmeriCorps members? How has this relationship and understanding changed over time? | Teacher interview - program schools | |
| 10 | What challenges have you experienced in implementing School Turnaround AmeriCorps members’ interventions in your classroom or in student counseling/tutoring programs? How were they resolved? | Teacher interview - program schools | |
| 11 | To what extent do you think School Turnaround AmeriCorps members influenced your school’s ability to exit SIG status? • [If members perceived to contribute to exit] What was most helpful and why? What was the value added of members during your school’s turnaround period? Probe: Did they affect relationships between students and staff? Involve families? Improve the school climate? Support the academic needs of specific groups of students? Support students’ behavioral and/or socio-emotional health needs? Provide before school, after school, or extended learning time programming? • [If members not perceived to contribute to exit] Why weren’t they helpful? What could they have done, if anything, to be more helpful? |
New to SIG-exiter protocol | |
| 12 | What type of additional supports from AmeriCorps members or other external partners would benefit your school, if any? | Teacher interview - program and comparison schools | |
| 13 | [If no longer have members this year] To what extent do you miss the members’ contributions this year? To what extent has your school been able to continue the activities they were involved in? | New to SIG-exiter protocol | |
| 14 | What lessons have you learned from observing and working with the program (and members) that might be helpful to other sites/schools? | Teacher interview - program schools | |
| AmeriCorps Members | |||||
| General Changes: Questions were re-ordered by topic. The topics for this interview are: Members' Duties, Relationships at School, and Perceived Impact. | |||||
| Instrument Name | Original Number | Approved Instrument | Final Number | Final Instrument | Explanation/Notes |
| AmeriCorps Members | 1 | Can you please describe your role and duties at the school where you served? | 1 | Can you please describe your role and duties at the school where you serve this school year (2015-16)? Probe: At how many schools do you serve? How many hours per week on average do you serve across all schools—and how many at [CASE STUDY SCHOOL]? |
Changed to improve clarity. |
| 3 | Did the activities change throughout the year? If so, then why? | 2 | Have your activities changed throughout the school year (2015-16)? If so, how and why? | Changed to improve clarity. | |
| 5 | What are the formal or informal channels among school staff and AmeriCorps members for communicating about school improvement goals (for example, grade- or team meetings, faculty meetings, informal meetings with teachers whose students you support, email communication from principal)? | 3 | What have you been told about the reasons that School Turnaround AmeriCorps members are needed in your school? Who told you this information and how? Who told you this information and how? • Note: this may include the grantee organization, school leader, other school staff, students, written materials, etc. |
This question did not work very well in Y1 and did not seem very meaningful. Re-worded to better understand how AmeriCorps members learn about their role and purpose at the school. | |
| N/A | N/A | 4 | Are you familiar with your school’s improvement goals and/or turnaround plan? What are some of those goals? (School improvement goals = Formal goals communicated by school leadership) | Added question to gauge whether members are aware of turnaround plan. | |
| 7 | In which of the following ways did you (AmeriCorps members) contribute to the school’s capacity to implement school turnaround program activities? | 5 | What are the ways that AmeriCorps members (yourself or as a whole) contribute to the school’s capacity to implement its turnaround plan? Please give me some details about what you mean by this. If there are multiple ways, list the top 3 for me. | Rephrased and added probe to improve clarity. | |
| N/A | N/A | 6 | Are there other school partners, volunteers, or external support staff working at your school? Do you interact at all with these other partners? • If there are other external partners, how are there activities similar to what you are engaging in? How are they different? |
Adapted from question in approved Y1 comparison principal and teacher protocols. Added to obtain better understanding how, if at all, members interact with external partners. | |
| N/A | N/A | 7 | Please describe for me how your work is structured in relation to other work that happens at the school. That is, is your work stand-alone or do you work closely with other school staff? How involved are you in school operations like faculty meetings, professional development, or other initiatives? | New question, added based on findings in Y1 to better understand how AmeriCorps members interact with other programs at the school. | |
| 4 | What types of supports have you received, or are available to you, to implement your planned activities at the school? | 8 | What types of supports have you received, or are available to you, to implement your planned activities at the school? Are there any ways that you could have been better prepared to do your work at the school? | Added follow-up question to align with principal and teacher questions already approved in Y1 protocols. | |
| 6 | What was the general attitude among school staff and faculty to your addition to the site/school when you first began serving? How did perspectives, collaboration, and/or relationships with the school staff (principal/teachers/counselor/school leaders) change throughout the year? | 9 | Now I’d like to hear about your relationships with the staff and faculty at [School name]. What was the general reaction among school staff and faculty to your presence at the school when you first began serving? How did perspectives, collaboration, and/or relationships with the school staff (principal/teachers/counselor/school leaders) change throughout the year, if at all? How well do you feel school staff understand your role at the school? | Rephrased to improve clarity. | |
| N/A | N/A | 10 | What are your relationships like with the students you work with directly? | New question, added to get better idea of member relationships with students (as opposed to staff members and other partners), based on findings in Y1. | |
| 8 | In your opinion, is your AmeriCorps program successful in helping students you directly worked with at this school to improve in the following areas: a) Academic achievement? b) Academic engagement? c) Behavior? d) Attendance? e) Socio-emotional health? |
11 | In your opinion, is your AmeriCorps program successful in helping students you work with directly at this school to improve in the following areas? Is so, how? How have you determined this? a) Probe specifically for: Academic achievement? Academic engagement? Behavior? Attendance? Socio-emotional health? |
Rephrased to improve clarity. | |
| 10 | Are these services helpful to the students you directly worked with at this school? Why or why not? Are there additional supports that you think need to be put in place? | 12 | [As needed, if not already covered] How helpful do you think the services that you are involved with are to the students you directly work with at this school? Why? | Rephrased to improve clarity. | |
| 10 | Are these services helpful to the students you directly worked with at this school? Why or why not? Are there additional supports that you think need to be put in place? | 13 | Are there additional supports that you think need to be put in place in order to help students succeed? Are there additional supports or supervision that would help you better meet the needs of students? | Originally part of Q10, broken into two questions in revised protocol. New sub-question, added in response to Y1 finding about available supervision to better understand supports that members think they need. | |
| 2 | Can you please tell me about your top three activities and responsibilities at the school, in terms of those activities on which you spent the greatest amount of time? For each of these, I’d like to know with whom you collaborated to carry out the activity, how frequently the activity took place, and approximately what percentage of time you spent on the activity. Probe: How did/does this activity support the school’s turnaround effort? Probe: Who was this activity intended to help or support? Probe: What was the frequency and amount of time spent on the activity? |
14 | I want to go back now to the specific services you provide to students to understand exactly what you’re doing in the school. Please describe your three most important activities/responsibilities at the school (the activities that require the greatest amount of your time)? (If at multiple schools, focus on the case study school.) For each of these, I’d like to know with whom you collaborate to carry out the activity, who the target audience is, how frequently the activity takes place, and approximately what percentage of time you spend on the activity. Note: fill in activities based on earlier responses as possible and ask time/frequency/audience/collaboration questions to clarify. Probe: Who do you collaborate with to carry out the activity? Probe: Who is this activity intended to help or support? Probe: What is the frequency and amount of time you spend on the activity? |
Re-ordered and wording changed slightly to improve clarity. | |
| N/A | N/A | 15 | My last question is, what lessons have you learned from serving as an AmeriCorps member at this school that might be helpful to other schools/organizations implementing the program? | Adapted from question in approved Y1 teacher and principal focus group protocol. Added to obtain member perspective on lessons learned. | |
| Parents | |||||||
| General Changes: Primary change is from phone interview to focus group. Other minor wording changes to improve clarity and understanding of the questions. | |||||||
| Instrument Name | Original Number | Approved Instrument | Final Number | Final Instrument | Explanation/Notes | ||
| N/A | N/A | 1 | Please tell us your first name, the grade your child attends at school, and how long you’ve been part of this school community. | Opening question for focus group format. | |||
| 1 | Are you aware of the new School Turnaround AmeriCorps program at your child’s school? The AmeriCorps program is also known as [FILL IN GRANTEE ORGANIZATION’S NAME HERE]. | 2 | How many of you are aware of the (new) School Turnaround AmeriCorps program at this school? The AmeriCorps program here is coordinated by [FILL IN GRANTEE ORGANIZATION’S NAME HERE] and is called [INSERT LOCAL PROGRAM NAME]. | Changed for Focus Group format. Added Local Program name to improve clarity. | |||
| 2 | How did you hear about the program? | 3 | For those who are aware of it, how did you first hear about the program [LOCAL PROGRAM NAME]? Do you continue to hear about the program or receive any information from the program? | Changed for Focus Group format. Added Local Program name to improve clarity. | |||
| 3 | Has your child participated in the program? | 4 | Have your children participated in the program this school year (2015-16)? | Changed for Focus Group format and to improve clarity. | |||
| 4 | What activities has your child taken part in? | 5 | What program activities have your children taken part in this year? (Please describe.) | Changed for Focus Group format and to improve clarity. | |||
| 5 | How does your child feel about the new AmeriCorps program? | 6 | What have your children told you about the program, and how do they feel about the program? | Changed for Focus Group format and to improve clarity. | |||
| 6 | Have you been involved in the program? | 7 | Have you been involved at all with the program? (In what ways?) How do you interact with the program, if at all? | Rephrased to improve clarity. | |||
| 7 | If so, how have you taken part? | (Combined with above) | |||||
| 8 | What is your opinion about the new AmeriCorps program? | DELETED | Reduce burden/Duplicative. | ||||
| 9 | Since your child started participating in the program, have you become more involved in your child’s school? | 8 | Since your children started participating in the program, have you become more involved in the school? If so, how? | Changed for Focus Group format and to improve clarity. | |||
| 10 | Since your child started participating in the program, have you received greater access to information and resources about how you can support your child’s education? | DELETED | Deleted question about greater access to info/resources. Did not ask this last year. Doesn’t seem necessary (very specific). Reduce burden. | ||||
| 11 | Have you seen positive changes brought about by the program? [If needed, probe on academics, behavior, school climate and safety] | 9 | What changes have you noticed, if any, since your children started participating in the program (for example, information sent to you from the school, your child’s homework, academic performance, the way the school feels (school climate))? | Changed for Focus Group format and to improve clarity. | |||
| 12 | Do you have other feedback about the program, either positive or negative? | 10 | Do you have any feedback or comments about the program, either positive or negative? | Slight wording change to improve clarity. | |||
| 13 | Now I will read several statements about your school and the AmeriCorps program. For each statement, please tell me whether you strongly disagree, disagree, agree, strongly agree, or don’t know. | 11 | Now I will read several statements about this school. For each statement, please raise your hand to indicate whether you strongly disagree, disagree, agree, strongly agree, or don’t know. | Changed for Focus Group format. | |||
| 14 | On a scale of 1-4, what is your perception of the overall success of the School Turnaround AmeriCorps program? | 12 | On a scale of 1-4 where “1” is “Very successful” and “4” is “Very unsuccessful,” what is your perception of the overall success of the School Turnaround AmeriCorps program (or use local name throughout) in terms of how successful it is in helping the school improve in general? (By show of hands.) | Changed for Focus Group format. | |||
| 15 | On a scale of 1-4, what is your perception of the School Turnaround AmeriCorps program’s success in improving student academic achievement? | 13 | What is your perception of the School Turnaround AmeriCorps program’s success in improving student academic achievement? (By show of hands.) | Changed for Focus Group format. | |||
| 16 | On a scale of 1-4, what is your perception of the School Turnaround AmeriCorps program’s success in improving student social and emotional health? | 14 | What is your perception of the School Turnaround AmeriCorps program’s success in improving student social and emotional health? (By show of hands.) | Changed for Focus Group format. | |||
| 17 | On a scale of 1-4, what is your perception of the School Turnaround AmeriCorps program’s success in improving school climate (for example, the school’s safety and physical appearance, staff attitudes and school culture, and the general atmosphere and instructional and disciplinary environment)? | 15 | What is your perception of the School Turnaround AmeriCorps program’s success in improving school climate (for example, the school’s safety and physical appearance, staff attitudes and school culture, and the general atmosphere and instructional and disciplinary environment)? (By show of hands.) | Changed for Focus Group format. | |||
| 18 | On a scale of 1-4, what is your perception of the School Turnaround AmeriCorps program’s success in improving the school’s ability to implement its school improvement plan? | DELETED | Reduce burden (Question did not work well last year). | ||||
| N/A | N/A | 16 | Is there anything else you’d like to share today about this school or the program [LOCAL PROGRAM NAME]? Do you have any suggestions for improvements? | Changed for Focus Group format (wrap up question). | |||
| Teacher/Counselor Focus Group - Treatment Schools | |||||
| General Changes: Added new questions to obtain additional context on the participants and their relationship to the program and knowledge of members. Re-phrased and added follow up questions to improve clarity and respondents' understanding of the questions. | |||||
| Instrument Name | Original Number | Approved Instrument | New Number | Final Instrument | Explanation/Notes |
| Teacher/ Counselor Focus Group | N/A | N/A | 1 | I’d like to start by giving each individual a few seconds to introduce himself or herself. Please tell us… • Your name • Grade/content areas that you teach • The number of years you have been teaching (total and in this school) |
Adapted from teacher survey question that was already approved. |
| N/A | N/A | 2 | In what capacity, if any, do you work directly with AmeriCorps members this school year? | Added question to obtain more context on association of participating teachers to program. | |
| N/A | N/A | 3 | What are the key activities School Turnaround AmeriCorps members are engaging in at your school this year? Have these activities changed over the course of the School Turnaround AmeriCorps program? | Adapted from approved question in Y1 teacher program school interview. Added follow up question to capture changes over time. | |
| N/A | N/A | 4 | Can you describe the relationship between School Turnaround AmeriCorps members and school staff? Do members and staff get along? Do staff understand the role of AmeriCorps members? Do members attend any school faculty meetings? | Based on Y1 findings, added question to better understand how AmeriCorps members fit into school community. | |
| 3 & 4 | What were the major strengths of the School Turnaround AmeriCorps program in supporting school turnaround at your school(s) this year? This year, what were the most effective ways that School Turnaround AmeriCorps members supplemented the turnaround resources already in place at the school? Least effective ways? |
5 | One central objective of School Turnaround AmeriCorps is to help schools meet their turnaround goals. How, if at all, do School Turnaround AmeriCorps affect your school’s ability to meet its turnaround goals? What is the added value of members? • What about the program helps it work well in your school? What allows members to work well with students and support turnaround goals? • What specific activities do members engage in that best support turnaround efforts, if any? |
Re-phrased to focus on value of members in addressing turnaround goals. Added follow-up questions to get more nuance on valuable activities and factors that help AmeriCorps members succeed. | |
| 2 | What barriers or challenges did you face in implementing school turnaround in your school(s)? | 6 | What barriers or challenges does the School Turnaround AmeriCorps program face in supporting school turnaround in your school(s) this school year? • What about the program doesn’t work well at your school? What prevents members from supporting turnaround goals? • What specific activities do members engage in that do not effectively support turnaround efforts, if any? |
Added follow-up questions to get more nuance on activities not working as well and challenges AmeriCorps members face. | |
| 5 | In what ways could School Turnaround AmeriCorps members have been better trained/prepared to serve? | 7 | Are members well-prepared and do they have the skills necessary to be effective in your school this year? If not, how do you think they could be better prepared? | Re-phrased to first find out whether staff members think AmeriCorps members are well prepared before asking how they could be better prepared. | |
| 6 | In what ways could schools have been better prepared to work with School Turnaround AmeriCorps members? | 8 | Do you feel that, as staff members, you are well-prepared to work with the AmeriCorps members and make use of their services? If not, in what ways do you think staff could be better prepared to work with School Turnaround AmeriCorps members? | Re-phrased to first find out whether staff members think they are well prepared before asking how they could be better prepared. | |
| N/A | N/A | 9 | What are the unmet needs in your school in terms of supporting students’ academic achievement and school turnaround efforts? How could AmeriCorps members and other external partners help address those needs? | Adapted from approved question in Y1 teacher program school interview. Added question to learn about unmet needs. | |
| 7 | What are your recommendations for improvements to the School Turnaround AmeriCorps program in supporting school turnaround? What lessons have been learned about implementing the program at [name of site/school] that might be helpful to other sites/school? | 10 | What lessons have you learned from working with members and separately any other partners about implementing the program at [name of site/school] that might be helpful to other sites/schools? What are your recommendations for improvements to the School Turnaround AmeriCorps program in supporting school turnaround? | Re-organized to improve question flow (based on experience in administering the question from Y1). | |
| Member Focus Groups | |||||||
| General Changes: Re-phrased to improve clarity and re-formatted questions for in person focus group format. | |||||||
| Instrument Name | Original Number | Approved Instrument | New Number | Final Instrument | Explanation/Notes | ||
| Member Focus Groups | 1 | Can you please describe your role and duties at the school where you served? | 1 | Please tell us your first name and how long you’ve worked in this school. | Simplified to improve clarity for opening with in-person focus group format. | ||
| 2 | Can you please tell me about your top three activities and responsibilities at the school, in terms of those activities on which you spent the greatest amount of time? For each of these, I’d like to know with whom you collaborated to carry out the activity, how frequently the activity took place, and approximately what percentage of time you spent on the activity. | 2 | To get started, I’d like to get a better understanding of exactly what kind of work you’re doing here in this school, I’d like you to list off important activities/responsibilities that you’re involved with at this school. [Moderator note: write these on an easel. Continue until they stop free-listing.] [Choose probes as appropriate] Can I see a show of hands for how many of you do [activity name]? [Moderator note: go through each activity.] Do all members here do the same activities? If not, how and why do activities differ for different members? |
Broke up question into two to improve clarity for in-person Focus Group format. | |||
| N/A | N/A | 3 | Now I’d like to get some more details about these activities. [Moderator note: go through activity by activity. If there are a lot of activities, poll the room for the top 3 activities and only do those.] Probe: Who do you collaborate with to carry out the activity? Probe: Who is this activity intended to help or support? Probe: What is the frequency and amount of time you spend on the activity? (If member serves at more than one school, percentage of time should refer to percentage of total hours across schools. Frequency is based on member’s time, not per student.) |
Broke up question into two to improve clarity for in-person Focus Group format. | |||
| 3 | Did the activities change throughout the year? If so, then why? | 4 | Have your activities changed throughout the school year (2015-16)? If so, how and why? | Slightly modified wording to improve clarity. | |||
| N/A | N/A | 5 | Please describe for me how your work is structured in relation to other work that happens at the school. [As needed: That is, is your work stand-alone or do you work closely with other school staff? How involved are you in school operations like faculty meetings, professional development, or other initiatives?] | Added question based on Y1 findings to better understand integration of members in host schools. | |||
| 4 | What types of supports have you received, or are available to you, to implement your planned activities at the school? | 6 | What types of supports have you received, or are available to you, to implement your planned activities at the school? Probe: Supports might include (but aren’t limited to) training, other people, and materials. Are there any ways that you could have been better prepared to do your work at the school? |
Added follow-up question to align with principal and teacher questions and permit comparative analysis. | |||
| 5 | What are the formal or informal channels among school staff and AmeriCorps members for communicating about school improvement goals (for example, grade- or team meetings, faculty meetings, informal meetings with teachers whose students you support, email communication from principal)? | 7 | What have you been told about the reasons that School Turnaround AmeriCorps members are needed in [School Name]? Probes: Who told you this information and how? Note: this may include the grantee organization, school leader, other school staff, students, written materials, etc. Are you familiar with your school’s improvement goals and/or turnaround plan? (Yes/No) What are some of those goals? (School improvement goals = Formal goals communicated by school leadership) |
This question did not work very well in Y1 and did not seem very meaningful. Re-worded to better understand how AmeriCorps members learn about their role and purpose at the school. | |||
| 6 | What was the general attitude among school staff and faculty to your addition to the site/school when you first began serving? How did perspectives, collaboration, and/or relationships with the school staff (principal/teachers/counselor/school leaders) change throughout the year? | 8 | Now I’d like to hear about your relationships with the staff and faculty at [School name]. Probes [as needed] What was the general reaction among school staff and faculty to your presence at the school when you first began serving? How did perspectives, collaboration, and/or relationships with the school staff (principal/teachers/counselor/school leaders) change throughout the year, if at all? How well do you feel school staff understand your role at the school? |
Rephrased question to improve clarity. Parallels what is asked on Principal Protocol to permit comparative analysis. | |||
| N/A | N/A | 9 | What are your relationships like with the students you work with directly? | Based on Yr 1 findings that highlighted importance of members' relationships with students, added question to obtain better understanding of member relationships with students (as opposed to staff members and other partners). | |||
| 7 | In which of the following ways did you (AmeriCorps members) contribute to the school’s capacity to implement school turnaround program activities? Specifically, how did you help the school build capacity through… | 10 | What are the ways that AmeriCorps members (yourself or as a whole) contribute to your school’s capacity to implement its school turnaround plan? Moderator note: probe for details about each way they improve the school’s capacity. | This question was unwieldy and difficult to administer (based on experience in administering the question in Y1). Changed wording and format to improve clarity. | |||
| 8 | In your opinion, is your AmeriCorps program successful in helping students you directly worked with at this school to improve in the following areas: | 11 | In your opinion, is your AmeriCorps program successful in helping students you work with directly at this school to improve? a) How about in these areas [on easel pad]: Academic achievement? Academic engagement? Behavior? Attendance? Socio-emotional health? (Yes/No – Try to get a consensus for each or note disagreement.) b) How have you determined this? |
Rephrased to improve clarity. | |||
| 9 | How have you determined this? Probe: Ask specifically if members participate in discussions with teachers about student performance based on interim assessment data. Also ask if members participate in discussions with teacher about student attendance, behavior, and engagement. | N/A | Deleted. | Combined as probe with question 11. | |||
| 9 | What kind of academic and social supports are currently in place to help the students in your assigned class/group succeed? Were you or other AmeriCorps members involved in providing these supports? How? | N/A | Deleted. | Deleted/reduce burden. | |||
| 10 | Are these services helpful to the students you directly worked with at this school? Why or why not? Are there additional supports that you think need to be put in place? | 12 | How helpful do you think the services that you are involved with are to the students you directly work with at this school? Why? | Rephrased to improve clarity. | |||
| 11 | How do you think AmeriCorps member services that were implemented this year will help students at this school? | 13 | What lessons have you learned from working as an AmeriCorps member at this school that might be helpful to other schools/organizations implementing the program? | Retained question approved in Y1 for the focus group protocol, and added focus group wrap up question. | |||
| Grantee Survey | |||||||
| General Changes: Added program name and re-phrased questions to improve clarity and aligned with questions in principal survey. | |||||||
| Instrument Name | Original Number | Approved Instrument | New Number | Final Instrument | Explanation/Notes | ||
| Grantee Survey | 1 | Did your organization's relationship with the school(s) exist before you established a partnership agreement for the School Turnaround AmeriCorps program? | 1 | Did your organization's relationship with the school(s) your grant is operating in exist before you established a partnership agreement for the School Turnaround AmeriCorps program? | Added several words to improve clarity. | ||
| 1a | If yes, how long has your organization been collaborating with the school? | 1a | How many years total has your organization worked at this school, including this year? Added response option: "Varies by school (please check all that apply above) | Reworded question to align with principal survey and added response option to obtain more accurate data. | |||
| 2 | To the best of your knowledge, do School Turnaround AmeriCorps members provide direct services to individual students, or to the whole classroom, or school? | 2 | To the best of your knowledge, do School Turnaround AmeriCorps members provide direct services to individual students, to the whole classroom, or to all students in the school during the 2015-16 school year? Added response option "Varies by school (please check all that apply above) and Don't Know | Added several words to improve clarity and added response options to obtain more accurate data. | |||
| 3 | Response option: For some, not all | 3 | Response option: Some but not all | Reworded response option to align with principal survey. | |||
| 5 | Which schools were served by the AmeriCorps members engaged by your organization? | 5 | Which schools were served by the School Turnaround AmeriCorps members engaged by your organization? | Added program name to improve clarity. | |||
| 5 | Response option: Target # of students to be directly served by AmeriCorps and Response option: Target number of students to complete AmeriCorps | 5 | Response option: Target number of students to be directly served by School Turnaround AmeriCorps program (2015-16) and response option: Target number of students expected to receive School Turnaround AmeriCorps program/services this school year (2015-16) | Reworded and added program name to improve clarity. | |||
| 6 | Which schools were served by the AmeriCorps members engaged by your organization? | 6 | Which schools were served by the School Turnaround AmeriCorps members engaged by your organization? | Added program name to improve clarity. | |||
| 6 | Response options: Number of AmeriCorps members, Average number of weeks AmeriCorps members spent in this school, Number of students served by AmeriCorps program, Number of students completing an AmeriCorps program | 6 | Response options: Number of School Turnaround AmeriCorps members, On average, number of hours per week each School Turnaround AmeriCorps members serves (2015-16), On average, number of weeks School Turnaround AmeriCorps members spend in this school this school year, Number of student served by School Turnaround AmeriCorps in this school this school year (2015-16), Number of students expected to complete AmeriCorps programs in this school this school year (2015-16) | Added text to improve clarity, and revised two response options to align with principal survey. | |||
| 7 | Response option: Once a week | 7 | Response option: More than twice per month. Additional response option: Don't know | Modified and added a response option to capture full range of possibilities. | |||
| 8 | To the best of your knowledge, which mechanisms did the school(s) use to identify students for activities led by School Turnaround AmeriCorps members? (Check all that apply.) | 8 | To the best of your knowledge, which mechanisms did the school(s) use to identify students tp participate in activities led by School Turnaround AmeriCorps members? (Check all that apply.) | Slightly modified question text to improve clarity. | |||
| 8 | Response options | 8 | Response option: Don't know | Added response option to capture full range of possibilities. | |||
| 9 | N/A | 9 | Response option: Don't know | Added response option to capture full range of possibilities. | |||
| 9 | Response options | 9 | Response options: Same with "to" in front of each and addition of don't know | Slightly modified wording of response options to improve clarity and to capture full range of possibilities. | |||
| 10 | Instructions: Please check all that apply. | 10 | Instructions "Please rank from 1 to 9, with 1 being most important. Please only rank an outcome if you believe it is important" | Rephrased question instructions to obtain more precise data. | |||
| 11 | Was there improvement in this area in one or more of the students in your schools served by members last year? | 11 | Was there improvement in this area for students in your schools served by School Turnaround AmeriCorps members last year (2014-15)? | Slightly modified question wording to improve clarity. | |||
| 11 | Response options: Yes and No | 11 | Response options: Yes, in all schools, yes in most schools (more than 50%), yes, in some schools (25-49%), Yes in a small number of scohols (less than 25%), No | Modified response option wording to obtain more precise data. | |||
| 12 | N/A | 12 | Added instruction: Please try to respond by thinking in reference to the typical school, if you work with more than one | Added instruction text for question to improve clarity and obtain more precise data. | |||
| 12 | Response option: Not applicable | 12 | Response option: Don't know | Added response option for consistency with other similar questions. | |||
| 12 | The school(s) follows up, if needed when I make contact | 12 | The school responds if needed, when I make contact | Modified question item to improve clarity. | |||
| 14 | For this school year, please indicate how satisfied you are with each of the elements listed below. | 14 | For this school year (2015-16), please indicate your level of satisfaction/dissastifaction with each ot he elements listed below | Rephrased question prompt to improve clarity. | |||
| 14 | Response options: Very different from school to school, Not applicable | 14 | Removed response option very different from school to school and added to not applicable (e.g., this is not part of the structure of services) | Removed response option to align with principal survey and added e.g. to improve clarity. | |||
| 14a | Communication between school and grantee | 14a | Communication between school leadership and grantee | Modified question item wording to improve clarity and to align with principal survey. | |||
| 14b | Communication and collaboration between teachers and AmeriCorps members | 14b | Communication and collaboration between teachers and School Turnaround AmeriCorps members | Modified question item wording to improve clarity and to align with principal survey. | |||
| 14h | Alignment of activities with school turnaround plan | 14h | Alignment of AmeriCorps member activities with school turnaround plan | Added several words to improve clarity. | |||
| 16 | How important are the following characteristics to successfully implementing your School Turnaround AmeriCorps program? | 16 | How important are the following to successfully implementing your School Turnaround AmeriCorps program? | Slightly modified question prompt to improve clarity. | |||
| 16c | Multi-layered supervisory structure to ensure fidelity of program implementation | 16c | Clarly defined, multi-layered supervisory structure to ensure fidelity of program implementation | Modified question item wording to improve clarity and to align with principal survey. | |||
| 16d | Defined framework (e.g. RTI) to guide objective instructional choices and allow for the assessment of program effectiveness | 16d | Clearly defined framework (e.g. RTI) to guide instructional choices and allow for the assessment of program effectiveness | Modified question item wording to improve clarity and to align with principal survey. | |||
| 16e | Limited set of highly scripted interventions that have been shown to be effective (i.e. research based) in achieving desired student level outcomes | 16e | Highly defined set of research-based scripted interventions to improve desired student-level outcomes | Modified question item wording to improve clarity and to align with principal survey. | |||
| 16f | N/A | 16f | Alignment of AmeriCorps member activities to the strategies outlined in the school turnaround plan | Added question item to align with principal survey. | |||
| 16g | Recruitment and selection process that is effective in idenifying members with characteristics that correspond with the program objectives | 16g | AmeriCorps member recruitment and selection process that effectively identifies members and selects members with characteristics/skills aligned with the program objectives | Modified question item wording to improve clarity. | |||
| File Type | application/vnd.openxmlformats-officedocument.spreadsheetml.sheet |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |