6 Coach Process

Process Evaluation of the Minnesota Reading Corps Site Visit Protocol: AmeriCorps Member

Process Evaluation of the Minnesota Reading Corps

Internal Coach

School Name:

Site Visitor Name:

Date:

Location:

Interviewee Name(s) and Title(s):

Introduction

Good morning/afternoon. My name is [INSERT NAME] and I am with [NORC at the University of Chicago/TIES]. We are working on behalf of the Corporation for National and Community Service to conduct a process evaluation of the Minnesota Reading Corps program to provide a thorough understanding of the operations and activities of the MRC as it is implemented in each school or institution. It will seek to determine if the actual activities and service are true to the intended model of the program, and to assess whether the actual process is likely to produce the intended outcomes and reach the intended target population.

This interview will address these topics and others as we seek to understand your project’s processes and strategies and any aspects of your program that can be replicated in other AmeriCorps programs. The interview should take approximately 75 minutes. Your open and honest opinions are appreciated, but participation is voluntary and you may choose to skip any questions or end the interview at any time. Please also be advised that we will not share your answers with MRC or other school staff; however, we will be providing an evaluation report for each individual project to CNCS. While we will not use your name or any others in this report, and we will attempt to minimize the use of identifiable information, it may be possible for CNCS to identify you through your position or through other details that you share in your interview.

If you have questions about the study after this interview, please contact the Project Director, Carrie Markovitz, at 301-634-9388. If you have questions about your rights as a participant in this research project, please call the NORC Institutional Review Board Administrator at 866-309-0542.

Do you consent to participate in this discussion?

[IF YES, then proceed. IF NO, then terminate interview.]

I would like to record this interview in case my notes are not clear and comprehensive, and to make sure that we accurately report your responses. The recording will be shared only with the small team of researchers working on this study and will be deleted at the end of the project. Do you agree to have this interview recorded for note-taking purposes only?

[IF YES, then proceed. IF NO, then: “That’s fine. Please be patient as I take notes.”]

To begin, we would like to ask some background questions about your site/school. Then we would like learn about the process for selecting an AmeriCorps Member and the staffing and management of the MRC program. Other topics include your coaching responsibilities, alternative programs at your school, MRC program implementation and organizational support for AmeriCorps Members and the MRC program. Finally we would like to discuss facilitators and barriers to program implementation and conclude with learning about results of the MRC program and lessons learned while implementing it.

BACKGROUND

We are interested in learning about the process for developing and implementing the MRC program and hosting the AmeriCorps member at [name of site/school].

1. We understand that the program was implemented here at [name of site/school] in __________.

Why did [name of site/school] apply for the program?

2. Who was involved in deciding to apply? Were you involved?

3. Did staff have any concerns about adhering to the MRC model as a condition for becoming an MRC site?

4. Were any school policies and procedures changed to support program implementation? If so, what were they?

COACHING RESPONSIBILITIES

We would like to discuss your position as a coach in the program and the basic structure of the MRC program within your school.

5. How long have you been a staff member at [name of site/school]?

6. How long have you been an Internal Coach for MRC?

7. How did you become a coach for MRC?

[PROBE: Did you volunteer? Why? Were you asked to be a coach by your principal? Hired? What attracted you?]

8. Are you a MN state-certified reading specialist? YES NO

[If YES] Can you provide some details about what that involves?

9. [For PreK] How many AmeriCorps Members are you currently working with? ___________

1. Are they Professional or Community Corps?

2. Are they full-time (40 hours) or part-time (20 hours) members? Both?

10. [For K-3] How many AmeriCorps Members are you currently working with?

3. Are they full-time (40 hours) or part-time (20 hours) members? Both?

11. About how many AmeriCorps Members have you coached over the years?

AMERICORPS MEMBER SELECTION PROCESS

We are interested in learning about the process for selecting the AmeriCorps Member to serve at [name of site/school].

12. What was the school’s role in this process?

13. Who was involved? [Identify by role/function]

14. What characteristics or qualities were you looking for in an AmeriCorps member?

15. Were any school policies and procedures changed to support hosting an AmeriCorps member? If so, what were they?

STAFFING/MANAGEMENT OF PROGRAM

16. Who is involved in program implementation? What are their roles?

1. Principal

2. Reading Corps Program Coordinator

3. Classroom Teacher (s)

4. External MRC Program Coordinator

17. How have Reading Corps staff and Master Coaches supporting you in implementing the program at your school? [PROBE: planning, monitoring, training]

18. How do all the parties involved in overseeing and implementing the program at [name of site/school] communicate?

[PROBE: Internal Coach, Master Coach, Program Coordinator, Director/Principal].

STUDENTS

We would like to discuss the students served by the program. We ask that you consider the methods used to identify students in need of assistance, the types of students the program was designed to serve, and the extent to which this has been accomplished.

19. Please describe the process of identifying students for literacy interventions/ tutoring. [TARGETING STUDENTS]

1. PreK How are students identified for Tier 2 or Tier 3 interventions?

2. K-3 How do you determine which children will be assessed for tutoring assistance?

3. Both PreK and K-3 How are they screened? What other risk factors do they have?

20. If there are multiple AmeriCorps Members serving at the site/school, please describe the process/criteria for matching students with tutors.

21. How are students, parents, and teachers notified of students’ eligibility?

1. Do you require parental permission in order to provide services? YES NO

[If YES] What is the process for obtaining permission (passive or active consent)?

2. Have any parents declined the offer to have his/her child tutored? YES NO

[IF YES] How does the school handle these cases?

22. What are the desired student outcomes of the tutoring programs?

1. Pre-K

2. K-3

23. How do students feel about being tutored [K-3]?

[PROBE: Enjoy it? Cooperative? Resistant to being tutored? Upset at being pulled out of class? Upset about having to do more work?]

23. Are students making progress to be on track to meet their spring target? YES NO

1. [IF NO] Briefly explain.

CORE CURRICULUM AND SUPPLEMENTAL INSTRUCTION

We would also like to discuss other programs your school uses for serving a similar population.

25. What is the core curriculum for literacy instruction at this site/school? Please describe.

1. Pre-K

2. K-3

25. What other supplemental program(s) is/are used in your site/school for students that are struggling? Please identify.

1. How is/are it/they similar to or different from MRC?

[PROBE: Eligibility criteria; Time on task; Types of interventions; Assessment processes; etc ]

2. How many students and what types of students are served by this/these other program(s)?

3. Do you refer eligible students who cannot be tutored by MRC to another program? (If so, which one, and why?)

4. [IF OUTSIDE THE SCHOOL] Have you worked with any of these other service providers before?

AMERICORPS MEMBERS

Please tell us about the AmeriCorps Member(s).

25. What qualifications does/do the selected AmeriCorps Member(s) have?

25. We are interested in learning about how the AmeriCorps Members serving at [name of site/school] are trained in [name of intervention]. What kind of training does the AmeriCorps Member(s) receive from/by:

1. MRC?

2. [Name of site/school]?

3. Literacy coach?

4. Class-room teacher?

5. Individual training?

25. In your opinion, does the training adequately prepare AmeriCorps Member to provide literacy interventions [PreK]/tutor students [K-3]? [PROBE for intensity, quality]

25. What is the role of the AmeriCorps Member at [school name]?

[PROBE: Provide tutoring only; Recruit and /or train volunteers to support school-based literacy efforts (e.g., read with students during after-school time); Develop literacy resources for families; Coordinate literacy events for families; Other: _____________________________]

MRC PROGRAM IMPLEMENTATION

We would like to learn more about how the interventions and tutoring actually occurs at [name of site/school] with [Pre-K; K-3] students.

25. Ask PreK only:

1. What types of interventions are provided to Tier 2 and Tier 3 students? Briefly describe.

2. Is tutoring provided one-on-one, in small groups? to the whole class? Do you agree that the AmeriCorps Member(s) contribute to a literacy rich environment? YES NO

   1. [IF YES] In what ways?

   2. IF NO] Why not?

25. Ask K-3 only: When you started the school year were only students nearest to target performance receiving services? YES NO Don’t Know

1. [IF NO] As the school year progressed did other students performing below target receive services? Please describe. [PROBE: English Language Learners, Students with an Individualized Education Plan ( IEP), Tier 3 students]

25. We would like to know when the interventions/tutoring sessions are conducted

1. PreK How many days per week do students attend?

[PROBE: 5-days, 4-days, 3-days.]

1. Are literacy interventions provided each day?

2. K-3 Are tutoring sessions provided each day?

[PROBE: Daily, twice weekly, weekly, bi-weekly; day(s) of week; time of day]

25. What is the duration of:

1. PreK Tier 2 or Tier 3 intervention? ________________

2. K-3 tutoring session? ________________

25. Does each of your targeted students receive the same number of

1. PreK Tier 2 or Tier 3 interventions? YES NO

1. [IF YES] Is this an effective strategy?

2. [IF NO] How are these decisions made for each student and by whom?

2. K-3 tutoring sessions? YES NO

1. [IF YES] Is this an effective strategy?

2. [IF NO] How are these decisions made for each student and by whom? [

25. Where do Tier 2 or Tier 3 interventions [PreK]/tutoring sessions [K-3] take place?

Indicate place: ___________________________________________________

1. [IF THE INTERVENTION IS PROVIDED TO THE WHOLE CLASS] Does the teacher stay in the classroom while the tutoring session is conducted?

2. [IF THE STUDENTS ARE REMOVED FROM THE CLASSROOM]

   1. Where exactly does the tutoring take place?

   2. Is this place conducive to [one-on-one tutoring or small group sessions]?

37. There are a number of literacy interventions that are used in the MRC model. Can you tell us how frequently you use the following interventions? Please refer to the list. [Give Respondent the PreK or K-3 list]

37a.	PreK Literacy Rich Daily Schedule	Never	Seldom	About half the time	Usually	Always	If used Always, Usually, or About half the time, why do you use this?	If used Never or Seldom why don’t you use this?
	Arrival
	Sign-in
	Meal time
	Daily Message *
	Repeated Read Aloud * Required for Professional Corps Teacher
	Tier 1 Small Group
	Journal (weekly)
	Choice Time (active learning)
	Tier 2 or Tier 3 (interventions daily)

37b.	MRC K-3 Interventions	Never	Seldom	About half the time	Usually	Always	If used Always, Usually, or About half the time, why do you use this?	If used Never or Seldom why don’t you use this?
	Letter/Sound Correspondence
	Phoneme Blending
	Phoneme Segmenting
	Blending Words
	Repeated Reading with Comprehension
	Strategy Practice
	Repeated Reading Chart
	Newscaster Reading
	Duet Reading
	Pencil Tap
	Stop / Go
	Great Leaps

37. Is any technology used during the tutoring session?

37. Who coordinates the time when students can:

1. PreK receive targeted interventions?

2. K-3 tutoring scheduling?

37. What kinds of assessments are done for each student?

[PROBE: Progress Monitoring; Benchmarking] Who conducts the assessment?

What is the process for assessing whether the AmeriCorps Member is conducting the assessments and/or interventions or delivering the program components according to the MRC model? Please describe the process.

1. Is each component implemented as intended?

37. How often do you meet with the AmeriCorps Member(s) for coaching sessions? ______________

1. What kinds of topics are addressed?

[PROBE: Engagement, delivery of content, challenges, discipline]

2. [Do you] provide additional training? If so, in what areas?

3. How do you handle tutors that you determine require additional training?

42. Did you adapt the materials or processes provided by MRC in any way at [name of site/school]? YES NO

[IF YES] How? Please describe. [PROBE: Who was involved? Was the adaptation approved by the Master Coach? Was MRC involved? Did MRC approve the adaptation?]

ORGANIZATIONAL SUPPORT FOR AMERICORPS MEMBERS AND MRC PROGRAM

We are interested in how the AmeriCorps Member(s) fit in with the rest of the school staff. Also, we’d like to discuss the types of supports provided by MRC to your school and how your school communicates about the program to the community.

43. Do AmeriCorps Member(s) participate in staff meetings, parent-teacher conferences, teacher development sessions, etc?

43. What types of supports does the school staff provide the AmeriCorps Member(s)?

[PROBE: Meetings with Internal Coach, participation in professional development activities, etc.]

43. Have you ever asked to have an AmeriCorps Member(s) removed during the school year? [PROBE: Details on circumstances, MRC’s response, repercussions for school]

43. What types of supports or services does MRC provide your site/school? How are these services delivered?

FACILITATORS AND BARRIERS TO PROGRAM IMPLEMENTATION

We would like to talk with you about what has worked well in implementing the program and where there may have been some challenges.

47. How well do you think the MRC program has been implemented at this site /school? Please explain.

47. What problems or challenges were encountered during implementation? How were they resolved?

47. Have changes in leadership or personnel occurred at [name of site/school]? YES NO

[IF YES] What effect did these changes have?

47. From your perspective, do any current policies and procedures make it hard to implement the program as intended [in this site/school? In this class?] Please describe.

[PROBE: School district, MRC, AmeriCorps State/National]

47. Is there sufficient organizational support for implementing the program? YES NO

[PROBE: Minnesota Reading Corps? Region or school district? Site/school? Other?]

1. [If NO] Could any changes be made with to better support program implementation?

   1. Minnesota Reading Corps

2. Region or school district

3. Site/school

4. Other

47. Is there anything you would change about the MRC program? YES NO

[PROBE: With the model? As implemented at this site/school? Need to adapt to the population?

1. [IF YES] What?

RESULTS AND LESSONS LEARNED

As we conclude our interview, we would like to ask you about your perceptions about the effectiveness of the MRC program and what have been some of the key lessons learned.

53. What kind of feedback have you received about the program:

1. From teachers?

2. From parents?

3. From community members?

4. In each case, what factors do you think contribute to this perception?

54. In your opinion, has MRC helped students better meet their reading proficiency targets? Why or why not?

54. What lessons have been learned about implementing the program at [name of site/school] that might be helpful to other sites/schools?

Thank you for your time. We appreciate it very much.

Note to Interviewer. Be sure to collect secondary documents:

• Marketing materials/brochures

• Parental consent forms

• Assessment forms

• Information on alternative programs

• Schedule template

• School demographics (,may be on website)

16