Evaluation of the ESSA Title I, Part C, Migrant Education Programs (Study Instruments)

September 5, 2017

The Study of Implementation of the ESEA Title I—Part C Migrant Education Program Serving Children of Agricultural Workers and Fishers

Draft (Revised) Appendix F- Local Subgrant Coordinator/Administrator of the Local Migrant Education Program Interview Protocol

LOCAL EDUCATION AGENCY (LEA)

Contract GS-10F-0554N/BPA Order ED-PEP-16-A-0005/TO01

SRI Project P24149

Submitted to:

Joanne Bogart

Policy and Program Studies Service

U.S. Department of Education

400 Maryland Avenue, SW

Washington, DC 20202

Prepared by:

Policy Studies Associates

Leslie Anderson

SRI International

Deborah Jonas

Rebecca Schmidt

Introduction to the Interview

Introduce the interviewer(s).

Explain the purpose of the study and the topics to be covered in the interview. Interviewers will read the following statement to interview respondents at the beginning of each interview:

The Study of Implementation of the Elementary and Secondary Education Act (ESEA) Title I—Part C Migrant Education Program (MEP) is being carried out under contract for the U.S. Department of Education by Policy Studies Associates (PSA), a research organization, Arroyo Research Services (ARS), a research organization, and SRI International (SRI), an independent, nonprofit research institute (collectively “the study team”). The implementation study will describe how state and local MEP-funded programs have begun to change in response to changing requirements under ESEA as reauthorized by the Every Student Succeeds Act (ESSA) and how they are serving the needs of a changing migratory student population.

Explain the provisions for protecting respondent’s privacy. Interviewers will read the following statement to interview respondents at the beginning of each interview:

As part of the implementation study, the study team will share its findings with the U.S. Department of Education. However, the results of the interviews will be summarized across all the states, districts, and programs participating in interviews. Individuals will not be identified by name in any report, and data will be reported in a manner that does not reveal your identity or the identity of your program. However, we may use anonymous quotes to help illuminate the findings. Except for that which is already public, every effort will be made to maintain the confidentiality for all information collected and we will not provide information that identifies you or your district to anyone outside the study team, except as required by law. Participation in this study is required under Section 8306(a)(4) of the Elementary and Secondary Education Act.

Advise the respondent that the interview will last approximately 45-60 minutes.

Invite questions from the respondent.

Ask permission to audio-record the interview using the following statement:

We would like to record this conversation in order to ensure that we accurately capture your comments. If you agree, we would retain the recording only until we are able to validate the transcripts, at which point the voice recording would be destroyed. The transcripts will be destroyed at the end of the study. If at any point you would like to say something off the record, I will stop the recording. Do you agree to allow us to record the interview?

Confirm respondent’s current roles and responsibilities within the agency/organization and in terms of the MEP Program. Ask if there are others we should speak with regarding MEP.

Interview Questions

NOTE: Interviewers will ask all follow-up questions as specified in the protocol. In addition, interviewers will be directed to seek clarification of any response they judge to be vague or incomplete. The primary strategy for seeking clarification will be to ask respondents to provide specific examples to illustrate their responses. A second strategy will be to ask respondents to “say more” or to “expand on” or “explain the meaning of” a particular comment.

Program Staffing and Administration

1. What is your professional background?

   1. How long have you worked for the Local Education Agency (LEA)?

   2. How long have you worked with the Migrant Education Program (MEP) within the LEA? Outside the LEA?

   3. How many hours a week do you estimate you devote to MEP-related activities?

   4. What other responsibilities, if any, do you hold within the LEA?

2. In which of the following MEP-related activities is the LEA involved?

1. Identifying and recruiting (ID&R) migrant students (e.g., setting identification and recruitment (ID&R) policies; directly managing the process of identifying and recruiting eligible migrant students, contracting with an external provider to conduct ID&R activities, etc.)

2. Identifying and/or providing program services and supports to serve the needs of eligible migrant students and out-of-school youth (OSY)

3. Collaborating with other agencies and organizations working with migrant or other at-risk populations to serve their needs

4. Evaluating/monitoring service delivery, quality, and outcomes/impact

5. Providing professional development and support to providers (e.g., classroom teachers, school counselors, before or after school program staff, etc.) working with migratory children and youth

6. Other

3. Please describe the MEP staffing.

   1. How many LEA staff [number of staff positions and FTEs] are assigned to the MEP? How many staff are full-time on MEP?

   2. What are the administrative responsibilities and time commitments of LEA staff assigned to the MEP? [Probe for: delivering program services and supports for migrant students; collaborating with other agencies and organizations; providing professional development and support to providers and parent involvement activities; identifying and recruiting eligible migrant students; evaluating/monitoring service delivery and quality; COE review, data entry, and management; responding to state requirements, etc.]

   3. If LEA MEP staff have administrative responsibilities to other programs within the SEA, what are those other responsibilities?

4. In your opinion, is the number of MEP-funded staff sufficient to administer the MEP effectively? If so, why? If not, why not?

5. Do you contract with any outside contractors or consultants for services related to the MEP? If not, why not?

   1. [If yes:] How many outside contractors/consultants does your program work with and what services do they provide?
      [Prompt: evaluation, technical assistance, re-interviewing, service delivery plan, comprehensive needs assessment, professional development, recruiting, OSY services]

      1. Are contractors providing educational services (e.g. tutors)?

      2. What percentage of your administrative set-aside under the MEP is allocated to outside contractors/consultants?

Identification and Recruitment

6. [If indicated in Question 2 that LEA is involved in ID&R] What is the size of the eligible migrant student population in the district?

7. How does your LEA identify and recruit migrant students?

   1. How many LEA staff are involved in ID&R activities?

   2. What ID&R strategies are used? [Probe: Strategies/criteria for selecting recruiters; number of recruiters; particular regions or school districts that receive special focus/are of highest priority; strategies for identifying hard-to-reach populations]

   3. How, if at all, does your LEA work with other agencies, organizations, schools, and community-based organizations to coordinate its migrant student identification and recruitment efforts?

   4. What criteria are used for identifying eligible migrant students?

   5. Does your LEA apply any additional criteria beyond the federal criteria for determining migrant student eligibility?

      1. If so, what criteria and why?

   6. Are there processes in place to ensure the accuracy of eligibility determinations?

      1. If so, what are they?

8. What changes has the district—made to its migrant student identification and recruitment policies and practices in response to the Every Student Succeeds Acts (ESSA)? (Probe for: definitions of migratory workers and fishers; migratory child; qualifying move; PFS; OSY; funding allocations]

   1. What identification and recruitment strategies have been added or changed so far in response to ESSA—or for other reasons—and why?

      1. What were the benefits and challenges, if any, of these additions or changes?

   2. What plans, if any, does your district have to change its ID&R practices in the future in response to ESSA—or for other reasons—and why?

   3. Does the LEA apply any additional criteria beyond the federal criteria for determining migrant student eligibility?

9. In what ways do identification strategies differ for migratory out-of-school-youth (OSY) (i.e., those who have already dropped out of school, are working towards their GED, or are “here to work” only) compared with migratory youth who are still in school?

10. Please describe some of the successes and challenges with ID & R activities at the district level. In your opinion, are all or the vast majority of eligible migratory children identified and recruited in your LEA? If so, how do you know? If not, why not?

Migrant Student Needs

11. What are the most significant barriers or challenges that migrant students in your district face that may disrupt or delay their educational progress? [Probe for: cultural and language barriers, social isolation, academic achievement, educational continuity, access to services, health-related problems, family support, etc.]

12. How, if at all, have changes in migrant communities in your LEA affected the current needs of migrant students? [Probe for: small/large migrant population size, concentrated or spreading population, fluctuating migrant population numbers, funding, cross-agency collaboration]

13. Does your State’s Service Delivery Plan (SDP) align with or influence local priorities?

    1. In what ways did your program provide input on your state’s SDP?

    2. In what ways does the state’s SDP change what you do in terms of the students served and the services provided?

    3. Do you believe the MPOs focus on the appropriate outcomes for the MEP? If not, why not?

       1. What migrant student needs have been identified that the state’s SDP and MPOs do not address? Why?

       2. What MPOs do you believe are particularly challenging for your program to achieve? Why?

MEP-funded Services and Activities

NOTE TO INTERVIEWER: Request a copy of their Service Delivery Plan (SDP).

14. Please describe the types of instructional services your LEA funds or directly delivers to support the needs of migrant students.
    [Probe for: instructional services including ELA, math, and other supplemental instruction; credit recovery/online courses; guidance/advocacy services; preschool services; preparation for post-secondary transition, etc.]

    1. Who delivers these instructional support services to eligible migrant students?

       1. How many school staff are involved in delivering those services

       2. In what ways have they received training in working with migrant student populations?

    2. What are the challenges, if any, to delivering these instructional services to eligible migrant students?
       [Probe for: small/large migrant population size, concentrated or spread population, fluctuating migrant population numbers, funding, level cross-agency collaboration]

       1. How are these challenges addressed?

    3. To your knowledge, what benefits have resulted from the instructional services delivered to eligible migrant students? How do you know?

    4. How, if at all, are these services evaluated?

15. Please describe the types of support services your LEA funds or directly delivers to support the needs (of migrant students.
    [Probe for: health, dental, and eye care; school supplies; clothing; transportation; parent education and support; referrals etc.]

    1. Who delivers these support services to eligible migrant students?

       1. How many school staff are involved in delivering those services

       2. To what extent have they received training in working with migrant student populations?

    2. When are they usually delivered (e.g., during the school day, before or after school, in the summer months or intercession periods)?

    3. What are the challenges, if any, to delivering these services to eligible migrant students?
       [Probe for: small/large migrant population size, concentrated or spread population, fluctuating migrant population numbers, funding, cross-agency collaboration]

       1. How are these challenges addressed?

    4. To your knowledge, what benefits have resulted from the support services delivered to eligible migrant students? How do you know?

    5. How, if at all, are these services evaluated?

16. What OTHER services or activities are provided to support the needs of migratory children and youth (including OSY)? [Probe specifically for professional development and parent involvement activities]

    1. In what ways does your district fund or deliver professional development (i.e., for local program staff, school practitioners on ID&R, service delivery, and/or data collection and reporting)? How are professional development needs of staff members determined? How many staff members receive professional development?

       1. Who delivers these services?

       2. What challenges, if any, are associated with delivering these professional development services?
          [Probe for: lack of interest; lack of funding, lack of quality, etc.]

       3. How, if at all, does the LEA address these challenges?

       4. To your knowledge, what benefits have results from these professional development services and activities? How do you know?

    2. In what ways does your LEA fund or deliver parent involvement and education services and activities (e.g., adult education, ESL, GED, etc., or parent training on supporting their children’s educational growth)? How many parents participate in parent involvement and education activities?

       1. Who delivers these services?

       2. What challenges are associated with delivering these services?
          [Probe for: lack of interest; lack of funding, lack of quality, etc.]

       3. How, if at all, does the LEA address these challenges?

       4. What benefits have results from these services and activities?

17. How does the LEA ensure that MEP-funded services and supports are supplementing rather than supplanting services already delivered to migratory students?

18. Looking across the services and supports provided to migrant students, which have been the most effective in meeting migrant student needs? Why? How do you know? [Probe: better outcomes, greater satisfaction or participation, easier implementation, more cost-effective, other reason]?

19. Among the portfolio of services and supports offered to migrant students in your LEA (e.g., academic, non-academic/advocacy, health), which, if any, do think should be added, dropped, or adjusted? Why?

    1. Are there needed services and supports that your LEA lacks the capacity to provide? If so, why?

    2. To what extent do you believe that ESSA will affect the services (e.g., type, frequency, duration, distribution, etc.) provided to eligible migrant students? If ESSA has not yet had an impact on the type and delivery of services and supports for migratory children and youth, do you anticipate that changes will come? Why? How, if at all, do you expect those changes might affect your capacity to meet the needs of eligible migrant children and youth?

20. What types of outreach materials and activities, if any, does your LEA use to encourage migratory children and their families to participate in and/or receive MEP educational, health, nutrition, and social services?

    1. Does the LEA disseminate materials to migrant children and their families to encourage participation? If so, how? If not, why not?

    2. Does the LEA sponsor recruitment activities targeted at migratory children and their families?

       1. What is the purpose of these activities?

       2. How often are they held?

    3. Who at the LEA is responsible for coordinating outreach activities?

    4. How, if at all, does the LEA coordinate outreach efforts with schools/projects?

Services for Migrant Special Populations

21. Which MEP-funded services has your LEA prioritized providing for students who have moved within the year, are at-risk of failing, or dropped out of school, or otherwise have been identified as PFS?
    [Probe for: instructional services including ELA, math, and other supplemental instruction; guidance/ advocacy services; preschool services; preparation for post-secondary transition; staff professional development, health, dental, and eye care; school supplies; clothing; transportation; parent education and support; referrals; staff professional development, etc.]

    1. How did your LEA determine those priorities?

    2. What MEP-funded services does your district provide to students identified as PFS compared with other migrant children and youth who are eligible to receive MEP-funded services and supports but are not identified as PFS?

    3. [For serving OSY]: How and when are services delivered to eligible migrants who are OSY? [Probe for: through what structures/personnel?] What factors dictate when these services are provided to OSY?

22. What specific instructional and support services, if any, has your LEA prioritized providing for older migrant students who are working toward high school completion, post-secondary education, and the workforce?
    [Probe for: Review and revise local attendance and credit accrual policies; professional development to educators and social service providers on the special needs of migratory youth; alternative schooling options, such as individualized instruction, credit recovery, online courses, and providing access to adult education courses; after school opportunities for supplemental instruction and enrichment; health and wellness support, including medical, dental, and mental health services; life skills development, including communications, problem solving, critical thinking, and behavior management skills.]

    1. What factors did your LEA consider when selecting those priority strategies?

    2. How and when are these services to improve high school completion rates provided to eligible migrant students (i.e., year-round, summer, school-year)?

       1. What factors dictate when these services are provided?

Coordination and Collaboration

23. Does your LEA coordinate its administrative responsibilities, and/or program/activities with other departments, agencies, organizations, or programs in the district to address the needs of migrant students? If so, why? If not, why not?

24. [If yes:] With what departments, agencies, organizations, and programs does your LEA coordinate and collaborate?
    [Probe for: Title I, Parts A & D; Title III; Title IV, Part B; Title VI, Part B; Education for Homeless Children and Youth Program; IDEA; Department of Health; Department of Labor; Department of Agriculture; etc.]

    1. What is the purpose of these collaborations? What are the shared goals and objectives among the collaborating agencies, organizations, and programs?

    2. Who is involved in efforts to coordinate and collaborate? What is the frequency of interactions and the topics of communication?

    3. What strategies or processes have supported effective coordination and collaboration between the LEA and other agencies, organizations, and/or programs?

    4. What are the benefits of coordination and collaboration for the MEP and for migrant students? Has it increased the types, frequency, duration, volume, and/or quality of services provided to migrant students?

    5. What are the challenges or drawbacks, if any of coordinating and collaborating with other agencies, organizations, and programs?

25. Does your LEA make referrals to other service providers to directly provide services to migrant students?

    1. [If yes] Describe the portfolio of services offered through referrals.

    2. Are there areas of service (e.g., health care, counseling, etc.) for which your LOA does not currently refer migratory students and/or OSY to other service providers? If so, why?

    3. What gaps, if any, exist in service providers to whom your LEA can refer migratory students and/or OSY?

26. How, if at all, does your LEA coordinate and collaborate with the College Assistance Migrant Program (CAMP), the High School Equivalency Program (HSEP), and other programs funded by the U.S. Office of Migrant Education (OME)?

    1. What is the purpose of these collaborations?

    2. What are the shared goals and objectives?

    3. Who is involved in efforts to coordinate and collaborate?

    4. What is the frequency of interactions and the topics of communication?

    5. What strategies or processes have supported effective coordination and collaboration between LEA and CAMP, HSEP, and/or other OME-funded programs?

    6. What are the benefits of coordination and collaboration with CAMP, HSEP, and/or other OME-funded programs for the MEP and for migrant students? Has it increased the types, frequency, duration, volume, and/or quality of services provided to migrant students?

    7. What are the challenges or drawbacks, if any, of coordinating and collaborating with other OME-funded programs?

Accountability and Data Use

Technical Assistance, Monitoring, and Accountability

27. Has the state provided you with any technical assistance regarding MEP?

    1. If so, what was the nature of the assistance and how was it delivered? Was it helpful?

    2. If not, why not?

28. What other types of supports, if any, has the state MEP provided to you?
    [Probe for: statewide conferences, resource guides, etc.]

    1. To what extent did you find this support helpful?

       1. If not, why not?

29. From what other sources, if any, have you sought or received support in the administration of the MEP?

30. Does the state monitor your MEP-funded program/activities?

    1. If not, why not?

    2. If so, what specific strategies does the state use to monitor your MEP-funded program/activities?

       1. For example, do state administrators conduct in-person monitoring visits to your program? If so, how often?

    3. How, if at all, have state monitoring activities supported your efforts to administer your MEP-funded program?

    4. What changes, for example, have you made in response to monitoring and technical assistance?

31. Has your program been evaluated by an internal or external evaluator? If so, when? [NOTE TO INTERVIEWER: Obtain a copy of evaluation report]

    1. To what extent, if at all, have evaluation findings informed your efforts to administer your MEP-funded program?

Data Collection and Use

32. How are data on migrant students tracked in your state’s data system?

    1. How, if at all, does your office work with the state accountability office regarding migrant student data and formal accountability reporting?

    2. Are migrant students reported as a sub-population in formal LEA reporting?

    3. Which state data on migrant students are the most useful, and how are they useful?

    4. What have been the challenges, if any, associated with complying with state reporting requirements?

    5. What changes, if any, would you like to see in federal and/or accountability system and state reporting requirements related to migrant student data? Why?

    6. Does your LEA collect additional data that are not required by the state?

       1. Please describe the data and why they are collected.

33. Do you use outcomes data to inform ongoing decisions about changing, continuing, or discontinuing MEP-funded services and supports for eligible migrant students and OSY?

    1. [If yes:] What specific types of outcomes data are used to inform your decision making regarding the delivery of services and supports? For example, do you look at student outcomes data—academic (academic performance, graduation, grades, course completion) and non-academic (health, job attainment, housing)—to inform your decision making?

       1. Which outcomes data are most useful for purposes of informing local goals and objectives? Why?

       2. What is the process for reviewing these outcomes data for decision-making purposes, and how frequently does that review occur?

    2. What program decisions were made based on outcomes data?
       [If no:] Are there challenges or other reasons that inhibit the use of outcomes data or other evidence for program decision-making?

    3. What service-level data do you track, if any? How are those data tracked and how are they used when reviewing program outcomes?

34. What data sources, if any, does your LEA draw on to make decisions about the effectiveness of MEP services and activities for your eligible migrant student population?
    [Probe for: LEA student performance, enrollment, graduation data; participant/parent/staff surveys; individual student plans; program data, MEP participant/end-of-project reports]