Study of the Title III Native American and Alaska Native Children in School (NAM) Program

JANUARY 2015

Task Order 24: Study of the Title III Native American and Alaska Native Children in School (NAM) Program

Office of Management and Budget Clearance Request

Appendix A: Interview and Focus Group Protocols

Prepared for

U.S. Department of Education

Office of Planning, Evaluation and Policy Development

Policy and Program Studies Service

By

American Institutes for Research

June 29, 2016

Appendix A. Interview and Focus Group Protocols

This section includes interview and focus group protocols for all potential NAM (Native American and Alaska Native Children in School) grant stakeholders. However, each NAM grant is unique, with different configurations of stakeholders, so some protocols may not be used in every site (for example, only 16 of the 22 grant sites have separate professional development providers). The study team will identify all potential respondents in advance of each site visit.

This section includes the following interview and focus group protocols:

• NAM grant coordinator telephone interview protocol

• Tribal education director interview protocol

• District superintendent (or administrator of Bureau of Indian Education schools) interview protocol

• School principal interview or focus group protocol

• English-as-a-second-language teacher or program instructional staff focus group protocol

• Tribal college instructor focus group protocol

• Parent or family coordinator interview protocol

• Professional development provider focus group protocol

• Native American/Alaska Native language instruction or curriculum development specialist interview protocol

• Program administrator or evaluator interview protocol

To reduce the burden on respondents and to invite rich discussion and descriptions, the protocols have broad, open-ended questions. However, to capture all relevant information, a column on the right side of the protocol reminds the interviewer which information or details are critical and what to probe for. Text in the front matter of each protocol also reminds the interviewer about the most critical pieces of information to learn from each participant. The exception is the grant coordinator protocol, which is more similar to a traditional protocol due to the amount of detailed information necessary to gather from these individuals.

NAM grant coordinator telephone interview protocol

Grantee:	Interviewer:
Interviewee:	Date/Time:

Introduction

Thank you for taking the time to speak with me today. Just as a reminder, this interview is for a study of the Title III Native American and Alaska Native Children in School (NAM) Program, which is funded by the U.S. Department of Education. The Department is interested in learning how grantees have used the NAM program to support Native American and Alaska Native (NA/AN) academic English achievement and NA/AN language and culture revitalization, including how you prioritize use of funds, how you measure progress, and the challenges and lessons you’ve learned in providing services. The questions will pertain to your role in the program, the program’s history and goals, your partners, and the structure and content of the program.

Please know that, as with all federal studies, confidentiality of the information you provide will be protected to the extent provided by law. We will not associate you or your institution’s name with the data we collect, and we will protect your identity in our reports by summarizing findings across groups of grantees and masking any individual examples. We do this so that everyone feels comfortable in sharing their experiences regarding the planning and implementation of the NAM program. We will acknowledge your grant program in the final report, but we will do so separately from the information you provide.

We also would like to ask you permission to audio record this interview to ensure we are accurately capturing your feedback. Only the research team at American Institutes for Research will review the audio files and the transcripts. They will not be shared with any other personnel. At the end of this project, all interviews—audio files and transcripts—will be destroyed. Are you comfortable with being recorded?

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form, answer any questions, and have the respondent sign the form.]

We know that you are very busy, and we want to be respectful about your time, so we’ve made an effort to collect as much information as possible through available documents. We’ll sometimes reference that information during the course of the interview to confirm that our information is correct. This interview will take about 90 minutes. Do you have any questions before we begin?

Role of the Respondent

Most questions will be about the activities you have undertaken with funding specifically from the NAM program. [Clarify the name of the program and how other people will identify it.]

1. [Only ask these questions if the grant coordinator is not the same person as on the original application.] Could you tell us how long you’ve been involved in the [insert name of project] as grant coordinator? How did you come into this role?

2. Please briefly describe the activities you do in support of the NAM-funded program.

History and Context of Program

3. Please share the vision that guided the development of [insert name of project]. Who in the community was involved? What needs were identified that contributed to the development of the program?

4. Could you share tribal values and other tribal program priorities that align with this grant?

5. What are the goals and objectives of the program? Have these changed over time (e.g., across the grant cycles)?

6. From your perspective, are some goals greater priorities than others? Why?

Implementation for Students

7. Now I want to talk about the students this program serves. What is the target population of students in the program? (Examples: grade or age level; certain level of English proficiency; certain level of heritage language proficiency; low achieving academically)

8. What are the main components of [insert name of project] for students?

Are you supporting...	If yes...
Academic English	What activities do you implement to support students’ academic English?
NA/AN language instruction	What activities do you implement to support NA/AN language acquisition? What are the goals of [insert name of project] for students (e.g., maintain the students’ native language, introduce the students’ heritage indigenous language, tribal language revitalization)? What is the tribal community’s role in the language-learning program? [Probe: instructor selection or certification, material development, other resources provided]
Technology	What technology resources does the program have access to? What technology do you incorporate? Who uses the technology, and how do they use it?
Early childhood learning	Is there an early childhood program in the community as part of [insert name of project]? What program activities are in place for young children?
Data-based decision making	How are data used to support students’ academic achievement? What changes have been made to the school/district’s data-based decision making as a result of the grant?

Structure and Content of the Program for Teachers

9. [If the program supports educators] Which teachers, administrators, and instructional staff are targeted to receive training that is funded by the grant?

Prompts: classroom teachers only; all teachers; non-instructional staff (such as paraprofessionals, instructional support staff), administrators, counselors

10. Which of these content areas are taught in the educators’ trainings? What is the primary focus?

1. Linguistics and second language learning

2. Teaching English learners

3. Curriculum and assessment for English learners

4. Strategies to build language skills and subject area knowledge (e.g., how to incorporate language and literacy skills into content such as math, science, social studies)

5. Data-based decision making

6. Tribal language instruction

7. Curriculum development

8. Culture heritage and cultural sensitivity

9. Improving school climate and promoting indigenous culture and language

10. Other

11. How do the teachers receive training or professional development as part of the program?

Prompts: professional learning communities (PLCs); class observations or visits followed by a discussion group with peers; mentoring or coaching; co-teaching or collaboration (e.g., between bilingual education, ESL or language acquisition specialists and mainstream teachers); language or cultural immersion

12. Is the training required or optional?

    1. If required, how many hours per school year are required?

13. What is the role of the tribe or tribal government with respect to the training or professional development of educators?

Structure and Content of the Program for Families and Community

14. Who are the partners in the program? What do they do, and how did they get involved?

Prompts: school, district, tribal community, state agency, other entities

15. [If not included as a partner] What is the role of the tribe (or tribal government) in:

    1. Planning the program?

    2. Implementing the program?

    3. Evaluating the program?

    4. Helping the program to succeed?

16. [If not already mentioned] Please share any family engagement strategies that you incorporate into the program.

Grant Outcomes

17. In what way has this program contributed to the following groups of people? How do you know?

    1. Students:

• Have you witnessed any change in their English language proficiency?

• Have you witnessed any change in their NA/AN language proficiency?

• Have you witnessed any change in their engagement in language instruction and other education programs (e.g., afterschool programs, other academic initiatives)?

2. Educators

3. School

4. Tribal community

5. Families and the community

6. Partners through the collaboration in this program

18. In your application, you identified [outcome from application] as a key outcome for this program. What changes have you seen in [outcome from application]?

19. What measures have you used to track progress and outcomes for this program?

Challenges, Successes, and Sustainability

20. Please share the challenges you faced in the planning and implementation of [insert name of project]. How did these challenges affect your ability to carry out your initial plans?

21. Please share which people or community entities have assisted you to decrease or overcome these challenges. What additional support could you use (e.g., training and technical assistance, information technology, capacity building)?

22. What strategies have you developed in your role as the grant coordinator to overcome these challenges? Are there lessons you have learned that you would be willing to share?

23. What benefits for the students, teachers, or the community have you seen as a result of [insert name of project]? What in particular has worked out well? [Probe for specific activities that have worked well.]

24. What do you think were the factors that contributed to these benefits?

25. What types of promising practices have been established as a result of [insert name of project]? Which activities are planned to be made sustainable and maintained for the future? [Probe about: (1) to what extent community engagement and parental involvement contributed to sustainability and (2) current challenges that might impede ongoing efforts and how strategies for overcoming challenges might contribute to sustainability.]

[For 2011 cohort] After the conclusion of your 2011 grant, what specific elements or activities of [insert name of project] have remained in place?

26. Your application indicates that [number] people receive partial funding through the NAM grant. How did the NAM funding change their work overall (e.g., position expanded, complete change in work, no change)?

Wrap-Up

27. Can you please recommend anyone who we should speak to and any particular schools (or entities) to visit and observe?

28. Is there anything else you would like to tell us about your program?

Tribal education director interview protocol

Grantee:	Interviewer:
Interviewee(s):	Date/Time:
Interviewee(s’) Role(s): (select all that apply)
ᴏ Tribal education director or assistant director
ᴏ Other ____________________________________

Information for the interviewer:

Programs funded by the Native American and Alaska Native Children in School (NAM) Program occur in the context of other tribal activities and priorities for Native American and Alaska Native (NA/AN) education. All NAM grants serve students from one or more NA/AN tribes, and every tribe represented by the program has its own education department and services. We want to better understand educational priorities from a tribal perspective.

PAY ATTENTION TO:

• The types of professional development, curricula, and materials made available by the tribe for academic English and NA/AN language instruction (RQ1)

• The types of academic programming provided for students and how these services are delivered, especially academic English and NA/AN language instruction (RQ1)

• Other tribal policies, regulations, and priorities that might affect NAM implementation (RQ3)

• Descriptions of tribal relationships with school agencies and non-tribal grantees (RQ4)

• Challenges encountered with any of the above (RQ2)

• Participants’ perceptions of outcomes for students and families (RQ5)

By the end of this interview, you should have details and examples for each of these areas.

Introduction

Thank you for taking the time to speak with me today. Just as a reminder, this interview is for a study of the Title III Native American and Alaska Native Children in School (NAM) Program, which is funded by the U.S. Department of Education. The Department is interested in learning how grantees have used the NAM program to support Native American and Alaska Native (NA/AN) academic English achievement and NA/AN language and culture revitalization. The questions will pertain to your role in the program, the program’s history and goals, and the structure and content of the program.

Please know that, as with all federal studies, confidentiality of the information you provide will be protected to the extent provided by law. We will not associate you or your institution’s name with the data we collect, and we will protect your identity in our reports by summarizing findings across groups of grantees and masking any individual examples. We do this so that everyone feels comfortable in sharing their experiences regarding the planning and implementation of the NAM program. We will acknowledge your grant program in the final report, but we will do so separately from the information you provide.

We also would like to ask you permission to audio record this interview to ensure we are accurately capturing your feedback. Only the research team at American Institutes for Research will review the audio files and the transcripts. They will not be shared with any other personnel. At the end of this project, all interviews—audio files and transcripts—will be destroyed. Are you comfortable with being recorded?

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form, answer any questions, and have the respondent sign the form.]

We know that you are very busy, and we want to be respectful about your time, so we’ve made an effort to collect as much information as possible through available documents. We’ll sometimes reference that information during the course of the interview to confirm that our information is correct. This interview will take about 60 minutes.

Do you have any questions before we begin?

Role of Respondent

Question	Look For
1. How long have you been involved in the [insert name of project]? How long has your tribe been involved in the program? Please briefly describe the activities you do in support of the program.	Length of time involved in NAM program

If interviewee(s) helped plan the grant:

1a. Please share the tribe’s vision that guided the development of the grant.

1b. Which sectors of the community were involved?

1c. What strengths and needs of the community were identified that contributed to the development of the program?

Contextual Information

Question	Look For
2. What are the goals and objectives of the program? Have these changed over time (e.g., across the grant cycles)?	Alignment with other stakeholders’ goals and priorities for their grant cycle(s)
3. From your perspective regarding the needs of children in your tribe, are some goals greater priorities than others? If so, why?	Focus of the grant for the tribe, alignment of the grant with tribal goals

Grant Services

Question	Look For
4. How are the students identified to receive services funded by the grant?	Which students are targeted for services and how the students are grouped to receive services
5. What services do students receive through the grant? What is the tribe’s role in providing these services?	Which activities the grantee is implementing and details about: Academic English NA/AN language instruction Technology Early childhood learning Afterschool services that take place within the tribe Data-based decision making

Professional Development

Question

Look For

6. In what areas, if any, do tribal staff receive training through [insert name of project]? How are these areas related to tribal priorities for training? 7. [If the tribe is the grantee] What types of training do tribal staff provide through [insert name of project]?

Tribal knowledge or involvement in professional development and training for teachers, administrators, and instructional staff: Who partakes Structure of the trainings (e.g., workshops, seminars, classes, teach-the-trainers, PLCs, peer coaching or mentoring) Content: Linguistics and second language acquisition Teaching English learners Curriculum and assessment for English learners Strategies to build language skills and subject area knowledge (e.g., how to incorporate language and literacy skills into content such as mathematics, science, social studies) Tribal language instruction Curriculum development Culture heritage and cultural sensitivity Data-based decision making Improving school climate and promoting indigenous culture and language Tribal involvement in any training FOCUS ON the priority areas for each grantee.

Family Engagement

Question	Look For
8. Please share any family engagement strategies that you’re aware are supported by the [insert name of project]. How is the tribe involved?	Tribal knowledge of or involvement in family engagement activities

Partners

Question	Look For
9. Who are the partners in the program? What do they do?	The tribe’s perspective on who the partners are, especially school or district partners, and what their roles are in terms of planning, delivery, and management or oversight (e.g., providing materials, selecting or certifying language teachers)
10. How does the tribe communicate and work with the partners?	Relationship between the tribal community and the district
11. [If the tribe is the grantee] Who is responsible for the federal reporting requirements?

Challenges, Successes, and Sustainability

Question	Look For
12. [If the tribe is the grantee] How do you prioritize the use of the funding?	Priorities for the funding and use of supplemental funding (e.g., other grant money)
13. What benefits for the students, teachers, or the community have you seen as a result of [insert name of project]? What in particular has worked out well?	Level of family and community participation Level of student engagement Factors contributing to success
14. Please share the challenges you faced in the planning and implementation of the [insert name of project] in your tribe. How did those challenges affect your ability to carry out your initial plans? 15. What strategies have you developed in your role as the tribal education director to overcome these challenges? Are there lessons you have learned that you would be willing to share? 16. What additional supports do you draw on or could you use?	EXAMPLES
17. What types of promising practices have been established as a result of [insert name of project]? Which activities are planned to be made sustainable and maintained for the future? [For the 2011 cohort] After the conclusion of your 2011 grant, which elements or activities of [insert name of project] have remained in place?	How challenges impede ongoing efforts and how strategies for overcoming challenges contribute to sustainability To what extent community engagement and parental involvement have contributed to sustainability

Grant Outcomes

Question	Look For
18. What has changed for the students and community as a result of [insert name of project]?	Level of family and community participation Level of student engagement Educator involvement and engagement Family involvement and engagement
19. The NAM grant application identified [outcome from application] as a key outcome for this program. What changes have you seen in [outcome from application]?
20. [If the district is the grantee] What measures do you use to track progress and outcomes for this program?

Wrap-Up

21. Is there anything else you would like to tell us about your program?

District superintendent (or administrator of Bureau of Indian Education schools) interview protocol

Grantee:	Interviewer:
Interviewee(s):	Date/Time:
Interviewee(s’) Role(s): (select all that apply)
ᴏ District superintendent Administrator of Bureau of Indian Education schools
ᴏ Other ____________________________________

Information for the interviewer:

Programs funded by the Native American and Alaska Native Children in School (NAM) Program occur in the context of other district (or Bureau of Indian Education [BIE]) activities and priorities for Native American and Alaska Native (NA/AN) education. In addition, each tribe may have a different type of relationship with districts (or BIE), which could also affect NAM implementation. We want to better understand these priorities and relationships from a local education agency perspective. We also want to understand the role that the district (or BIE) plays in providing or facilitating NAM services.

PAY ATTENTION TO:

• The types of professional development, curricula, and materials made available by the district (or BIE administrator) for academic English and NA/AN language instruction (RQ1)

• The types of academic programming provided for students and how it is delivered, especially NA/AN language instruction, early childhood education, and technology (RQ1)

• Other district (or BIE) policies, regulations, and priorities that might affect NAM implementation (RQ3)

• Descriptions of tribal relationships with district (or BIE) education agencies (RQ4)

• Challenges encountered with any of the above (RQ2)

• Participants’ perceptions of outcomes for students and families (RQ5)

By the end of this interview, you should have details and examples for each of these areas.

Introduction

Thank you for taking the time to speak with me today. Just as a reminder, this interview is for a study of the Title III Native American and Alaska Native Children in School (NAM) Program, which is funded by the U.S. Department of Education. The Department is interested in learning how grantees have used the NAM program to support Native American and Alaska Native (NA/AN) academic English achievement and NA/AN language and culture revitalization. The questions will pertain to your role in the program, the program’s history and goals, and the structure and content of the program.

Please know that, as with all federal studies, confidentiality of the information you provide will be protected to the extent provided by law. We will not associate you or your institution’s name with the data we collect, and we will protect your identity in our reports by summarizing findings across groups of grantees and masking any individual examples. We do this so that everyone feels comfortable in sharing their experiences regarding the planning and implementation of the NAM program. We will acknowledge your grant program in the final report, but we will do so separately from the information you provide.

We also would like to ask you permission to audio record this interview to ensure we are accurately capturing your feedback. Only the research team at American Institutes for Research will review the audio files and the transcripts. They will not be shared with any other personnel. At the end of this project, all interviews—audio files and transcripts—will be destroyed. Are you comfortable with being recorded?

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form, answer any questions, and have the respondent sign the form.]

We know that you are very busy, and we want to be respectful about your time, so we’ve made an effort to collect as much information as possible through available documents. We’ll sometimes reference that information during the course of the interview to confirm that our information is correct. This interview will take about 45 minutes.

Do you have any questions before we begin?

Role of Respondent

Question	Look For
1. How long have you been involved in the [insert name of project]? [If not known] How long has your district [or the BIE] been involved in the program?	Length of time involved in NAM program

If interviewee(s) helped plan the grant:

1a. What were the needs that led to the development of the program?

Contextual Information

Question	Look For
2. What are the goals and objectives of the program? Have these changed over time (e.g., across the grant cycles)?	Alignment with other stakeholders’ goals and priorities for their grant cycle(s)
3. From your perspective, are some goals greater priorities than others? If so, why?	Focus of the grant for the district [or BIE], alignment of the grant with district [or BIE] goals

Grant Services

Question	Look For
4. Now I want to talk about the students this program serves. What is the target population of students in the program?	Which students are targeted for services and how the students are grouped to receive services
5. What services do students in your district [or the BIE] receive through the grant? What is the district’s [or the BIE’s] role?	Which activities the grantee is implementing and details about: Academic English NA/AN language instruction Technology Early childhood learning Afterschool services that take place within the district [or the BIE] Data-based decision making

Professional Development

Question	Look For
6. How do teachers, administrators, or instructional staff receive training through the [insert name of project]?	District [or BIE] knowledge or involvement in professional development and training for teachers, administrators, and instructional staff: Who partakes Content and structure of the trainings (e.g., workshops, seminars, classes, teach-the-trainers, PLCs, peer coaching or mentoring) Tribal involvement in any training

Family Engagement

Question	Look For
7. Please share any family engagement strategies that you’re aware of as part of the [insert name of project].	District [or BIE] knowledge of or involvement in family engagement activities

Partners

Question	Look For
8. Who are the partners in the program? What do they do?	The district’s [or the BIE’s] perspective on who the partners are, especially tribal partners, and what their roles are in terms of planning, delivery, and management or oversight (e.g., providing materials, selecting or certifying language teachers)
9. How does the district [or the BIE] communicate and work with the partners?	Relationship between the tribal community and the district [or the BIE]
10. [If the district or the BIE is the grantee] Who takes care of the federal reporting requirements?

Challenges and Successes

Question	Look For
11. [If the district or the BIE is the grantee] Is the grant funding level appropriate for the services you provide? How do you prioritize the use of the funding?	Priorities for the funding and use of supplemental funding (e.g., other grant money)
12. What benefits for the students, teachers, or the community have you seen as a result of [insert name of project]? What in particular has worked out well?	Level of family and community participation Level of student engagement Factors contributing to success
13. Please share the challenges you faced in the planning and implementation of the [insert name of project] in your district [or the BIE]. 14. What strategies have you developed in your role as the district superintendent [or BIE administrator] to overcome these challenges? Are there lessons you have learned that you would be willing to share? 15. What additional supports do you draw on or could you use?	EXAMPLES
16. What types of promising practices have been established as a result of [insert name of project]? Which activities are planned to be made sustainable and maintained for the future? [For 2011 cohort] After the conclusion of your 2011 grant, which elements or activities of [insert name of project] have remained in place?	How challenges impede ongoing efforts and how strategies for overcoming challenges contribute to sustainability Whether staff who are partially funded by the grant would continue in their positions To what extent community engagement and parental involvement have contributed to sustainability

Grant Outcomes

Question	Look For
17. How do you think the [insert name of project] contributed to the following people? How do you know? Students: Have you witnessed any change in their English language proficiency? Have you witnessed any change in their NA/AN language proficiency? Have you witnessed any change in their engagement in language instruction and other education programs? Educators in your district [or the BIE] Families and the community	Level of family and community participation Level of student engagement Educator involvement and engagement Family involvement and engagement
18. The NAM grant application identified [outcome from application] as a key outcome for this program. What changes have you seen in [outcome from application]?
19. [If the district or the BIE is the grantee] What measures do you use to track progress and outcomes for this program?

Wrap-Up

20. Is there anything else you would like to tell us about your program?

School Principal interview or Focus group protocol

Grantee:	Interviewer:
Interviewee(s):	Date/Time:
Interviewee(s’) Role(s): (select all that apply)
ᴏ Principal
ᴏ Vice principal or assistant principal
ᴏ Other ____________________________________

Information for the interviewer:

Programs funded by the Native American and Alaska Native Children in School (NAM) Program occur in the context of other school activities and priorities for Native American and Alaska Native (NA/AN) education. In addition, each tribe may have a different type of relationship with schools (some of which actually are tribal schools), which could also affect NAM implementation. We want to better understand these priorities and relationships from a school administrator perspective. We also want to understand the role that the school plays in providing or facilitating NAM services.

PAY ATTENTION TO:

• The types of professional development, curricula, and materials made available by the school for academic English and NA/AN language instruction (RQ1)

• The types of academic programming provided for students and how it is delivered, especially NA/AN language instruction, early childhood education, and technology (RQ1)

• Other school or district policies, regulations, and priorities that might affect NAM implementation (RQ3)

• Descriptions of tribal relationships with the school (RQ4)

• Challenges encountered with any of the above (RQ2)

• Participants’ perceptions of outcomes for students and families (RQ5)

By the end of this interview, you should have details and examples for each of these areas.

Introduction

Thank you for taking the time to speak with me today. Just as a reminder, this interview/focus group is for a study of the Title III Native American and Alaska Native Children in School (NAM) Program, which is funded by the U.S. Department of Education. The Department is interested in learning how grantees have used the NAM program to support Native American and Alaska Native (NA/AN) academic English achievement and NA/AN language and culture revitalization. The questions will pertain to your role in the program, the program’s history and goals, and the structure and content of the program.

Please know that, as with all federal studies, confidentiality of the information you provide will be protected to the extent provided by law. We will not associate you or your institution’s name with the data we collect, and we will protect your identity in our reports by summarizing findings across groups of grantees and masking any individual examples. We do this so that everyone feels comfortable in sharing their experiences regarding the planning and implementation of the NAM program. We will acknowledge your grant program in the final report, but we will do so separately from the information you provide.

We also would like to ask you permission to audio record this interview/focus group to ensure we are accurately capturing your feedback. Only the research team at American Institutes for Research will review the audio files and the transcripts. They will not be shared with any other personnel. At the end of this project, all interviews and focus groups—audio files and transcripts—will be destroyed. Are you comfortable with being recorded?

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form, answer any questions, and have the respondent sign the form.]

We know that you are very busy, and we want to be respectful about your time, so we’ve made an effort to collect as much information as possible through available documents. We’ll sometimes reference that information during the course of the interview/focus group to confirm that our information is correct. This interview/focus group will take about 60 minutes.

Do you have any questions before we begin?

Role of Respondent

Question	Look For
1. How long have you been involved in the [insert name of project]? How long has your school been involved in the program?	Length of time involved in NAM program

If interviewee(s) helped plan the grant:

1a. What were the needs that led to the development of the program?

Contextual Information

Question	Look For
2. What are the goals and objectives of the program? Have these changed over time?	Alignment with other stakeholders’ goals and priorities for their grant cycle(s)
3. From your perspective, are some goals greater priorities than others? If so, why?	Focus of the grant for the school, alignment of the grant with school goals

Grant Services

Question	Look For
4. What is the target population of students served through the program?	Which students are targeted for services and how the students are grouped to receive services
5. What are the main components of the [insert name of project] for students in your school?	Which activities the grantee is implementing and details about: Academic English NA/AN language instruction Technology Early childhood learning Afterschool services that take place within the school Data-based decision making

Professional Development

Question

Look For

6. How do teachers, administrators, or instructional staff receive training through the [insert name of project]? 7. Which of these content areas are taught in the educators’ trainings? What is the primary focus? Linguistics and second language learning Teaching English learners Curriculum and assessment for English learners Strategies to build language skills and subject area knowledge (e.g., how to incorporate language and literacy skills into content such as math, science, social studies) Tribal language instruction Curriculum development Culture heritage and cultural sensitivity Data-based decision making Improving school climate and promoting indigenous culture and language 8. Is your school involved in any of the training delivery? If so, how (e.g., peer teaching, peer mentoring, coaching, PLCs)? 9. Is the training required or optional? 10. Does anyone in the tribe or tribal government provide training to teachers or administrators?

This is one of the most important questions for principals. Find out about: Who partakes Content and structure of the trainings (e.g., workshops, seminars, classes, teach-the-trainers, PLCs, peer coaching or mentoring) Tribal involvement in any training

Family Engagement

Question	Look For
11. Please share any family engagement strategies that are part of the [insert name of project].	How families are involved Communication methods with families

Partners

Question	Look For
12. Who are the partners in the program? What are their roles?	The principal’s perspective on who the partners are, especially tribal partners, and what their roles are in terms of planning, delivery, and management or oversight (e.g., providing materials, selecting or certifying language teachers)
13. How does the school communicate and work with the partners?	Relationship between the tribal community and the school

Challenges, Successes, and Sustainability

Question	Look For
14. What benefits for the students, teachers, or the community have you seen as a result of [insert name of project]? What in particular has worked out well?	Level of family and community participation Level of student engagement Factors contributing to success
15. Please share the challenges you faced in providing (or receiving) services for teachers and students through [insert name of project]. 16. What strategies have you developed in your role as the principal to overcome these challenges? Are there lessons you have learned that you would be willing to share? 17. What additional supports could you use?	EXAMPLES
18. What types of promising practices have been established as a result of [insert name of project]? Which activities are planned to be made sustainable and maintained for the future? [For 2011 cohort] After the conclusion of your 2011 grant, which elements or activities of [insert name of project] have remained in place?	How challenges impede ongoing efforts and how strategies for overcoming challenges contribute to sustainability Whether staff who are partially funded by the grant would continue in their positions To what extent community engagement and parental involvement have contributed to sustainability

Grant Outcomes

Question	Look For
19. How do you think the [insert name of project] contributes to the following people? How do you know? Students: Have you witnessed any change in their English language proficiency? Have you witnessed any change in their NA/AN language proficiency? Have you witnessed any change in their engagement in language instruction and other education programs? Educators in your district Families and the community	Level of family and community participation Level of student engagement Educator involvement and engagement Family involvement and engagement
20. The NAM grant application identified [outcome from application] as a key outcome for this program. What changes have you seen in [outcome from application]?

Wrap-Up

21. Is there anything else you would like to tell us about your program?

English-as-a-second-language Teacher or program instructional staff focus group protocol

Grantee:			Interviewer:
Interviewee(s):			Date/Time:
Interviewee(s’) Role(s): (select all that apply)
ᴏ ELA teacher	ᴏ ESL or bilingual teacher	ᴏ Reading/literacy coach
ᴏ ELA coordinator	ᴏ ESL or bilingual coordinator	ᴏ Curriculum developer
ᴏ Other ____________________________________

Information for the interviewer:

One of the primary goals of the Native American and Alaska Native Children in School (NAM) Program is to improve students’ academic English. The students might be English language learners (ELLs) because they speak another language or another dialect of English at home. The students might not be ELLs but nonetheless have low academic English literacy.

There are a lot of different ways to improve students’ academic English, and we want to know what the NAM grantees are doing. We want to know who is providing services, how the services are structured, what the services are, any challenges the grantees have encountered, and how the grantees are measuring progress.

You might be interviewing stakeholders who work directly with students, who primarily work with other classroom teachers to train them how to provide instruction, who develop curricula and materials, or who perform a mix of these activities. The interviewees might be affiliated with the tribe, with the school, or with both.

PAY ATTENTION TO:

• How the program is structured to provide academic English services (RQ1)

  • Does someone provide professional development to regular classroom teachers, who then provide services to students? What does the professional development look like? Does the professional development serve teachers’ needs?

  • What do the services look like?

  • Does someone develop specialized materials or curricula for students?

• How students are selected to receive academic English services (RQ1)

• If anyone is using technology to provide academic English services (RQ1)

• If anyone is involving family or community members in serving students (RQ1)

• How the tribe is involved in providing services (RQ4)

• Challenges encountered with any of the above (RQ2)

• How the grantee knows if the academic English services are working (RQ5)

By the end of this interview, you should have details and examples for each of these areas.

Introduction

Thank you for taking the time to speak with me today. Just as a reminder, this focus group is for a study of the Title III Native American and Alaska Native Children in School (NAM) Program, which is funded by the U.S. Department of Education. The Department is interested in learning how grantees have used the NAM program to support Native American and Alaska Native (NA/AN) academic English achievement and NA/AN language and culture revitalization. The questions will pertain to your role in the program, the program’s history and goals, and the structure and content of the program.

Please know that, as with all federal studies, confidentiality of the information you provide will be protected to the extent provided by law. We will not associate you or your institution’s name with the data we collect, and we will protect your identity in our reports by summarizing findings across groups of grantees and masking any individual examples. We do this so that everyone feels comfortable in sharing their experiences regarding the planning and implementation of the NAM program. We will acknowledge your grant program in the final report, but we will do so separately from the information you provide.

We also would like to ask you permission to audio record this focus group to ensure we are accurately capturing your feedback. Only the research team at American Institutes for Research will review the audio files and the transcripts. They will not be shared with any other personnel. At the end of this project, all interviews and focus groups—audio files and transcripts—will be destroyed. Are you comfortable with being recorded?

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form, answer any questions, and have the respondent sign the form.]

We know that you are very busy, and we want to be respectful about your time, so we’ve made an effort to collect as much information as possible through available documents. We’ll sometimes reference that information during the course of the focus group to confirm that our information is correct. This focus group will take about 60 minutes.

Do you have any questions before we begin?

Role of Respondent

Question	Look For
1. Can you please share what your role is, both your professional role and your role in the tribe and community?	Specific title(s)
2. When and how did you first become aware of this grant?	Any involvement in planning the grant

If interviewee(s) helped plan the grant:

2a. What were the needs that led to the development of the program?

Contextual Information

Question	Look For
3. What are the goals and objectives of the [insert name of project] at your school? Have the goals changed?	Any changes in goals aligned with changes to the grant priorities
4. From your perspective, are some goals greater priorities than others? If so, why?	Any goals that are especially aligned with school, district, or tribal priorities

Grant Services

Question	Look For
5. Can you please tell me about the students who are served by the [insert name of project]?	Which students are targeted for services and how the students are grouped to receive services
6. Please describe the entrance and exit measures and criteria for your students served by [insert name of project].	English assessment testing, such as WIDA, ELFA, LAS-Links
7. Please describe the type of English language services you have implemented through [insert name of project]. Are these services the same for all students?	Possible answers: ESL Content-based ESL Two-way or dual language immersion bilingual education Structured English immersion Transitional bilingual education Bilingual heritage language instruction Sheltered instruction (SIOP)
8. What are the main components of the English language services you or other teachers provide through [insert name of project]?	Number of hours a day/week Community and tribal involvement Use of technology
9. What is your role in providing these services, and who else provides academic English services to students through the [insert name of project]?
10. What do you use as curriculum?	Whether curriculum is commercial or developed by the program

If curricula or materials are developed by the program:

10a. Who develops the curricula or materials?

10b. What is the focus of the curricula or materials?

10c. What support do you receive for curriculum or material development?

11. What is the goal of the language services provided through the [insert name of project]?	Improve academic English Improve literacy Learn and maintain an NA/AN language
12. What other academic English supports do students receive?	Supports provided through other school or community programs

Professional Development

Question	Look For
13. What are the greatest professional development needs of teachers in the school or district?	Providing instruction to ELLs Cultural and linguistic sensitivity
14. Please describe any professional development that has been provided through the grant.	Content of the trainings: Linguistics and second language acquisition Curriculum and assessment for language learners Strategies to build language skills and subject area knowledge (e.g., how to incorporate language and literacy skills into content) Tribal language instruction Curriculum development Culture heritage and cultural sensitivity Data-based decision making Improving school climate and promoting indigenous culture and language Structure of the trainings (e.g., workshops, PLCs, coaching or mentoring, online courses) Who received the training Who provided the training Any tribal role in the training

If the interviewee(s) provided training or coaching:

14a. How receptive were participants to the training or coaching?

14b. What challenges did you face, and how did you overcome them?

14c. What supports did you receive, and how helpful were they?

Family Engagement

Question	Look For
15. Does the program have a focus on family engagement? What activities do you do for family engagement?	How families are involved Communication methods with families

Partners

Question	Look For
16. Who are the partners in the program? What do they do, and what are your interactions with them?	Who the partners are and what their roles are in terms of planning, delivery, and management or oversight
17. What is the role of the tribe or tribal government?	Relationship between the tribal community and the school

Challenges and Successes

Question	Look For
18. What supports have you received, and how helpful were they?	Support from: OELA/Department of Education Tribal entities School or school district State agencies
19. What benefits for the students, teachers, or the community have you seen as a result of [insert name of project]? What in particular has worked out well?	Level of family and community participation Level of student engagement Factors contributing to success
20. What challenges do you face in implementing the [insert name of project], and how have you overcome these challenges?	EXAMPLES
21. What lessons have you learned that you would share with others?

Wrap-Up

22. Is there anything else you would like to tell us about your program?

Tribal college instructor focus group protocol

Grantee:		Interviewer:
Interviewee(s):		Date/Time:
Interviewee(s’) Role(s): (select all that apply)
ᴏ ELA instructor	ᴏ ESL or bilingual instructor		ᴏ Native language instructor	ᴏ Tutor
ᴏ PD provider	ᴏ Instructional coordinators		ᴏ Other _______________

Information for the interviewer:

Some Native American and Alaska Native Children in School (NAM) Program grantees are tribal colleges that serve Native American and Alaska Native (NA/AN) college students or that conduct bridge activities for high school seniors. In some cases, tribal colleges serve as partners to NAM grantees. We want to understand how colleges serve NA/AN students, including curricula, materials, instruction, and professional development opportunities for instructors.

PAY ATTENTION TO:

• The types of professional development, curricula, and materials made available by the college for academic English and NA/AN language instruction (RQ1)

• The types of academic programming provided for students and how it is delivered, especially academic English and NA/AN language instruction (RQ1)

• Descriptions of tribal relationships with college grantees (RQ4)

• Challenges encountered with any of the above (RQ2)

• Participants’ perceptions of outcomes for students and families (RQ5)

By the end of this interview, you should have details and examples for each of these areas.

Introduction

Thank you for taking the time to speak with me today. Just as a reminder, this interview is for a study of the Title III Native American and Alaska Native Children in School (NAM) Program, which is funded by the U.S. Department of Education. The Department is interested in learning how grantees have used the NAM program to support Native American and Alaska Native (NA/AN) academic English achievement and NA/AN language and culture revitalization. The questions will pertain to your role in the program, the program’s history and goals, and the structure and content of the program.

Please know that, as with all federal studies, confidentiality of the information you provide will be protected to the extent provided by law. We will not associate you or your institution’s name with the data we collect, and we will protect your identity in our reports by summarizing findings across groups of grantees and masking any individual examples. We do this so that everyone feels comfortable in sharing their experiences regarding the planning and implementation of the NAM program. We will acknowledge your grant program in the final report, but we will do so separately from the information you provide.

We also would like to ask you permission to audio record this interview to ensure we are accurately capturing your feedback. Only the research team at American Institutes for Research will review the audio files and the transcripts. They will not be shared with any other personnel. At the end of this project, all interviews—audio files and transcripts—will be destroyed. Are you comfortable with being recorded?

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form, answer any questions, and have the respondent sign the form.]

We know that you are very busy, and we want to be respectful about your time, so we’ve made an effort to collect as much information as possible through available documents. We’ll sometimes reference that information during the course of the interview to confirm that our information is correct. This interview will take about 60 minutes.

Do you have any questions before we begin?

Role of Respondent

Question	Look For
1. Can you please share what your role is, both your professional role and your role in the tribe and community?	Specific title(s)
2. How did you get involved with this grant?	Any involvement in planning the grant

If interviewee(s) helped plan the grant:

2a. What were the needs that led to the development of the program?

Contextual Information

Question	Look For
3. What are the goals and objectives of the [insert name of project]? Have the goals changed?	Any changes in goals aligned with changes to the grant priorities
4. From your perspective, are some goals greater priorities than others? If so, why?	Any goals that are especially aligned with school, district, or tribal priorities

Grant Services

Question	Look For
5. Can you please tell me about the students who are served by the [insert name of project]?	Which students are targeted for services, including young adults transitioning from high school and older adults returning to school through tribal college
6. Please describe the entrance and exit measures and criteria for your students served by [insert name of project].	How it is determined that students need assistance Possible answers: Accuplacer (College Board tests for reading and writing) TABE (test for adult ESL skills)
7. Please describe the type of English language [and/or Native language] services you have implemented through [insert name of project]. Is it the same for all students?	Possible answers: Summer program Transitional program Tutoring support for college content courses LEP instructional strategies Computer-assisted learning Learning cohorts
8. What are the main components of the English language [and/or Native language] services you or other instructors provide through [insert name of project]?	Number of hours a day/week Community and tribal involvement Use of technology
9. What is your role in providing these services, and who else provides academic English [and/or Native language] services to students through the [insert name of project]?
10. What do you use as curriculum?	Whether curriculum is commercial or developed by the program

If curricula or materials are developed by the program:

10a. Who develops the curricula or materials?

10b. What is the focus of the curricula or materials?

10c. What support do you receive for curriculum or material development?

11. What is the goal of the language services provided through the [insert name of project]?	Improve academic English Improve literacy Learn and maintain an NA/AN language Promote postsecondary retention and completion
12. What other academic English supports do students receive?	Supports provided through other school or community programs

Professional Development

Question	Look For
13. What are the greatest professional development needs of teachers and instructors at the college? How do you know? How have you responded to those needs?	Providing instruction to ELLs Cultural and linguistic sensitivity
14. Please describe any professional development that has been provided through the grant.	- Content of the trainings: Linguistics and second language acquisition Curriculum and assessment for language learners Strategies to build language skills and subject area knowledge (e.g., how to incorporate language and literacy skills into content) Tribal language instruction Curriculum development Culture heritage and cultural sensitivity Data-based decision making Improving school climate and promoting indigenous culture and language Structure of the trainings (e.g., workshops, PLCs, coaching or mentoring, online courses) Who received the training Who provided the training Any tribal role in the training

If the interviewee(s) provided training or coaching:

14a. How receptive were participants to the training or coaching?

14b. How do you know if the professional development is “working”? Have you seen instructors change their practices?

14c. What challenges did you face, and how did you overcome them?

14d. What supports did you receive, and how helpful were they?

Family Engagement

Question	Look For
15. Have you found other ways to engage families in students’ tribal college education?	How families are involved Communication methods with families

Partners

Question	Look For
16. Who are the partners in the program? What do they do, and what are your interactions with them?	Who the partners are and what their roles are in terms of planning, delivery, and management or oversight
17. What is the role of the tribe or tribal government?	Relationship between the tribal community and the college

Challenges and Successes

Question	Look For
18. What supports have you received, and how helpful were they?	Support from: Department of Education/Office of English Language Acquisition (OELA) Tribal entities School or school district State agencies
19. What benefits for the students, teachers, or the community have you seen as a result of [insert name of project]? What in particular has worked out well?	Level of family and community participation Level of student engagement Increase in number of students attending Increase in graduation rate Increase in English language learning scores Increase in continuation to four-year institution Factors contributing to success
20. What challenges do you face in implementing the [insert name of project], and how have you overcome these challenges?	EXAMPLES
21. What lessons have you learned that you would share with others?

Wrap-Up

22. Is there anything else you would like to tell us about your program?

Parent or family coordinator interview protocol

Grantee:	Interviewer:
Interviewee(s):	Date/Time:
Interviewee(s’) Role(s): (select all that apply)
ᴏ Parent/family activity coordinator	ᴏ Parent/family trainer
ᴏ Community outreach coordinator	ᴏ Social worker
ᴏ Other _______________

Information for the interviewer:

Community and family engagement is one of the priority areas of this grant. We want to understand the activities that grantees use to engage families, grantee outreach efforts, desired outcomes, and any challenges or lessons learned.

PAY ATTENTION TO:

• The types of family and community outreach activities grantees engage in (RQ1)

• The types of programming offered to parents and families and how it is delivered (RQ1)

• Challenges encountered with any of the above (RQ2)

• How the NAM program is perceived in the community (RQ5)

• Participants’ perceptions of outcomes for students and families (RQ5)

By the end of this interview, you should have details and examples for each of these areas.

Introduction

Thank you for taking the time to speak with me today. Just as a reminder, this interview is for a study of the Title III Native American and Alaska Native Children in School (NAM) Program, which is funded by the U.S. Department of Education. The Department is interested in learning how grantees have used the NAM program to support Native American and Alaska Native (NA/AN) academic English achievement and NA/AN language and culture revitalization. The questions will pertain to your role in the program, the program’s history and goals, and the structure and content of the program.

Please know that, as with all federal studies, confidentiality of the information you provide will be protected to the extent provided by law. We will not associate you or your institution’s name with the data we collect, and we will protect your identity in our reports by summarizing findings across groups of grantees and masking any individual examples. We do this so that everyone feels comfortable in sharing their experiences regarding the planning and implementation of the NAM program. We will acknowledge your grant program in the final report, but we will do so separately from the information you provide.

We also would like to ask you permission to audio record this interview to ensure we are accurately capturing your feedback. Only the research team at American Institutes for Research will review the audio files and the transcripts. They will not be shared with any other personnel. At the end of this project, all interviews—audio files and transcripts—will be destroyed. Are you comfortable with being recorded?

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form, answer any questions, and have the respondent sign the form.]

We know that you are very busy, and we want to be respectful about your time, so we’ve made an effort to collect as much information as possible through available documents. We’ll sometimes reference that information during the course of the interview to confirm that our information is correct. This interview will take about 60 minutes.

Do you have any questions before we begin?

Role of Respondent

Question	Look For
1. Please share the activities you do in support of the [insert name of project].	Specific role(s) and title(s)
2. How did you get involved with this grant?	Any involvement in planning the grant

If interviewee(s) helped plan the grant:

2a. What were the needs that led to the development of the program?

Contextual Information

Question	Look For
3. What are the goals and objectives of the [insert name of project]? Have the goals changed?	Any changes in goals aligned with changes to the grant priorities
4. From your perspective, are some goals greater priorities than others? If so, why?	Any goals that are especially aligned with school, district, or tribal priorities

Grant Services

Question	Look For
5. Can you please tell me about the students who are served by the [insert name of project]?	Which students are targeted for services and how the students are grouped to receive services
6. Does the [insert name of project] have a focus on family engagement? If so, what activities are provided to foster family engagement?	Community and tribal involvement Types of activities Opportunities for family involvement (e.g., decision making, involvement in extracurricular events, parent education classes, volunteering at the school)
7. Does technology have a role in community engagement? How?	-Use of technology
8. What is your role in providing these services, and who else engages in family engagement through the [insert name of project]?
9. What is the goal of the community engagement services provided through the [insert name of project]?	Improve academic outcomes Learn and maintain an NA/AN language Promote postsecondary retention and completion

Professional Development

Question	Look For
10. Are family and community members invited to participate in training for the [insert name of project]? In what ways do they participate?	Community members receiving training or providing training Content and structure of trainings Who attends Tribal role in trainings
11. Are there any trainings for educators on family and community engagement?

If the interviewee(s) provided training or coaching:

11a. How receptive were participants to the training or coaching?

11b. What challenges did you face, and how did you overcome them?

11c. What supports did you receive, and how helpful were they?

Family Engagement

Question	Look For
12. Have you found other ways to engage families in [insert name of project] activities?	How families are involved Communication methods with families

Partners

Question	Look For
13. Who are the partners in the program? What do they do, and what are your interactions with them?	Who the partners are and what their roles are in terms of planning, delivery, and management or oversight
14. What is the role of the tribe or tribal government?

Challenges and Successes

Question	Look For
15. What supports have you received, and how helpful were they?	Support from: OELA/Department of Education Tribal entities School or school district State agencies
16. What benefits for the students, teachers, or the community have you seen as a result of [insert name of project]? What in particular has worked out well?	Level of family and community participation Level of student engagement Increase in number of students attending Increase in graduation rate Increase in English language learning scores Increase in continuation to four-year institution Factors contributing to success
17. What are your thoughts about how the [insert name of project] is perceived in the community?	EXAMPLES
18. What challenges do you face in implementing the [insert name of project], and how have you overcome these challenges?	EXAMPLES
19. What lessons have you learned that you would share with others?

Wrap-Up

20. Is there anything else you would like to tell us about your program?

Professional development provider focus group protocol

Grantee:		Interviewer:
Interviewee(s):		Date/Time:
Interviewee(s’) Role(s): (select all that apply)
ᴏ PD provider (school affiliated)	ᴏ Instructional coach		ᴏ Curriculum developer
ᴏ PD provider (not school affiliated)	ᴏ NA/AN language or culture teacher
ᴏ ESL teacher or coordinator	ᴏ Other ______________________________

Information for the interviewer:

Professional development is one of the priority areas of this grant. We want to understand, in a very detailed way, the professional development that is provided.

PAY ATTENTION TO:

• Who provides professional development, and what the provider’s background is (RQ4)

• Who receives professional development and whether the professional development is required (RQ1)

• How the professional development is delivered (i.e., format) (RQ1)

• The content of the professional development (RQ1)

• How the tribe is involved in providing services (RQ4)

• Challenges encountered with any of the above (RQ2)

• Grantees’ perceptions of outcomes for students and families (RQ5)

By the end of this interview, you should have details and examples for each of these areas.

Introduction

Thank you for taking the time to speak with me today. Just as a reminder, this interview is for a study of the Title III Native American and Alaska Native Children in School (NAM) Program, which is funded by the U.S. Department of Education. The Department is interested in learning how grantees have used the NAM program to support Native American and Alaska Native (NA/AN) academic English achievement and NA/AN language and culture revitalization. The questions will pertain to your role in the program, the program’s history and goals, and the structure and content of the program.

Please know that, as with all federal studies, confidentiality of the information you provide will be protected to the extent provided by law. We will not associate you or your institution’s name with the data we collect, and we will protect your identity in our reports by summarizing findings across groups of grantees and masking any individual examples. We do this so that everyone feels comfortable in sharing their experiences regarding the planning and implementation of the NAM program. We will acknowledge your grant program in the final report, but we will do so separately from the information you provide.

We also would like to ask you permission to audio record this interview to ensure we are accurately capturing your feedback. Only the research team at American Institutes for Research will review the audio files and the transcripts. They will not be shared with any other personnel. At the end of this project, all interviews—audio files and transcripts—will be destroyed. Are you comfortable with being recorded?

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form, answer any questions, and have the respondent sign the form.]

We know that you are very busy, and we want to be respectful about your time, so we’ve made an effort to collect as much information as possible through available documents. We’ll sometimes reference that information during the course of the interview to confirm that our information is correct. This interview will take about 60 minutes.

Do you have any questions before we begin?

Role of Respondent

Question	Look For
1. Can you please share what your role is, both your professional role and your role in the tribe and community?	Specific title(s)
2. Did you work with the district, school, tribe, or Bureau of Indian Education prior to [insert name of project]? If so, what kind of work did you do?
3. How did you get involved with this grant?	Any involvement in planning the grant

If interviewee(s) helped plan the grant:

3a. What were the professional development needs that you were looking to address?

Contextual Information

Question	Look For
4. What are the goals and objectives of the grant as you understand them? What are the main priorities for professional development?	Any goals that are especially aligned with school, district, or tribal priorities
5. What are the greatest professional development needs of teachers involved in the program?	Providing instruction to English learners Cultural and linguistic sensitivity
6. Can you please tell me what you know about the students who are served by the [insert name of project]?	Which students are targeted for services and how the students are grouped to receive services

Professional Development

Question	Look For
7. Could you please share how you (or your group) were selected to provide the professional development?	Why interviewee and the institution were selected

If interviewee is at an institution of higher education:

7a. Do you provide pre-service or in-service training or both to teachers?

7b. What are the degrees and endorsements? Do the in-service teachers receive credits of some type (e.g., continuing education)?

If interviewee is at a commercial provider:

7c. Does your professional development connect to textbooks or resources that the school, tribe, or Bureau of Indian Education purchased?

7d. Is the professional development at no cost?

If interviewee is part of the NAM grantee institution:

7e. What specialized training did you receive for the topics that you present?

8. Please describe the professional development you provide.	Continuous Hands-on Modeling or coaching Classroom materials introduced and implemented Based on needs assessment:
9. Please describe the topics of the professional development that has been provided through the grant.	Linguistics and second language acquisition Curriculum and assessment for language learners Strategies to build language skills and subject area knowledge (e.g., how to incorporate language and literacy skills into content) Tribal language instruction Curriculum development Culture heritage and cultural sensitivity Data-based decision making Improving school climate and promoting indigenous culture and language
10. Does the professional development you provide include learning through technology? [If yes] Have you increased the teachers’ understanding and capacity to use technology? In what way?
11. Who participates?	Teachers, administrators, aides, parents
12. Are you aware of any other professional development or training that is occurring?	How different professional development is prioritized
13. How receptive were participants to the training or coaching?

Partners

Question	Look For
14. Who are the partners in the program? What do they do, and what are your interactions with them?	Who the partners are and what their roles are, especially in terms of training and certification and selection of instructors
15. What is the role of the tribe or tribal government?

Challenges and Successes

Question	Look For
16. What supports have you received, and how helpful were they?	Support from: OELA/Department of Education Tribal entities School or school district State agencies
17. What benefits for the students, teachers, or the community have you seen as a result of [insert name of project]? What in particular has worked out well?	Level of family and community participation Level of student engagement Factors contributing to success
18. What challenges do you face in implementing professional development through [insert name of project], and how have you overcome these challenges?	EXAMPLES
19. What lessons have you learned that you would share with others?

Wrap-Up

20. Is there anything else you would like to tell us about your program?

Native american/Alaska Native Language instruction or curriculum development specialist interview protocol

Grantee:		Interviewer:
Interviewee(s):		Date/Time:
Interviewee(s’) Role(s): (select all that apply)
ᴏ NA/AN language or culture teacher	ᴏ Instructional coach		ᴏ Curriculum developer
ᴏ NA/AN language or culture coordinator	ᴏ PD provider		ᴏ Other ____________

Information for the interviewer:

The Native American and Alaska Native Children in School (NAM) Program allows grantees to provide instruction in a Native American and Alaska Native (NA/AN) language or culture, and the funder would like to make this one of the grant priorities in the future, so it is important to understand how grantees are implementing these activities. There are many ways to provide NA/AN language instruction. Common methods include:

• Full immersion or dual language immersion

• Dual language immersion (with English)

• Language instruction as a school subject

• Language or culture instruction through community-based activities

NA/AN language instructors often assume multiple roles in schools, including teaching, curriculum development, and providing professional development to other teachers, as well as leadership and educational roles within their communities and advocacy or policy work at local, state, and national levels. It will, therefore, be important to understand instructors’ roles in providing NAM-supported services within the larger context of their work.

You might be interviewing people who provide other types of services for the program as well.

PAY ATTENTION TO:

• The types of academic programming provided for students and how it is delivered, especially NA/AN language instruction, early childhood education, and technology (RQ1)

• The types of curricula or materials used and who develops them (RQ1)

• How students are selected to receive services (RQ1)

• If anyone is involving family or community members in serving students (RQ1)

• How the tribe is involved in providing services (RQ4)

• Challenges encountered with any of the above (RQ2)

• Grantees’ perceptions of outcomes for students and families (RQ5)

By the end of this interview, you should have details and examples for each of these areas.

Introduction

Thank you for taking the time to speak with me today. Just as a reminder, this interview is for a study of the Title III Native American and Alaska Native Children in School (NAM) Program, which is funded by the U.S. Department of Education. The Department is interested in learning how grantees have used the NAM program to support Native American and Alaska Native (NA/AN) academic English achievement and NA/AN language and culture revitalization. The questions will pertain to your role in the program, the program’s history and goals, and the structure and content of the program.

Please know that, as with all federal studies, confidentiality of the information you provide will be protected to the extent provided by law. We will not associate you or your institution’s name with the data we collect, and we will protect your identity in our reports by summarizing findings across groups of grantees and masking any individual examples. We do this so that everyone feels comfortable in sharing their experiences regarding the planning and implementation of the NAM program. We will acknowledge your grant program in the final report, but we will do so separately from the information you provide.

We also would like to ask you permission to audio record this interview to ensure we are accurately capturing your feedback. Only the research team at American Institutes for Research will review the audio files and the transcripts. They will not be shared with any other personnel. At the end of this project, all interviews—audio files and transcripts—will be destroyed. Are you comfortable with being recorded?

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form, answer any questions, and have the respondent sign the form.]

We know that you are very busy, and we want to be respectful about your time, so we’ve made an effort to collect as much information as possible through available documents. We’ll sometimes reference that information during the course of the interview to confirm that our information is correct. This interview will take about 60 minutes.

Do you have any questions before we begin?

Role of Respondent

Question	Look For
1. Can you please share what your role is, both your professional role and your role in the tribe and community?	Specific title(s)
2. How did you get involved with this grant? What is the process for becoming a language instructor in your community?	Any involvement in planning the grant

If interviewee(s) helped plan the grant:

2a. What were the needs that led to the development of the program?

Contextual Information

Question	Look For
3. What are the goals and objectives of the [insert name of project]? Have the goals changed?	Any changes in goals aligned with changes to the grant priorities
4. From your perspective, are some goals greater priorities than others? If so, why?	Any goals that are especially aligned with school, district, or tribal priorities

Grant Services

Question	Look For
5. Can you please tell me about the students who are served by the [insert name of project]?	Which students are targeted for services and how the students are grouped to receive services
6. What are the main components of your school’s (or district’s) program for students?	NA/AN language or culture Early childhood education Technology Community and tribal involvement

Are you supporting…	If yes…
NA/AN language instruction	Please share with us the number of hours, teachers, grade level, and community (tribal) involvement. What is the goal of the instruction (e.g., maintain heritage language, tribal language revitalization)? What results are you seeing?
Technology	What technology do you use in the [insert name of project], and how do you use it?
Early childhood education	Are there any programs or services for children 0–4 years of age? Please describe. What language focus do young children receive (e.g., English, heritage language)?
Data-based decision making	How are data used to support students’ academic achievement? What changes have been made to the school/district’s data-based decision making as a result of the grant?

7. What is your role in providing these services, and who else provides services to students through the [insert name of project]?
8. What do you use as curriculum?	Whether curriculum is commercial or developed by the program

If curricula or materials are developed by the program:

8a. Who develops the curricula or materials?

8b. What is the focus of the curricula or materials?

8c. What support do you receive for curriculum or material development?

Professional Development

Question	Look For
9. What are the greatest professional development needs of teachers in the school or district?	Providing instruction to English learners Cultural and linguistic sensitivity
10. Please describe any professional development that has been provided through the grant.	Content of the trainings: Linguistics and second language acquisition Curriculum and assessment for language learners Strategies to build language skills and subject area knowledge (e.g., how to incorporate language and literacy skills into content such as math, science, social studies) Tribal language instruction Curriculum development Culture heritage and cultural sensitivity Data-based decision making Improving school climate and promoting indigenous culture and language Structure of the trainings (e.g., workshops, PLCs, coaching or mentoring, online courses) Who received the training Who provided the training Any Tribal role in the training

If the interviewee(s) provided training or coaching:

10b. How receptive were participants to the training or coaching?

10c. What challenges did you face, and how did you overcome them?

10d. What supports did you receive, and how helpful were they?

Family Engagement

Question	Look For
11. Does the program have a focus on family engagement? What activities do you do for family engagement?	How families are involved Communication methods with families
12. What is the role of the community in your classroom and school?

Partners

Question	Look For
13. Who are the partners in the program? What do they do, and what are your interactions with them?	Who the partners are and what their roles are in terms of planning, delivery, and management or oversight
14. What is the role of the tribe or tribal government?	Relationship between the tribal community and the school

Challenges and Successes

Question	Look For
15. What supports have you received, and how helpful were they?	Support from: OELA/Department of Education Tribal entities School or school district State agencies
16. What benefits for the students, teachers, or the community have you seen as a result of [insert name of project]? What in particular has worked out well?	Level of family and community participation Level of student engagement Factors contributing to success
17. What challenges do you face in implementing the [insert name of project], and how have you overcome these challenges?	EXAMPLES
18. What lessons have you learned that you would share with others?

Wrap-Up

19. Is there anything else you would like to tell us about your program?

Program administrator or evaluator interview protocol

Grantee:		Interviewer:
Interviewee(s):		Date/Time:
Interviewee(s’) Affiliation: (select all that apply)
ᴏ Institution of higher education	ᴏ Tribe
ᴏ Consulting firm	ᴏ Other ______________________________

Information for the interviewer:

Every Native American and Alaska Native Children in School (NAM) Program grant has an external evaluator to examine grant success and outcomes. We want to understand the evaluators’ processes and findings.

PAY ATTENTION TO:

• Challenges encountered with grant implementation (RQ2)

• Lessons learned (RQ2)

• Grantees’ perceptions of outcomes for students and families (RQ5)

By the end of this interview, you should have details and examples for each of these areas.

Introduction

Thank you for taking the time to speak with me today. Just as a reminder, this interview is for a study of the Title III Native American and Alaska Native Children in School (NAM) Program, which is funded by the U.S. Department of Education. The Department is interested in learning how grantees have used the NAM program to support Native American and Alaska Native (NA/AN) academic English achievement and NA/AN language and culture revitalization. The questions will pertain to your role in the program, the program’s history and goals, and the structure and content of the program.

Please know that, as with all federal studies, confidentiality of the information you provide will be protected to the extent provided by law. We will not associate you or your institution’s name with the data we collect, and we will protect your identity in our reports by summarizing findings across groups of grantees and masking any individual examples. We do this so that everyone feels comfortable in sharing their experiences regarding the planning and implementation of the NAM program. We will acknowledge your grant program in the final report, but we will do so separately from the information you provide.

We also would like to ask you permission to audio record this interview to ensure we are accurately capturing your feedback. Only the research team at American Institutes for Research will review the audio files and the transcripts. They will not be shared with any other personnel. At the end of this project, all interviews—audio files and transcripts—will be destroyed. Are you comfortable with being recorded?

[Provide respondent with a copy of the informed consent form. Ask the respondent to read the form, answer any questions, and have the respondent sign the form.]

We know that you are very busy, and we want to be respectful about your time, so we’ve made an effort to collect as much information as possible through available documents. We’ll sometimes reference that information during the course of the interview to confirm that our information is correct. This interview will take about 60 minutes.

Do you have any questions before we begin?

Role of Respondent/Evaluator

Question	Look For
1. How long have you been an evaluator of the [insert name of project]? 2. Could you please share what you do in your role as an evaluator of the program? What does the evaluation process entail? Has it changed? 3. Do you have any other roles, for example, do you provide any technical assistance or professional development? [If yes] Could you please share what that involves?	Length of time involved in NAM program Role of evaluator, including other activities: Conducting a formative and/or summative evaluation Using qualitative or quantitative methods Writing reports; how often Presenting data in other ways

Contextual Information

Question	Look For
4. What are the goals and objectives of the program? Have these changed over time (e.g., across the grant cycles)?	Alignment with other stakeholders’ goals and priorities for their grant cycle(s)

Grant Services

Question	Look For
5. Could you please briefly share what kinds of services are provided through [insert name of project]? How are the provided services related to the outcomes you are evaluating?	The evaluator’s perspective on the kinds of services provided through the NAM grant and theory of change for these services

Partners

Question	Look For
6. Who are the partners in the program? What do they do? How have they contributed to the outcomes you are measuring? 7. [If not included as a partner] What is the role of the tribe (or tribal government)?	The evaluator’s perspective on who the partners are, especially tribal partners, and what their roles are in terms of planning, delivery, and management or oversight (e.g., providing materials, selecting or certifying language teachers)
8. How do the grantees meet federal reporting requirements? What requirements are there for you as the evaluator?

Challenges and Successes

Question	Look For
9. What benefits for the students, teachers, or the community have you seen as a result of [insert name of project]? What in particular has worked out well? 9a. What do you think were the factors that contributed to these benefits?	Level of family and community participation Level of student engagement Factors contributing to success
10. Please share the challenges that the grantee has faced in the planning and implementation of [insert name of project]. 10a. Have any of these challenged affected the outcomes you are measuring? 11. Please share which people or community entities have assisted the grantee to decrease or overcome these challenges. 11a. What additional supports could the grantee use?	EXAMPLES

Grant Outcomes

Question	Look For
12. In what way has this program contributed to the following groups of people? How do you know? Students: Have you witnessed any change in their English language proficiency? Have you witnessed any change in their NA/AN language proficiency? Have you witnessed any change in their engagement in language instruction and other education programs? Educators School Tribal community Family and community Partners through collaboration in this program	Level of family and community participation Level of student engagement
13. The [insert name of project] grant application identified [outcome from application] as a key outcome for this program. What changes have you seen in [outcome from application]?
14. What measures do you use to track progress and outcomes of funded services?

Wrap-Up

15. Is there anything else you would like to tell us about the program?

6290_06/16