National Assessment of Educational Progress (NAEP) 2015 National Indian Education Study (NIES) Report Focus Group

National Center for Education Statistics

National Assessment of Educational Progress

Volume II

Interview Protocol

National Assessment of Educational Progress (NAEP)

2015 National Indian Education Study (NIES) Report Focus Group

OMB# 1880-0542

April 2019

Table of Contents

Paperwork Burden Statement 2

Moderator’s Interview Script 3

Moderator’s Interview Script PowerPoint Slides 7

The Paperwork Reduction Act and the National Center for Education Statistics (NCES) confidentiality statement are indicated below. Appropriate sections of this information are included in the consent forms and letters. The statements also will be included in the materials used in the study.

Paperwork Burden Statement

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid Office of Management and Budget (OMB) control number. The valid OMB control number for this voluntary information collection is 1880-0542. The time required to complete this information collection is estimated to average up to 60 minutes prior to the focus group to allow ample time to review the report and then 90 minutes for the focus group, including the time to review instructions and complete and review the information collection. If you have any comments concerning the accuracy of the time estimate, suggestions for improving this collection, or any comments or concerns regarding the status of your participation, please write to: National Assessment of Educational Progress (NAEP), NCES, Potomac Center Plaza, 550 12th St., SW, 4th floor, Washington, DC 20202.

Authorization and Confidentiality Assurance

National Center for Education Statistics (NCES) is authorized to conduct NAEP by the National Assessment of Educational Progress Authorization Act (20 U.S.C. §9622).

All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C.§151).

OMB No. 1880-0542 Approval Expires 07/31/2020.

Moderator’s Interview Script

Interviewer Script for National Indian Education Study 2015 Report Focus Group

Materials needed for Focus Group:

• PowerPoint presentation

• Laptop

• Hard copies of NIES 2015 Report

• Colored paper to be passed out at the end of session for comments

• Pens and/or pencils

• Hand-outs with links if participants are interested in learning more about NAEP

Room set-up:

• Tables and chairs positioned so that each participant can see the moderator and other members of the focus group

Time:

• 90 minutes total

  • Facilitator introduction and welcome, purpose, ground rules, questions, participant intros: 20 minutes

  • Discussion questions/responses: 60 minutes

  • Wrap-up: 10 minutes

Utilize PowerPoint to provide visual prompts: Facilitator introduction, welcome and purpose of focus group, ground rules, questions, participant introductions, discussion questions, and wrap-up script follows.

Introduction: My name is [NAME] and I will be facilitating our discussion today.

Brief introduction of Tribal Tech, LLC and role in the focus group process.

• Tribal Tech, LLC an American Indian, woman-owned company based in Alexandria, VA, specializing in the education, health, and well-being of Native communities.

• Government contractor assisting the Department of Education in conducting a focus group for the NIES 2015 Report.

Welcome and purpose of focus group:

I want to thank you for participating in today’s focus group. You are all here to provide feedback on the National Indian Education Study (NIES) 2015 Report, which is designed to describe the condition of education for American Indian and Alaska Native (AI/AN) students in the United States. The NIES consists of the following sections:

• The Introduction section, which includes information on sampling, participation, reporting, the NIES 2015 states, and the school types reported on in the survey question tables throughout the report.

• The Survey Questionnaires section describes the purpose of the survey questions and the development process used to create them.

• The Students and Native Culture section includes information about students’ exposure to and participation in their Native culture and their experiences with Native language at home and in school.

• The Teacher Characteristics section examines the ways in which teachers acquired the knowledge and skills they need to teach AI/AN students, teachers’ self-identified race/ethnicity, and the culturally specific instructional practices used in reading and mathematics lessons.

• The Schools and Community Engagement section examines AI/AN community member events that take place at school and the availability of good quality AI/AN books and other materials in school libraries or media centers.

• The Performance Results section provides information about the performance of AI/AN students at grades 4 and 8 in the nation and in the NIES states on NAEP reading and mathematics assessments from 2005 to 2015.

Our goal for this focus group is to evaluate the NIES report and to recommend enhancements for future NIES reports.

First, let me begin with a brief background of both NAEP, and NCES. NAEP was first administered in 1969 and is the largest continuing and nationally representative assessment of what our nation’s students know and can do in mathematics, reading, science, writing, and other subject areas too. NAEP is a standardized assessment, meaning that the assessment is administered under the same conditions in each participating school, resulting in a common measure of student achievement. The NIES 2015 Report is administered as part of the NAEP to allow more in-depth reporting on the achievement and experiences of American Indian/Alaska Native students in grades 4 and 8. The results of NAEP are released as The Nation’s Report Card, and are available for the nation and, in some cases, states and urban districts – known as the Trial Urban District Assessment. This program is run by NCES, which is part of the U.S. Department of Education.

The NIES report focuses primarily on two themes identified during the development of the NIES survey questionnaires.

• To what extent are AI/AN culture and language part of the curricula?

• To what extent are school resources available for improving AI/AN student achievement?

As I mentioned earlier, I will be facilitating a discussion about the NIES 2015 Report.

Ground Rules:

Before we get started today, I would like to offer some reminders and set a few ground rules for our discussion. Remember, NCES is a statistical agency and NCES’ role is reporting the data and not offering policy or curriculum recommendations.

This focus group is being transcribed and audio recorded so that researchers can review the recordings later. This recording is only meant to help us summarize what we learn today. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (if needed: 20 U.S.C. §9573 and 6 U.S.C. §151).

1. What is said here today will be aggregated based on the themes and topics we will be addressing. No names or any identifiable information will be associated with responses or appear on any presentation or report on this focus group. We will be using a recording device to ensure that we preserve a complete rendering of what is shared in this group, and the recording will be destroyed when the project is completed. Only the U.S. Department of Education and its affiliated contractors will be privy to the outcomes of this session.

2. I want to point out that I am not involved in the development of the report we will discuss, so please feel free to be open and honest in your comments.

3. We want to have an open discussion. It is important to hear what everyone thinks, so please speak up and please let others do the same.

4. You do not have to wait for me to call on you, but please speak one at a time.

5. You can respond to each other as well as to me.

6. You might disagree about certain topics, but please be respectful of each other.

7. There are no right or wrong answers. All comments, positive and negative, what you know and do not know, are important.

8. If I ask a question you are not comfortable answering, feel free not to answer.

9. We are recording the discussion so I can write a report. However, no names will be included in the report.

10. My job is to keep us on the right track and moving along so that we are done in an hour and a half. From time to time, I may have to jump in to keep our conversation moving.

11. If you need to go to the restroom or need a break, feel free to step away, but please come back as soon as possible because all opinions are important.

12. Please turn off or silence all of your electronic devices to avoid distractions to yourself and those around you.

Do you have any questions before we get started? [Pause for questions]

Let’s get started with introductions – Please tell us your name, your position in education, and what grade and subject(s) you teach, if you are a teacher. For administrators, tell us what grades or school levels you lead. For other stakeholders, tell us your position and interest in education. [Participant introductions]

Thank you. Now to our questions. The first couple of questions will ask you about your involvement and use of the NIES 2015 Report and your general feelings, thoughts, and reactions to it.

Discussion questions/responses: 60 minutes

1. Please share something about your understanding of the NIES 2015 report, and how you might use the results of this report.

Prompts

• NIES is not designed to provide individual or school level results. Additionally, although students from across the country participate, the majority of states will not have enough AI/AN students participating to be able to report state-level results. Given this design, how should the results be presented in a way that is still relevant for stakeholders? How will you be using the report to inform the work you do?

2. It was recommended you review the report, up to 60 minutes, prior to the focus group. In reviewing the NIES 2015 data of AI/AN students at grades 4 and 8, what stands out as the report’s most relevant features to you or to your work?

Prompts

• What was your initial reaction to seeing this report and, after some reflection, what if anything has altered or reinforced that perception?

• Describe how the presentation of the material in this report can affect the reader’s ability and/or interest in using it.

• Is the organization and presentation of the report helpful when accessing key information?

• Does the presentation of the report affect its utility within the educational setting of a school?

3. Were there aspects of the report that impeded your understanding?

Prompts

• Describe the report’s effectiveness in conveying and/or presenting its findings to the reader.

• Please share your thoughts on the presentation of the data in relation to being culturally responsive.

The following topics will address your feelings and thoughts related to specific aspects of the NIES 2015 Report.

4. How does the content organization of the NIES 2015 Report affect how the reader might interpret the findings?

Prompts

• What aspect of the content organization assisted you the most in your understanding of the material?

• What could be the different ways educators, parents, or community members understand the data based on the how the report was organized?

5. How did the graphic elements (color, font, tables, or graph) enhance your understanding about the report?

Prompts

• What effect did the use of color have on the transmission of information?

• What influence did color have on your ability to differentiate the sections in the report?

• Discuss the layout in terms of accessibility of data to readers.

• Explain how the way the data are presented (e.g., order, layout, graphs, use of photographs and examples) can affect the reader’s view of the information.

6. Given the design of the study and subgroups we are able to report on, which group(s) should be our target audience?

Prompts

• What key points would you be most interested in sharing with other educators, parents, community members, or others?

The last question will deal with your thoughts about the NIES 2015 Report.

7. Is there anything else about the NIES 2015 Report that we have not discussed that you feel should be addressed?

Wrap-up: 10 minutes

Final thoughts…this is your opportunity to let us know what you want and need in order to make this report truly useful to you in your work. I would love to hear your suggestions [Pause for responses]. Thanks for your responses. If you did not get a chance to respond, feel free to write your suggestions on a piece of paper [Hand out half-sheets of colored paper] and give them to me on your way out.

Thank you again for your time and your willingness to share your experiences and thoughts. They will be invaluable to the developers as they consider how to provide the information you need, in a format that is helpful to you in your work. Have a great evening!

END

Moderator’s Interview Script PowerPoint Slides