Justification

This material is being submitted under the generic U.S. Department of Education clearance agreement (OMB #1880-0542), which provides for the Department to conduct surveys and other studies regarding customer satisfaction.

Background and Study Rationale

The purpose of this focus group study is to evaluate the National Center for Education Statistics’ (NCES) report, National Indian Education Study (NIES) 2011, and to eventually recommend enhancements for future NIES reports. In 2004, the NIES was created to fulfill the United States’ responsibility for the education of American Indian and Alaska Native (AI/AN) children to meet the academic achievement standards set forth in the Elementary and Secondary Education Act of 2001 (ESEA 2001). The study is sponsored by the Department of Education’s Office of Indian Education. The focus of the NIES is the academic achievement of AI/AN students and their educational experiences. The NIES report gives an in-depth view of the academic performance of fourth and eighth grade AI/AN students on the National Assessment of Educational Progress (NAEP) 2011 reading and mathematics assessments. The report further explores the educational experience of AI/AN students who participated in the NAEP assessments. NIES was previously administered biennially from 2005 to 2009, and since 2011 is currently administered every four years.

As part of the continuing effort to enhance the NIES reports, NCES will conduct a focus group study on the NIES 2011 report that NCES released in 2012. To ensure that appropriate and relevant feedback is obtained from meaningful sources, participation will be sought from National Congress of American Indian (NCAI) members and conference attendees.

The focus groups will seek to discern the knowledge, attitudes, and beliefs of the participants concerning various aspects of the NIES 2011 report. These aspects include: the utility of the presentation of the data (e.g., accessibility and functionality of language), layout and data presentation, clarity of the information being reported, ease of presentation for accessing relevant information, and effectiveness of form to elucidate content (e.g., the role of color in alerting the reader to a change in subject matter or how user-friendly the color scheme is). The focus groups will provide needed feedback on the content, presentation, and usability of the reports for end-users for improving existing methods of presentation of data for the varied audiences that use the reports to improve academic performance and the educational experience of AI/AN children in the U.S. The focus of this activity will include the investigation of specific items (graphs, examples, statistics, etc.), mode of presentation, and knowledge conveyed. A report that summarizes the focus group findings will be produced.

Study Design and Context

Two focus groups will be held, both addressing the publication of the NIES 2011 report. The focus group method is a qualitative, group approach to data collection. It involves holding group sessions guided by a moderator, who follows a topical outline containing questions or topics focused on a particular issue. The questions are purposefully open-ended in nature, incorporating several prompts to encourage a deeper, multifaceted exploration of an issue or topic. The method is useful for going beneath the surface of a response. As a research tool, focus groups are useful for understanding a targeted group’s views on complex social issues such as challenges and issues surrounding cultural issues and education. This data collection method has been frequently used in Indian country with considerable success (see e.g., Devlin, et al., 2006).

To obtain a broad geographical and cultural representation, to minimize the cost of the focus group study, and to reduce the potential burden of cost and time for the participant, NCES has decided to hold the two sessions at a location and time already scheduled for participants. The NCAI mid-year conference is being held in St. Paul, MN, from June 28 to July 1, 2015. This conference was chosen as the site for the focus group study because of its broad-based membership, which includes researchers, educators, administrators, and policy makers. It is also anticipated that many of these individuals will also be AI/AN parents, with children in public or Bureau of Indian Education (BIE) schools.

Recruitment

Kauffman and Associates, Inc. (KAI), will recruit participants and conduct the focus groups for NCES. A sample of NCAI members who would have interest in the reports will be recruited through an e-mail solicitation sent from KAI by the NCAI Meetings and Events team (see Appendix A: Recruitment Letter). This e-mail will be sent to approximately 800 NCAI members and will instruct interested participants to respond to KAI with their relevant contact and organizational affiliation information. Based on experience, a response rate of five percent for recruitment of this kind is expected. Respondents who are selected for participation will receive a response e-mail from KAI (see Appendix B: Participation Letter) to thank them for their interest, provide information regarding the focus group study, and provide the web link to the study report for their review prior to the focus group. Participants will be informed in the Participation Letter that they are expected to spend approximately 60 minutes reviewing the study reports in preparation for their participation in the focus group, and that they will be provided the physical copy of the study report for their focus group at the time of registration at the NCAI conference. When the needed purposive sample has been reached, respondents whose participation is not needed will receive a short e-mail letter thanking them for their interest and informing them that their participation will not be needed (see Appendix C: Non-Selection Notification).

The ideal focus group size is 12 people. Having a group this size enables discussion and provides for a diversity of ideas to be heard. With more than 12 participants, it is not possible to ensure all views are heard or that a discussion can occur. It also has the potential of limiting the number of questions that can be addressed in the time allotted. Having a group with less than nine individuals limits the number of perspectives that are heard. It is also difficult to have a discussion developed when there are very few participants engaged in the process. It is anticipated that the NIES 2011 report focus groups will have 12 participants each, for a total of 24 people engaged in the study.

Focus Group Logistics

At the time of registration at the NCAI conference, participants will receive a letter (see Appendix D: Registration Letter) that outlines the logistics of their focus group participation, and is accompanied by both the Consent Form (see Appendix E: Focus Group Consent Form) and a hard copy of the study report for their focus group. The Registration Letter will ask participants to read and bring the consent form to the focus group. The consent form outlines the name of the project and its purpose, the criteria and procedure for participation, including the ability to withdraw from participation at any point during the process without negative consequence. It also outlines the risks and benefits of taking part in the study, the process for handling confidential information, any costs or compensation for involvement, and the project director’s contact information. All participants will receive a copy of the signed consent form for their records.

Each of the focus groups will be approximately 90 minutes long. They will take place during the early evening hours from 5:30 to 7:00 p.m., on June 29 and 30,when most conference participants are in attendance. This time schedule will help to ensure that the maximum number of individuals will be available to attend without interfering with participants’ participation in the conference’s sessions or evening events.

Data Collection Process

At the chosen time, the group will convene where the focus group moderator will introduce the process, collect consent forms, collect basic demographic data (see Appendix F: Demographic Survey) and conduct the focus group discussion. The purpose of the demographic survey is to gather basic data with respect to age, gender, tribal and institutional affiliation, and association to AI/AN education (whether as professionals, parents, or both). This data will be used in aggregate form to contextualize the findings of the focus groups. The focus groups will be moderated by an experienced KAI researcher trained in conducting focus groups. This individual has an appreciation and knowledge of how and when to use prompts as well as how to keep a discussion moving and on target. This person is also knowledgeable about how to clarify questions without directing respondents toward a particular response. A member of the research team will take notes during the focus group and an audio recorder will be used to ensure all comments are captured. The moderator will use the focus group guide for each of the two sessions (see Appendix G: Focus Group Protocol). Each focus group session will take approximately 90 minutes to complete.

The project team will employ a systematic approach to the analysis of the qualitative data and the verifiability of the findings (Krueger, 1998). The information collected will be transcribed and coded by themes to facilitate the write up of the focus group findings.

Consultations Outside of the Agency

KAI is an American Indian, woman-owned organization that has worked on issues of education, health, and well-being of Native communities for over 20 years. KAI has over six years of experience in directly supporting the Office of Indian Education (OIE). With a corporate office located in the heart of Indian Country in Spokane, WA, and an office in Washington, DC, KAI currently employs 22 enrolled members of federally recognized tribes. KAI has worked closely with hundreds of tribal communities and thousands of tribal members to address a range of concerns. KAI is known for its culturally appropriate and culturally sensitive work in support of Native communities, and is highly respected by tribal leaders for the services they provide.

Assurance of Confidentiality

Prior to the start of the focus groups, participants will be notified that their participation is voluntary and that their answers may be used only for research purposes, and may not be disclosed or used in identifiable form for any other purpose except as required by law (20 U.S.C. §9573). Personally identifiable information will not be maintained for the participants.

During the informed consent process, participants will be asked to read and sign the consent form (see Appendix E: Focus Group Consent Form). Participant responses will be audio-recorded and written notes will be taken. However, names will not appear in any oral presentation or written report and any information provided will be aggregated with other participants’ information.

After the participant and the moderator sign the consent forms, copies will be made for the participants to have for their records. Only individuals who sign the consent form will be eligible to participate in the focus group.

Justification for Sensitive Questions

Throughout the focus group development process, effort has been made to avoid asking for information that might be considered sensitive or offensive.

Estimate of Hourly Burden

This study includes two focus groups. The total number of respondents will be 24 (12 per group). The overall burden encompasses the NCAI personnel sending the recruitment letter, the 800 NCAI members receiving the recruitment letter, the 40 NCAI members recruited who are expected to respond, and the 24 focus group participants. Respondents are NCAI members who work with AI/AN children in an educational setting, as researchers, tribal policy makers, and/or parents of AI/AN children.

The NCAI personnel burden is estimated at three hours. The focus group participants’ total response burden is comprised of .017 hours (one minute) to read the recruitment letter, .083 hours (five minutes) to write a response expressing interest in participation, 1.0 hours to review the NIES report before the focus group, and 1.5 hours for the focus group interviews.

Table 1. Estimated Burden Hours

Respondent Type	Number of Respondents	Number of Responses	Burden Hours per Respondent	Total Hours
NCAI Personnel Involved in Recruitment	1	1	3	3
NCAI Members Contacted during Recruitment	800	800	.017	14
NCAI Members Responding to Recruitment Letter	40*	40	.083	4
Focus Group Participants	24*	24	2.5	60
Total Respondents	801	865		81

Subset of the contacted group

Estimate of Costs for Recruiting and Paying Respondents

To encourage participation and thank the participants for their time and effort, a $50 gift card will be offered to each participant of the focus groups. The total estimated cost for this participation is $1,200.

Cost to Federal Government

The estimated cost to the government of all recruitment and data collection, analysis, and reporting activities for this study is $17,301.

Project Schedule

The following table depicts the schedule for efforts related to the implementation of the focus groups.

Table 2. Project Schedule

Activity	Date
Coordinate with NCAI logistics manager for recruitment process	April 2015
Conduct two focus groups (NIES 2011 report) at the 2015 National Congress of American Indians Mid-Year Conference	June 2015
Complete transcription of focus groups	July 2015
Draft and submit a summary report on focus group studies	July 2015
NCES review of draft summary report	July 2015
Revise and submit a final summary report on focus group studies	August 2015

REFERENCES

Devlin, H., Roberts, M., Okaya, A., and Iang, Y. M. (2006). Our lives were healthier before: Focus groups with African American, American Indian, Hispanic/Latino people with diabetes. Health Promotion Practice, 7(1): 47-55.

Krueger, R. (1998). Analyzing & reporting focus group results. Thousand Oaks, CA: Sage Publications.

Appendix A RECRUITMENT LETTER

Dear National Congress of American Indians Attendee,

As you may know, the National Indian Education Study (NIES) was created in a 2004 Executive Order to fulfill the United States’ responsibility to educate American Indian and Alaska Native children according to nationwide standards for academic achievement. The study contains two parts: the first part assesses fourth and eighth grade students in reading and mathematics, and the second part asks students, teachers, and principals about the integration of the students’ native cultures in their educational experience. The study, which was conducted biennially until 2011, is currently conducted every four years. The report for the 2011 NIES was released in July of 2012.

On behalf of the U.S. Department of Education’s National Center for Education Statistics (NCES), Kauffman & Associates, Inc. (KAI) will conduct two audio-recorded focus group sessions on the report at the National Congress of American Indian (NCAI) mid-year conference in St. Paul, MN, on the evenings of June 29 and 30, 2015, from 5:30 to 7:00 p.m. KAI is an American Indian, woman-owned company based in Spokane, WA, specializing in the education, health, and well-being of Native communities.

The purpose of the focus groups is to provide feedback to NCES on the content, presentation, and usability of the NIES reports in order to improve future publications. The focus group sessions will give selected NCAI participants an opportunity to provide input on the NIES. Should you choose to participate, and if selected, you will be randomly assigned to one of the focus group sessions and receive a $50 gift card for your time and effort.

If you are willing to participate in these discussions and have your opinion heard, please respond by sending an email with your name, organization, position, and contact phone number to Kristen Hudgins at [email protected]. Feel free to call Dr. Hudgins at (301) 395-9401 should you have any questions related to this project, the reports, or what your participation would entail. We hope that you will be able to join us for these important discussions and we appreciate your participation.

Sincerely,

Kristen Hudgins, PhD

Senior Researcher

Kauffman & Associates, Inc.

Appendix B PARTICIPATION LETTER

Dear [Prefix]. [First name] [Last name],

On behalf of Kauffman & Associates, Inc. (KAI) we would like to thank you for agreeing to participate in the National Indian Education Study 2011 Report focus group. As you may recall, the National Indian Education Study has two parts: the first part assesses American Indian/Alaska Native students in grades four and eight in reading and mathematics, and the second asks the students, teachers, and principals about the integration of the students’ Native cultures in their educational experience.

For the purposes of this focus group, you are being asked to share your thoughts and opinions in an audio-recorded discussion about the National Indian Education Study 2011.

In preparation for your participation in the focus group discussion, we ask that you take 60 minutes to review the report online before the conference to familiarize yourself with its content. The report is available for download at: http://nces.ed.gov/nationsreportcard/pdf/studies/2012466.pdf.

Please keep in mind that we do not expect you to spend any more than an hour to review the report prior to the focus group. To thank you for your time and effort, you will receive a $50 gift card at the close of the focus group.

Upon registering at the National Congress of American Indians (NCAI) mid-year conference in St. Paul, MN, you will be given physical copy of this report as well. Your input is very important to the future of American Indian and Alaska Native children, and we appreciate the time and effort you are investing by participating in the focus group. If you have any questions or concerns regarding the report, the project, or what your participation will entail, please contact me at 301-395-9401 or at [email protected].

Thank you very much and we look forward to seeing you at the NCAI mid-year conference in St. Paul, MN.

Sincerely,

Kristen Hudgins, PhD

Senior Researcher

Kauffman & Associates, Inc.

Appendix C NON-SELECTION NOTIFICATION

Dear [Prefix]. [First name] [Last name],

On behalf of Kauffman & Associates, Inc. (KAI) we would like to thank you for your willingness to participate in the National Indian Education Study 2011 Report focus group. Unfortunately, due to the volume of responses received from interested National Congress of American Indians (NCAI) members, you have not been selected to participate in this project. If you have any questions or concerns regarding this communication, please contact me at 301-395-9401 or at [email protected].

Thank you very much for your willingness, and best to you at the NCAI Mid-Year Conference in St. Paul, MN.

Sincerely,

Kristen Hudgins, PhD

Senior Researcher

Kauffman & Associates, Inc.

Appendix D REGISTRATION LETTER

Dear NCAI Mid-Year Conference Participant,

Welcome! We are glad you are here and look forward to meeting you at the National Indian Education Study focus group session, which will be held at [INSERT Day, Date, Time, Duration, and Location].

As you know, the focus group you are attending will be a discussion of the National Indian Education Study 2011. We have enclosed a copy of this report for you to review prior to the focus group session. As was mentioned in previous correspondence, in preparation for your participation in the focus group discussion, we have asked that you take approximately 60 minutes to review the report to familiarize yourself with its content. Please keep in mind that we do not expect you to spend any more than 60 minutes reviewing this report.

Also enclosed is the Focus Group Consent Form. We ask that you review this form and bring it with you to the focus group session. You will have an opportunity to ask any questions concerning privacy and confidentiality before signing the form.

Thank you, and we look forward to meeting you.

If you have any questions, please feel free to contact me at [email protected] or 301-395-9401.

Sincerely,

Kristen Hudgins, PhD

Senior Researcher

Kauffman & Associates, Inc.

Appendix E FOCUS GROUP CONSENT FORM

Introduction

In 2004, the National Indian Education Study (NIES) was created to fulfill the United States’ responsibility for the education of American Indian and Alaska Native (AI/AN) children to meet the academic achievement standards set forth in the Elementary and Secondary Education Act of 2001. The National Center for Education Statistics (NCES) reports on the results of the National Assessment of Educational Progress (NAEP) on reading and mathematics assessments and on the educational experiences of AI/AN students in grades 4 and 8.

Kauffman & Associates, Inc. (KAI), an American Indian-owned company based in Spokane, WA, is conducting focus groups regarding the National Indian Education Study Report 2011 on behalf of NCES. The purpose of the focus groups is to assess the data presentation in terms of accessibility and functionality of language, layout, ease of accessing relevant information, etc. The goal of this project is to provide feedback to NCES on the content, presentation, and usability of the reports for end users in order to improve the publications.

Participation and Confidentiality

Your participation is entirely voluntary. If you agree to participate in this project, you will be asked to take part in a 90-minute focus group. You have the right to refuse to answer any question that is asked, including demographic questions, and to refuse to participate at any point.

Participant responses will be used only for research purposes and will not be disclosed or used, in identifiable form, for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) 20 U.S.C., § 9573].

Your responses will be audio-recorded and written notes will be taken. However, your name will not appear in any oral presentation or written report and any information that you provide to us will be aggregated with other participants’ information. Electronic and hard copies of focus group notes and other data will be stored without personal identifiers in secure locations and will be destroyed at the close of the project. If you have any questions regarding this project or your participation, please contact Kristen Hudgins, PhD, Senior Researcher at KAI at (301) 395-9401 or at [email protected].

Risks and Benefits

There are no known risks from taking part in this project. The main benefit of your participation in the project is the enhancement of the NIES Report. Finding out this kind of information will enable NCES to better serve AI/AN students and their education.

Incentives

To encourage participation and thank participants for their time and effort, each will receive a $50 gift card.

Consent to Participate in Project
I have read and reviewed the above information about this project. I hereby consent and voluntarily agree to participate in the NCES-NIES project. By signing, I am not waiving any legal claims, rights, or remedies.

___________________________ ___________________________
Participant’s Printed Name Participant’s Signature Date

___________________________ ___________________________ ___________________

Interviewer’s Printed Name Interviewer’s Signature Date

According to the Paperwork Reduction Act of 1995, “no persons are required to respond to a collection of information unless such collection displays a valid OMB control number.” The valid OMB control number for this information collection is 1880-0542. The time required to complete this information collection is estimated to average 2.5 hours, including the time to review the report.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS
Author	jeanette.hassin
File Modified	0000-00-00
File Created	2021-01-27