Part B Title VI Study_12-6-17

This study of the U.S. Department of Education’s Title VI Indian Education LEA Grants Program is designed to examine how grantees provide culturally responsive services and activities and help American Indian/Alaska Native (AI/AN) students meet state academic standards; align and leverage program-funded services with those funded by other federal, state, and local sources; identify eligible AI/AN students; establish and implement program priorities with parent, community, and tribal involvement; and assess progress towards Title VI service objectives.

In 2015, the Department awarded 1,282 Title VI grants to local education agencies (LEAs), Bureau of Indian Education (BIE)-operated schools, BIE grant and contract schools, and Tribes. All Title VI Grant Coordinators will be invited to participate in the study survey and nine grantees will be invited to participate as case study sites. The national survey will provide information on the full population of grantees while the case studies will offer a more detailed and nuanced description of the work of a small subset of Title VI grantees in their efforts to serve AI/AN students. Exhibit 1 presents the universe of grantees, the number of respondents that will be selected to participate in each data collection activity, and the expected response rate.

Exhibit 1. Universe of respondents and sample selection

Data collection activity	Universe of respondents	Sample selection	Expected response rate
Survey, Title VI Grant Coordinator	~1,300 Grant Coordinators (estimated based on grants awarded in 2015)	Universe (~1,300 grantees)	> 85 percent
Case study interviews, Title VI Grant Coordinators	~1,300 Grant Coordinators (estimated based on grants awarded in 2015)	9 Grant Coordinators (1 in each of 9 site selected for case study participation)	100 percent
Case study interviews, other grantee staff	unknown	45 (5 in each of 9 sites selected for case study participation)	100 percent
Case study interviews, parents	unknown	144 (16 in each of 9 sites selected for case study participation)	100 percent
Case study interviews, tribal leaders	unknown	18 (2 in each of 9 sites selected for case study participation)	100 percent

2. Procedures for the collection of information

Survey sampling

The Title VI Grant Coordinator survey will be administered to the universe of Title VI grantees. Surveying the entire population of current Title VI grantees (~1,300), rather than a random sample, is preferable for the following reasons:

Small population of grantees. With only approximately 1,300 Title VI grantees, surveying the population is feasible with respect to survey administration and follow-up activities.
Smallest margin of error. Surveying the population would yield the smallest margin of measurement error, given expectations of an 85 percent response rate.
Disproportionate number of grantees by grantee type and region, and variation in award size. Surveying the population of grantees will be more efficient than designing a sampling frame that accounts for the disproportionately large number of LEA grantees, grantees in particular regions of the U.S., and the substantial range in grant award size (from $4,000 to over $3 million).

Case study sampling

The case study sample consists of nine grantees, including LEAs, BIE-operated schools, and tribes. The study team will use an iterative approach to selecting potential Title VI Indian Education grantees for the case studies that comprises the following steps.

Select in rough proportion to distribution of grantees by:

Grant award size
Grantee type (e.g., LEA, BIE, tribe)
Geographic region

Consider demographic information:
1. Concentration of AI/AN students
2. Number of AI/AN students
Consider project-level information:
1. Grade levels targeted (pre-K, elementary, middle, high)
2. Objectives and associated activities/services

Select for variation by grant size, grantee type, and geographic region

The average grant size across all grantees for the 2015–16 school year was $78,259 with a median award less than that amount, and grants ranged from $4,000 to $3,144,787.¹ The study team will select at least one grantee with a grant size in the first quartile, two grantees in the second quartile, two grantees in third quartile, and four grantees in the fourth quartile (Exhibit 2). The study team will sample four projects at the high end of the distribution because grant sizes range much more widely in the fourth quartile (from $80,963 to $3,144,787). Sampling from across the full range of grant sizes will help ensure that the study provides relevant information for a broader range of grantees. Oversampling grantees with larger grants will allow the study to provide more information about the settings where large numbers of AI/AN students are served.

Exhibit 2. Proposed case study sample by grant size

Grant award size [distribution and amount]	Total number of grantees	Number of projects proposed
Large award [above the 75th percentile ($80,963)]	320	4
Medium award [at or below the 75th percentile ($80,963) and above the median ($38,490)]	321	2
Small award [at or below the median ($38,490) and above the 25th percentile ($19,069)]	321	2
Very small award [at or below the 25th percentile ($19,069)]	320	1
Source: Electronic Application System for Indian Education (EASIE) Budget Report.

Grantees also vary by administering agency (“grantee type”). As shown in Exhibit 3, LEAs are the most common type of grantee: 85 percent of the total grantees are single-applicant LEAs, and 3 percent are LEAs leading consortia. The remaining are BIE operated (6 percent), BIE grant and contract (5 percent), and tribes that apply in lieu of one or more LEAs (2 percent). To select a sample that is inclusive of all grantee types and roughly representative of the various types of grantees, the study team will select six LEA grantees (including one consortium leader), one BIE-operated grantee, one BIE grant and contract grantee, and one tribe grantee that applied in lieu of one or more LEAs.

Exhibit 3. Proposed sample by grantee type

Grantee type	Total number of grantees	Percent of total	Number of projects proposed
LEA (single applicant)	1,090	85%	5
LEA (consortium leader)	39	3%	1
BIE-operated school	71	6%	1
BIE grant and contract school	58	5%	1
Tribe applying in lieu of LEA/LEAs	24	2%	1
Total	1,282	100%	9
Exhibit reads: Of the 1,282 Title VI Grants Program grantees, 1,090 (or 85 percent) are LEAs that applied as a single LEA applicant. The study team is proposing to sample five projects from this pool. Note: Percentages do not add to 100 because of rounding. Source: EASIE Budget Report.

Next, because the context of AI/AN schooling varies by geographic region (e.g., the percentage of AI/AN students who attend BIE schools, the population density of the communities served by the AI/AN students’ schools, and the percentage of AI/AN students who are English learners),² the study team will sample by region. The study team sorted grantees by the five U.S. Census regions used in the National Assessment of Educational Progress (NAEP) National Indian Education Study and examined the distribution by grantee type, total number of grantees, and percentage of total grant funds by region (Exhibit 4).

Exhibit 4. Region by grantee type, number of grantees, and percentage of total grant funds

	Number of grantees by grantee type					Total number of grantees	Percentage of total federal grant funds
Region	LEA	LEA consortia	BIE grant and contract	BIE operated	Tribe [in lieu of LEA(s)]	Total number of grantees	Percentage of total federal grant funds
Pacific	211	20	5	4	4	244	24%
Mountain	179	2	36	46	5	268	26%
North Central	231	14	24	6	6	281	17%
South Central	425	2	3	1	3	434	28%
Atlantic	44	1	3	1	6	55	6%
Total	1,090	39	71	58	24	1,282	100%
Exhibit reads: Of the 1,282 Title VI Grants Program grantees, 244 are located in the Pacific region. Of those 244 grantees, 211 are single applicant LEAs, 20 are LEAs that applied to lead consortia, five are BIE grant and contract schools, four are BIE-operated schools, and four are tribes that applied in lieu of one or more LEAs. In total, the Pacific region received 24 percent of all Title VI Grants Program funds. Note: Percentages do not add to 100 because of rounding. Source: EASIE Budget Report.

The South Central, Mountain, and Pacific regions receive similar percentages of the total Title VI Grants Program funds (28 percent, 26 percent, and 24 percent, respectively), followed by the North Central region (17 percent) and the Atlantic region (6 percent). The South Central region includes the most individual grantees (by a wide margin). Given this distribution across the regions, the study team plans to sample at least two grantees each from the South Central, Pacific, and Mountain regions, and one grantee from the North Central and Atlantic regions.

Because state education agencies (SEAs) are not involved with the administration of the Title VI Grants Program, the study team is not recommending states as a key variable for sampling purposes. However, the study team will sample at least one grantee from each of the three states to which the most funding flows: Oklahoma ($24.3 million), Alaska ($13.4 million), and Arizona ($10.5 million). Moreover, because Alaska is a unique location and includes both very large and very small grantees, the study team will sample two grantees from Alaska.

All of this information on the distribution of grantees by award size, grantee type, and region informs a preliminary sampling framework (Exhibit 5). For example, because 46 of the 58 BIE-operated grantees (79 percent) are located within the Mountain region and because BIE-operated grantees tend to receive smaller awards, the study team will select a small BIE-operated grantee from the Mountain region. In contrast, single-LEA grantees are common across all regions and award size categories.

Exhibit 5. Potential sampling framework based on grant size, grantee type, and region

	LEA (single)	LEA (consortium)	BIE grant and contract	BIE operated	Tribe in lieu of LEA(s)	Total
Pacific	1 large award 1 medium award	1 large award				3
Mountain				1 small award	1 large award	2
North Central			1 medium award			1
South Central	1 large award 1 very small award					2
Atlantic	1 small award					1
Total	5	1	1	1	1	9

Once a list of potential grantees for the case studies has been identified using the steps described above, the following additional information will help further refine the selection process.

Consider demographic information

Having confirmed a basic sampling framework, the study team will hone in on specific grantees by considering both the concentration and number of AI/AN students.

Concentration of AI/AN students. To ensure variation across the sampled LEA grantees with respect to the concentration of AI/AN students, the study team will consider the percentage of AI/AN students (see Exhibit 6). According to definitions provided by the U.S. Office of Indian Education,³ “high-density American Indian schools” are defined as non-BIE public schools with 25 percent or more American Indian student enrollment.
Number of AI/AN students. The study team will also consider the absolute number of AI/AN students served in combination with the concentration of students served.

Exhibit 6. Distribution of LEA grantees by concentration of AI/AN students

Concentration of AI/AN students	Total number of LEA grantees	Percent of total
Low (under 5%)	342	31%
Medium (5%<25%)	340	31%
High (25% or more)	409	37%
Total	1,091	100%
Exhibit reads: Of the 1,091 Title VI Grants Program LEA grantees, 342 (or 31 percent) enroll a population including less than 5 percent AI/AN students. Notes: (1) The total number of LEA grantees does not match above tables because some did not have NCES codes with a match in the Common Core of Data. (2) Percentages do not add to 100 because of rounding. Source: EASIE Budget Report and Common Core of Data.

Consider selection of grantees by project-level characteristics

Finally, the study team will consider the following project-level characteristics.

Grade levels targeted. The study team will try to select a sample of Title VI projects that reflect the variation in the grade levels targeted, i.e., pre-K, elementary school, middle school, and high school (reported on the EASIE report).
Objective(s) and associated activities/services on the Electronic Application System for Indian Education (EASIE). Grantees choose from among 12 potential service objectives and 16 potential services. Selecting grantees that offer some of the more prevalent types of services may be useful (e.g., culturally responsive academic support is the most frequently offered, by 623 grantees). Additionally, it may be useful to select grantees offering specific services that align with the program focus on meeting the cultural needs of AI/AN students (e.g., Indian education, including language and history).

The study team will present a short list of potential grantees for the case studies, using the steps described above, to PPSS and OIE for review and to support final decisions on the sample of case studies sites.

3. Methods to maximize response rates and to deal with issues of nonresponse

The study contractors, SRI International and Policy Studies Associates, have extensive experience in gaining access to districts and schools for research purposes. The contractors partnered with Arizona State University’s (ASU) Center for Indian Education and the University of Alaska Anchorage’s (UAA) Center for Alaska Education Policy to ensure that the study team has a deep understanding of the context for the study, thereby facilitating increased support from local grantees.

As a first step towards maximizing participation in the study, the study team will send the Title VI Grant Coordinators at the administering agencies (LEA, tribe, BIE) a notification letter from the U.S. Department of Education inviting them to participate in the study. The notification letter will have been prepared as part of the OMB clearance package and will include: (1) a study description with a discussion of its importance, purposes, and products; (2) notification that OMB clearance has been secured, and that the Elementary and Secondary Education Act requires grantees to have given an assurance of cooperation with program evaluations; ; (3) specific information on the data collection schedule and plans; (4) provisions for maintaining anonymity of participants and data security; (5) the organizations and staff involved in the study; and (6) the benefits to be derived from the study. The letter will encourage Title VI Grant Coordinators to respond to participation requests and include names and contact information of Department staff and study team leaders who can answer questions about the study. For the nine grantees selected for the case studies, the notification letter will invite grantees to participate as a case study site, provide more detail on the planned site visit, and inform the Title VI Grant Coordinators that a study team leader will follow up by phone to provide additional information and respond to any questions or concerns. (See Appendix A for notification letters.)

Before receiving OMB clearance, the Department’s Office of Indian Education (OIE) will provide the study team with a final list of Title VI Grant Coordinators’ names and contact information. Immediately upon receiving OMB clearance, the study team will mail the notification letters.

Survey data collection and follow-up. One week after mailing the notification letters to local grant coordinators, the study team will begin survey data collection by sending an email to the coordinators, requesting their participation in the online survey via a unique link.

A week after survey launch, the study team will begin following up with nonrespondents by email. The email will remind respondents of the survey due date and invite them to contact the survey administrator with any questions or concerns. The study team will continue following up with nonrespondents via email approximately once a week for three weeks. For persistent nonrespondents, the study team will follow up by telephone. The study team anticipates successfully recruiting approximately 85 percent of Title VI Grant Coordinators to respond, ensuring that data can be reliably generalized to the universe of grantees.

Case study data collection and follow-up. One week after mailing the notification letters to local grant coordinators whose sites were selected for the case studies, a study leader will follow up by phone to discuss the planned site visit. Study leaders will encourage participation by providing useful and clear background information, emphasizing the importance and value of the study to inform educators, policymakers, parents, and other key stakeholders, and being readily available to answer questions. If the Title VI Grant Coordinators agree to participate, researchers will request information on gaining access to the site for fieldwork. If a grantee declines to participate, the study team will identify a replacement site (relying on the same criteria) to ensure a 100% response rate and a total of nine case study sites.

The study team recognizes and will adhere to formal tribal government human, cultural, and intellectual protections (similar to IRB processes) as part of a culturally responsive approach to working with AI/AN populations. The study leader will also work with the Title VI Grant Coordinators to identify respondents for case study interviews and focus groups. The study team will send letters to key staff, parents, and tribal leaders that include (1) a study description with a discussion of its importance, purposes, and products; (2) specific information on the data collection schedule and plans; (3) provisions for maintaining anonymity of participants and data security; (4) the organizations and staff involved in the study; (5) the benefits to be derived from the study; (6) notification that OMB clearance has been secured and/or that all legal and ethical requirements will be followed. To maximize participation, the study team will work with the Title VI Grant Coordinators to develop a site visit schedule that accommodates respondents’ schedules and minimizes burden.

The study team will take the following steps to maximize the accuracy of the data collected for the case studies. Site visitors will attend detailed training and will review background information before planning their visit to ensure efficient, consistent, and accurate use of the data collection protocols. Furthermore, all interviews (subject to the permission of the respondent) will be recorded to improve the accuracy of reporting. Accuracy of analysis will be further facilitated through the use of qualitative data analysis software (e.g., Dedoose) that will support reliability across those involved in coding responses.

4. Tests of procedures or methods to be undertaken to minimize burden and improve utility

The study team pilot tested the survey with seven local grantees who were purposively selected from the study sample to include coordinators of diverse Title VI projects. An expert member of the study team debriefed each pilot test participant to ensure that all questions are clear and are measuring the concepts the study intends. In addition, the pilot test provided accurate information on the length of the survey and informed decisions about fine-tuning, adding, and deleting questions.

5. Names and telephone numbers of individuals consulted on statistical aspects of the design and the names of the contractors who will actually collect or analyze the information for the agency

Agency

Dr. Jean Yan in the Policy and Program Studies Service, Office of Planning, Evaluation and Policy Development, at the U.S. Department of Education is the Contracting Officer’s Representative for the study. She can be reached at 202-205-6212.

Contractors

Dr. Brenda Turnbull of Policy Studies Associates is the Project Director; she can be reached at 202-939-5324. Dr. Katrina Woodworth of SRI International is the Deputy Project Director; she can be reached at 650-859-5256.

1 EASIE Budget Report, “Indian Education Formula Grant EASIE Budget Report: OIE-XLS-8.”

2 National Assessment of Educational Progress (NAEP). National Indian Education Study, Regional and State Summary. Retrieved from http://nces.ed.gov/nationsreportcard/nies/nies_2011/statereg_sum.aspx.

3 R. Moran, B. D. Rampey, G. Dion, and P. Donahue, 2008, National Indian Education Study 2007 Part I: Performance of American Indian and Alaska Native Students at Grades 4 and 8 on NAEP 2007 Reading and Mathematics Assessments (NCES No. 2008-457). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics. Available at http://eric.ed.gov//?id=ED501263.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Deborah Jonas
File Modified	0000-00-00
File Created	2021-01-22