NSTCS OMB Part B Final

SUPPORTING STATEMENT

2014 National Survey of Tribal Court Systems

B. Collection of Information Employing Statistical Methods

1. Respondent Universe and Selection

Defining the Tribal Courts Universe

The 2014 National Survey of Tribal Court Systems (NSTCS-14) will be a census of tribal court systems. A standard and accepted definition of tribal court systems needed to be established prior to the distribution of the survey instrument. The Bureau of Justice Statistics and the data collection agent, Kauffman & Associates, Inc. (KAI), assembled an expert panel comprised of individuals with knowledge and expertise in the area tribal justice systems and law to assist in the development and implementation of this project.

In May 2012, the expert panel met in Silver Spring, MD and Washington, DC, to discuss topics including: an operational definition of tribal courts, developing a universe of tribal courts, the purpose of the NSTCS-14, and the benefits to participating tribes (see Attachment 4). Through these discussions, the panel agreed that a tribal court system mirrors the definition put forth by the Indian Tribal Justice Technical and Legal Assistance Act of 2000¹, which states that:

The term “tribal court,” “tribal court system,” or “tribal justice system” means the entire judicial branch, and employees thereof, of an Indian tribe, including, but not limited to, traditional methods and fora for dispute resolution, trial courts, appellate courts, including inter-tribal appellate courts, alternative dispute resolution systems, and circuit rider systems, established by inherent tribunal authority whether or not they constitute a court of record. ²

This definition excluded Code of Federal Regulations Courts (CFR courts) operated by the Bureau of Indian Affairs (BIA) and courts operated by the state or federal government. However, recognizing that many tribes are serviced by CFR courts, the panel agreed that CFR courts in operation during the collection period should be incorporated.

Identifying Courts within the Tribal Court Universe

In order to distribute the survey to all eligible tribal courts, BJS used the established definition of the tribal court system to create the universe of tribes that fit the definition. The federal government currently recognizes 566 American Indian and Alaska Native (AI/AN) tribes and villages. The precise number of tribal courts is unknown because some tribal councils may act as judicial forums and some tribes share courts and may change alignments periodically. In order to capture the entire tribal court universe, BJS utilized the following five-step strategy to identify tribal courts nationwide:

Step 1. Create database of tribal courts using existing resources

BJS consolidated the five tribal court lists from available sources to develop a comprehensive list of tribal courts:

1. 2002 Census of Tribal Justice Agencies Data File provided by Bureau of Justice Statistics³

2. United States Tribal Court Directory, 4^th Edition⁴

3. 2012 Alaska Tribal Court Directory developed by the Alaska Legal Services Corporation⁵

4. List of Tribal Courts provided by the Tribal Law and Policy Institute (TLPI)⁶

5. List of Tribal Courts funded by the Department of Interior, provided by the Bureau of Indian Affairs (BIA)

The five lists were sorted to remove duplicates. When duplicates were present with conflicting information, priority was given to the most current sources. For tribal courts missing key contact information, KAI (on behalf of BJS) conducted internet searches to and/or followed up directly with those tribes to request the missing information. Through this process, BJS identified 426 AI/AN tribal courts known to be available.

Step 2. Confirm tribal court contact information with tribes

In February 2012, KAI (on behalf of BJS) sent a letter signed by the BJS Director to the tribal leaders of all state- and federally recognized tribes in the continental United States and Alaska (n=583). The letter served purposes: 1) the letter introduced the NSTCS-14 project; and 2) the letter served as a way to confirm the existence of the tribes’ court. The letter asked the tribes to confirm the tribal court name, address and contact information. The letter could be returned via email, fax, or an included self-addressed stamped envelope (see Attachments 6-11).

Step 3. Confirm contact information with non-responsive tribal courts

For courts that did not respond to the initial verification mailings, KAI (on behalf of BJS) cross-checked their contact information against the National American Indian Court Judges Association’s (NAICJA) National Directory of Tribal Justice Systems.⁷ NAICJA is a national nonprofit organization comprised of tribal justice personnel and other staff dedicated to improving tribal justice systems throughout the United States. The Directory is an online database that includes contact information for all known tribal justice systems. Tribal Court Universe data was cross-checked with the NAICJA directory; if contact information in the Universe matched NAICJA’s National Directory, contact information was considered confirmed. If a tribal court’s information did not match, KAI directly contacted the court to confirm its information. The tribal court universe was also cross-checked against the NAICJA’s database to account for all existing tribal courts. When discrepancies arose, KAI directly contacted the tribal court to confirm its existence and information. Through this process, KAI confirmed the contact information for all of the known 426 courts or judicial forums. The total number of entities that will receive the NSTCS is 595. This number includes each federally recognized tribe and the current number of operating CFR courts.

Table 1. Number of the Tribal Court Respondents, by location or court type.

Type of Tribal Court (and Related Court Survey)	Tribal Court Universe (Total N = 426)	NSTCS-14 Distribution (Total N = 595)
Lower 48 Tribal Court (NSTCS-14L48)	247	336
Alaska Tribal Court (NSTCS-14AK)	150	230
Code of Federal Regulations Court (NSTCS-14CFR)	29	29

To improve the initial draft survey instrument, KAI (on behalf of BJS)—with guidance from the tribal justice experts and BJS—identified and created a tribal court judges panel to review the draft instrument. The tribal court judge’s panel consisted of 13 tribal court officials—including court judges, directors, and court program managers from various tribes throughout the United States (see Table 2).

Table 2. Tribe or Court Representation at Panel Meeting (with State/Region)

Panel Attendance by

Tribe or Court

Tulalip Tribes (WA)

Eastern Band of Cherokee (NC)

Intertribal Court of Southern California (CA)

Tesuque Pueblo (NM)

Organized Village of Kake (AK)

Sisseton-Wahpeton Oyate of the Lake Traverse Reservation (SD)

Citizen Potawatomi Nation (OK)

Bristol Bay Native Association (AK)

Rosebud Sioux Tribe (SD)

Pokagon Band of Potawatomi (MI)

To’Hajiilee Tribe (NM)

Central Council of Tlingit and Haida Tribes (AK)

The tribal court judge’s panel met in Rapid City, SD on August 21 and 22, 2012, to review and discuss the revised draft of the NSTCS (see Attachment 5). Panel members received hard-copies of the survey instrument via e-mail before the meeting and were asked to review its content; identify problematic questions; identify information that may be difficult to obtain; and provide recommendations for improvements; . While gathered at the meeting, the leaders were then asked to complete a Question Content Rating Form which ranked each question on a scale from one to five (with one being poor and five being great) based on the following criteria:

• The question is easy to understand

• The question is important to the section being measured

• It is easy to gather information for this question

The leaders were also asked to make comments and provide feedback on the general content and question relevance to court operations and administration. Panel member comments and ratings were incorporated into the survey instrument, where applicable. With the assistance of the tribal justice experts supporting the project, BJS incorporated the panel’s comments to help develop the draft survey instrument.

2. Procedures for Collection of Data

The procedures for collecting data for the NSTCS-14 involve several major components to collect timely and accurate data on tribal court systems, including a series of mailings and non-response follow-up activities, emphasizing questionnaire completion that will be monitored via a secure web-based reporting system. Once finalized and approved by OMB, KAI will distribute the NSTCS-14 to all identified tribal courts and to the tribal governments for which the existence of a justice system is unknown. The three versions of the survey—the NSTCS-14L48 (see Attachment 1), the NSTCS-14AK (see Attachment 2), and the NSTCS-14CFR (see Attachment 3)—will be sent to the tribes based on their location (the lower 48 states or Alaska) and/or the type of court system currently in operation (CFR courts). Each of these components is described below.

Obtaining data on tribal court systems

Phase 1: Send a preview announcement letter and email. Approximately 2-3 weeks prior to the distribution of the survey, an initial email and paper-based letter on BJS letterhead containing the BJS Director’s signature will be sent to each individual tribal court point of contact (see Attachment 12). The email and letter will provide a brief introduction and explain the purpose of the NSTCS-14 and the anticipated distribution date. A brochure will also be included with the pre-notification letter. The brochure offers tribal courts a preview of the types of questions contained in the NSTCS-14 based on content from the NSTCS-14L48, NSTCS-14CFR or NSTCS-14AK survey instrument (see Attachment 13). The brochure distributed will be determined by the location and type of court operated by each individual tribal court.

Phase 2: Send announcement letter and email, with appropriate hardcopy and electronic versions of the survey. Approximately 2-3 weeks after completion of the Phase I mailings, KAI will disseminate the NSTCS-14 to all tribal courts listed in the Tribal Court Universe, using the address and point of contact information collected. An announcement email and paper-based letter on BJS letterhead containing the BJS Director’s signature will be sent to each individual tribal court point of contact (see Attachment 14).

The email and letter will reintroduce the NSTCS-14, explain the purpose of the survey, and provide directions for the three submission methods of completed surveys:

• US Postal Mail (using a pre-paid business-reply envelope),

• Fax, or

• Email.

The letter and email will also contain a toll-free number (877-528-4025) and e-mail address ([email protected]) that respondents may contact should they have any questions or need for assistance. As there will be three versions of the survey (i.e., the NSTCS-14L48, the NSTCS-14AK, and the NSTCS-14CFR), tribal courts will be sent the appropriate survey instrument based on their location (the lower 48 states or Alaska) and the type of court system currently in operation (CFR courts). Each survey contains specific directions, references, definition clarification and necessary instructions to guide respondents through skip patterns.

The emailed survey announcements will provide an attached electronic copy of the appropriate survey (in PDF form) that recipients can download and complete. A delivery and read receipt/notification will be included with all emails. Past experience with tribal court data collection suggests that some courts will need or prefer a paper-based data collection. For this reason a hard copy of the appropriate survey will be sent with the paper-based letters to each tribal court along with a prepaid business-reply envelope.

The various Native American organizations that participated on the expert panel involved in the development of the NSTCS-14 have agreed to either provide letters of support for the NSTCS-14 or assist in any nonresponse follow-up. A letter(s) of support for the NSTCS-14 will be enclosed with both the announcement letter and email. These letters will emphasize the support by the various tribal organizations for the NSTCS-14, highlight the vital importance of the survey, and advocate for the tribal courts full cooperation in the timely completion of the NSTCS-14 instrument. The following Native American organizations that have regular contact with and work within Indian country have partnered with BJS for the NSTCS-14 project:

• The National Congress of American Indians (NCAI)

• The National American Indian Court Judges Association (NAICJA)

• The Tribal Law and Policy Institute (TLPI) (see Attachment 15)

• The National Tribal Judicial Center

• The Center for Indigenous Law, Governance & Citizenship at Syracuse University

• American Indian Development Associates (AIDA)

• The Tanana Chiefs Conference of Alaska

Phase 3: Contact survey participants by telephone and email to confirm receipt of appropriate hardcopy and/or email version of the survey. KAI’s data collection specialists will be assigned a specific portion of the tribal courts. These specialists will serve as the KAI-based points of contact for those tribal courts. Specialists will reach out to the point of contact for each respondent in their assigned area approximately one week after the Phase 2 survey mailing by telephone and email, following designated scripts when speaking with court contacts (see Attachment 16). This will serve to verify receipt of materials, answer questions, determine and attempt to resolve potential problems with timely submission, and prompt courts for questionnaire completion. If the survey has not been received, the court contact will be asked to confirm the mailing and email contact information for the tribal court. This information will be updated within the tribal court database and a hardcopy and/or electronic copy of the survey will be resent to the corrected mailing or email address (along with the endorsement letter and a prepaid business-reply envelope). Contact information (email and toll-free telephone number) for questions and comments regarding the survey will also be provided. A delivery and read receipt/notification will be included with all emails.

Phase 4: Send second wave of hardcopy and electronic versions of the survey. Survey participants will be informed that they are encouraged to respond to the initial dissemination of the NSTCS-14 within 3 to 4 weeks following receipt of the survey instrument. After this time, KAI will send a second wave of the survey to each individual tribal court point of contact. The email and paper-based letter will repeat the directions for submitting the completed survey instrument and again stress the goals and purpose of the survey effort (see Attachment 17). Contact information for questions and comments regarding the survey will also be included. A delivery and read receipt/notification will be included with all emails. Throughout this period, KAI data collection specialists will maintain regular telephone and email communication with tribal court points of contact to confirm receipt of appropriate materials and to provide support and guidance to assist tribal courts toward successful completion of the NSTCS-14.

Verifying and validating the submitted data

Upon receipt, each questionnaire will be reviewed and edited, and, if needed, the data provider will be contacted to clarify responses or provide missing information. Prior to contacting the respondent, KAI will attempt to address data various inconsistencies. KAI will also ensure that responses fall within the proper coding schemes specified by BJS.

3. Response Rate Maximization and Non-response Issues

Methods to Maximize Response Rates

BJS anticipates a 90% or greater response rate for the National Survey of Tribal Courts data collection. BJS and the KAI will undertake various steps to ensure that high response rates are achieved for all versions of the NSTCS-14 survey.

KAI also has internal subject matter expertise familiar with the past and current legislation and case history that has shaped the face of tribal court systems operating throughout Indian Country. These experts include a former appellate judge for the Northwest Intertribal Court System. These subject matter experts will provide KAI project staff (including data collection staff) with training on tribal court structure, relevant federal policy affecting tribal court operation and administration, and best practice strategies when reaching out and interacting with tribal court personnel. They will also serve as an internal resource for KAI staff for additional direction, communication with and guidance on outreach and collaboration with tribal court contacts.

To maximize survey response rates, both electronic and hard copy versions of the NSTCS-14 will be made available to respondents. BJS is taking this multi-mode approach because the limited or sporadic nature of internet connections in some rural and isolated communities may make access to an electronically delivered survey difficult for some respondents. For example, a help desk will be provided for both substantive and technical assistance. KAI data collection specialists will be provided with training to enable them to assist tribal courts in the completion of the NSTCS-14 instrument. KAI staff will be assigned to specific areas which will allow them to become more familiar with the respondents.

The NSTCS-14 project will also receive additional support from the Tribal Law and Policy Institute (TLPI), a non-profit organization with deep roots in the tribal justice community; and Joseph Little, a recently retired Associate Director for the Division of Tribal Justice Support at BIA (and current Associate Judge for the Pueblo of Isleta Tribal Court in New Mexico) who has worked with the 184 tribes supported by the BIA and has served extensively as litigator in tribal matters. Additional support will be sought from the agencies and organizations who publically supported the NSTCS-14 during the advance mailing. In instances where KAI data collection specialists experience repeated difficulty eliciting a response from tribal courts, representatives from TLPI, these supporting organizations or Mr. Little will make direct contact with the tribal courts. Consultants will be brought in as needed to provide additional support for outreach efforts to tribal courts throughout Indian Country.

Additionally, KAI is able to maximize response rates because the organization has professional ties within tribal communities, thereby facilitating cooperation. KAI is an American Indian woman-owned and operated business with extensive experience and professional ties with tribal communities. Through its relationships in and reputation throughout Indian Country, along with the work of its team members, KAI is uniquely positioned to successfully ensure at least a 90% response rate from the tribal courts. KAI has conducted previous work with tribal court systems operating in the United States and is familiar with their structure, personnel and stakeholders involved in the operation of tribal court systems, as demonstrated by its previous project work including:

• The Tribal Court and CFR Court Review Tribal Court Reviews – District VIII, Portland, OR (Office of Tribal Justice Support, Office of Justice Services, Bureau of Indian Affairs, U.S. Department of Interior): KAI is currently contracted with the Tribal Justice Support (TJS) Division of the Bureau of Indian Affairs (BIA) to conduct court assessments in the Idaho, Oregon, Washington, and Metlakatla, AK. This contract calls for KAI to conduct 6 tribal court assessments within the current performance period. The court assessments consist of several phases including an initial meeting with the tribal councils, a 3-day court assessment where KAI meets with several key stakeholders within the tribal court systems, a court assessment report that combines the information KAI gathers for each tribal court assessment, and a presentation to each of the tribal councils to proffer the final court assessment. KAI has conducted five initial meetings with tribal councils, three 3-day court assessments, and submitted one court assessment report for review by TJS. KAI is currently on schedule to complete all phases of the 6 contracted court assessments by the conclusion of the performance period.

• The National American Indian Court Judges Association (NAICJA) 2007-2012 Strategic Plan: In 2007, KAI (under contract to NAICJA) developed the organization’s 2007-2012 national Strategic Plan. Activities included two planning sessions with 10 representatives from individual tribal courts throughout Indian Country, execution of the plan and a period of review and comment prior to the plan’s completion.

• Interdepartmental Tribal Justice, Safety and Wellness Session (DOJ, OJP): In 2009, KAI provided administrative, consultative and logistical support to the OJP to coordinate and facilitate a series of its Tribal Justice, Safety and Wellness Sessions. The sessions were designed to improve tribal justice systems and enhance their overall tribal community public health and safety, and served as a collaborative effort involving planning, consultation, training, technical assistance, and evaluative components of several federal agency partners, national and regional tribal organizations, individual tribes, state and local governments, and other organizations. Over 930 participants took part in these sessions (four sessions in total).

• Logistical Support Services for the Office of Violence against Women’s Annual Tribal Consultation (DOJ, Office of Violence against Women): The Violence against Women Act of 2005 requires the U.S. Attorney General and the Secretary of the Department of Health and Human Services (DHHS) to hold an annual consultation with Indian tribal governments on the subject of violence against women. KAI’s role provided logistical support to the OVW to reach out to tribal governments to help facilitate the consultation meeting.

Survey Nonresponse

Non-response Follow-up, NSTCS-14L48 Strategies: KAI will send hardcopy letter and/or email to all survey recipients who have not yet responded to first two waves of surveys or to wave of telephone and/or email outreach. Approximately three weeks after dissemination of the second wave of the surveys is completed, KAI data collection specialists will send fax and e-mail reminders to all remaining non-responding tribal courts. The fax and e-mail reminders will attempt to gain the respondent’s attention by stressing the importance of the study and prompting the return of the questionnaire (see Attachment 18). A delivery and read receipt/notification will be included with all emails.

In cases where possible, KAI will utilize assistance from agencies and organizations that have publicly stated their support for the NSTCS-14 data collection effort. (See Section 2: Procedures for Data Collection, Phase 2 for more detail.) A request will be made for a representative from one or more of these advocate groups to intervene and re-establish communication with the non-responsive court via email or telephone. In cases where tribal courts remain non-responsive despite agency intervention, consultants will be utilized in an effort to make contact with these courts (via email, telephone or in-person visitation).

To further increase the survey response rate, all remaining non-responsive survey recipients will be contacted by KAI data collection specialists via telephone approximately 2 to 3 weeks after completing the third outreach attempt (see Attachment 19). This “last chance” outreach will alert respondents of the scheduled data collection end date. A last chance contact has been implemented on previous studies, and it serves to motivate non-responders who had failed to complete and return the survey. Tribal court contacts will be asked to confirm if the survey has been completed and/or submitted. Efforts to identify barriers preventing survey completion and assist in the completion of the NSTCS-14 will be made during this outreach. When contacting non-responsive participants during this final phase, KAI will work with tribal court contacts to obtain responses for the survey items deemed critical for completion by BJS (see Table 3).

Table 3. Critical NSTCS-14 Items (by Survey Type)

Survey (by Type)	Critical Items (by Section)
NSTCS-14L48	Sections A, B, C, D
NSTCS-14AK	Sections A, B, C, D
NSTCS-14CFR	Sections A, B

Non-response Follow-up, NSTCS-14CFR Strategies: Initial non-response outreach to recipients of the NSTCS-14CFR will mirror the steps undertaken for non-response follow-up with NSTCS-14L48 survey recipients: letter, fax, and/or email communication reminding CFR courts to complete the survey sent approximately three week after the second wave of survey dissemination. A single KAI data collection specialist will be assigned responsibility for conducting outreach (including all non-response follow-up) with the 29 CFR courts in the NSTCS-14 participant sample.

Additional support for outreach will be sought from contacts from agencies who provided support or letters of support during the advanced mailing, particularly the Bureau of Indian Affairs. Contacts from these agencies will be used to establish communication with a point of contact for an individual CFR court. The designated KAI data collection specialist will then work with the CFR court point of contact to support the court’s completion of the NSTCS-14CFR. This includes working with CFR court contacts to obtain responses for the survey items deemed critical for completion by BJS (see Table 3) during a “last chance” outreach phase following three separate outreach attempts.

Non-response Follow-up, NSTCS-14AK Strategies: Outreach and non-response follow-up in Alaska present unique data collection challenges including remote location/lack of physical access to courts taking part in the survey, language barriers (for tribes and villages that speak a language other than English), and infrastructure issues (lack of Internet access, inconsistently reliable telecommunications or postal service, or conflict with subsistence hunting schedules). In anticipation of these challenges, KAI and BJS will consult with supporting agencies such as the Bristol Bay Native Association and the Tanana Chiefs Conference to (1) develop an effective communication and outreach plan for survey distribution to Native villages throughout Alaska prior to the survey’s dissemination and (2) to engage the services of consultants in the region to assist in non-response follow-up activities.

In cases where repeated attempts at contact with NSTCS-14AK respondents have been unsuccessful, these consultants will assist KAI data collection specialists by serving as the points of contact for data collection with area tribes and villages. This includes providing on-the-ground support during “last chance” outreach phase efforts. During this last non-response follow-up phase, KAI data collection specialists and consultants will contact (via telephone or in-person) Alaska tribal court contacts to obtain response for survey items deemed critical for completion by BJS (see Table 3).

Despite the best efforts made during data collection, however, some data will not be collected. There are two major types of nonresponse: “unit,” when no data are collected for a tribal court system, and “item,” when some questions are answered but others are left unanswered. KAI will use imputation and weighting adjustments if needed to address concerns of unit and item non-response bias in the estimates. In addition, for those respondents that refuse or fail to fully complete the questionnaire, BJS has directed KAI to follow-up with non-responsive tribal courts to capture data for selected critical items, as a last resort. The survey sections on tribal justice systems, court administration, appellate courts, and operations have been identified as critical sections. Within these sections, certain questions like court staffing, budget/expenditures, and caseloads for criminal and civil offenses will be the focus.

Although not ideal, reducing the number of items and respondent burden will allow for collection of the most important data items from those respondents who previously could not complete the entire survey because of time and/or reporting constraints. Critical items for the NSTCS-14 will include information on court staffing, budget/expenditures, and caseloads for criminal and civil offenses. In 2002, BJS excluded the Alaska tribes and the CFR courts. It is anticipated that these will be challenging areas and require additional focus. The use of a questionnaire specifically geared toward these respondents will help ensure a higher response rate.

BJS non-response follow-up plans are designed to give survey respondents the opportunity to complete the NSTCS-14 at a pace consistent with their own day-to-day workloads. In order to ensure the desired response rate, KAI data collection specialists will follow up with regular emails, postal reminders, and phone calls to urge tribes and villages to respond:

Response Tracking System

In order to promote item completion, KAI will monitor item responses on all surveys as they are completed and submitted. KAI staff will utilize a survey response management system linked to the www.tribalcourtsurvey.org website designed to track incoming survey responses on a rolling basis, flagging non-response items and invalid responses as surveys are completed. Data collection managers will oversee personal telephone or e-mail contacts with individual respondents to clarify missing or invalid responses and to take corrective action. These changes will also be tracked for follow up, if necessary. Three full-time staff members at KAI, in addition to the project manager, will have primary responsibility for the response follow up. They will be supplemented on an as-needed basis by one senior KAI researcher, and the BJS project manager.

Non-response Bias Assessment and Imputation

Due to the three distinct NSTCS-14 surveys for Lower 48 (L48), Alaska (AK) and Code of Federal Regulations (CFR) courts, all nonresponse adjustments would be done within each survey group. Overall, BJS anticipates a relatively high response rate for the Lower 48 survey, but recognizes that there is a possibility we may have to do some non-response adjustments. BJS’s initial study of the Lower 48 tribal justice systems garnered a relatively high unit response rate. In 2002, over 92% (314) of the 341 federally recognized American Indian tribes located in the lower 48 States responded to in the Census of Tribal Justice Agencies (CTJA). However, the new Lower 48 survey asks more in-depth items, which may reduce some item response rates. It is highly likely that we will receive complete reporting from the Code of Federal Regulations (CFR) courts. These are courts managed by the Bureau of Indian Affairs and BIA is very interested in the results of this work. In comparison, BJS has little information to judge the likely response rates of the Alaskan tribes. The tribes in Alaska were excluded from the 2002 CTJA, so we have no direct evidence of the likely response rate from these (mostly) villages.

BJS plans to conduct non-response bias adjustments using known external sources of information about the tribes for the Lower 48 survey subjects. Multiple factors will be used to stratify the tribes in the Lower 48 survey to develop imputation weights and values. The stratification factors will include the following items:

• State in which tribes are located based on PL 280 status

• Population of the tribes using the 2013 American Indian Population and Labor Force Report published by the Bureau of Indian Affairs (BIA).

• Law enforcement agency characteristics derived from the BJS Census of State and Local Law Enforcement Agencies (CSLLEA) that will include data on about 180 tribal law enforcement agencies.

• Jail characteristics for the tribes that report data to the BJS annual collection Jails in Indian Country on the population on inmates held in jails or detention facilities.

• Reported crime statistics from about 158 tribal law enforcement agencies that annually report to the FBI’s Uniform Crime Reporting Program.

As mentioned, we do anticipate challenges gathering information from some Alaskan tribes, especially those that may be in very remote regions of the state. For Alaska, BJS has enlisted the Bristol Bay Association to assist with non-response follow-up in the region. However, where these efforts prove unsuccessful, BJS proposes to use the known population of the tribes and the 12 Alaska Native Corporations regional designations to stratify the tribes to develop imputation values for similarly situated tribal villages that do not respond. The 12 Alaska Native Regional Corporations were established in 1971 to administer tribal land claims. These twelve geographic regions were created as far as practicable of Natives having a common heritage and sharing common interests. Imputation of missing data for the Alaskan survey will be done within each of these 12 regions.

Alaska Native Corporation regional designations.

4. Procedural and Methodological Testing

Testing of Procedures

The questions for the initial version of the NSTCS-14 were derived from a variety of sources, including existing survey instruments and consultations with subject matter experts. BJS also pursued consultation-related activities as part of the NSTCS-14 effort to obtain and utilize feedback from 13 tribal court officials, as well as TLPI, Judge Joseph Little. Initial survey drafts were also reviewed by different United States Department of Justice agencies, including the Bureau of Justice Assistance, the National Institute of Justice, the Office of Juvenile Justice and Delinquency Prevention, and the Office of Tribal Justice. Input regarding the clarity and accuracy of the survey instrument was sought through follow-up discussions with representatives of these agencies. BJS and KAI reviewed all feedback, and appropriate revisions were incorporated into the final proposed survey instrument.

Following consultations with the expert and tribal leadership panels, as well as discussions with TLPI, Judge Joseph Little, and other sources, BJS determined that it would be inappropriate to create and distribute a single survey instrument to all AI/AN tribal courts and CFR courts. Due to variations in region, tribal structure and federal regulations, tribal court systems differ considerably in terms of operation and administration. Consequently, questions that may be relevant or appropriate for one region or court type may not be for another.

For example, questions about expanded criminal jurisdiction over non-Indians are relevant to tribal courts operating in the lower 48 states. However, tribal courts operating in Alaska—with the exception of the Metlakatla Indian community—are exempt from this expansion; questions regarding expanded jurisdiction are subsequently irrelevant and inappropriate. In order to capture similar data about operations and administration across these varying tribal court systems, BJS created three separate NSTCS-14 surveys: a survey for tribal courts operating in the lower 48 states (NSTCS-14L48), tribal courts operating in Alaska Native villages (NSTCS-14AK) and CFR courts (NSTCS-14CFR). Each survey is tailored to reflect characteristics unique to each type of tribal court system, while still managing to capture information about common tribal court system components.

KAI and BJS sought additional guidance on survey content, respondent burden and design during pilot testing of all three versions of the NSTCS-14 instrument. Piloting of the NSTCS-14L48, the NSTCS-14AK and the NSTCS-14CFR took place during July and August 2013. Working with TLPI as well as panel members, nine pilot sites were identified:

• Five NSTCS-14L48 field test sites located in Montana, New Mexico, California, and Washington;

• Three NSTCS-14AK field test sites located in Alaska; and

• One NSTCS-14CFR field test site located in Oklahoma.

Field test sites were selected to represent a range of geography, court size, and structure. Selected sites also demonstrate the capacity to receive the test survey electronically (confirmed through a functioning email address and/or website). KAI reached out to points of contact from each site via telephone and explained the purpose of the survey. If they expressed interest, KAI sent the contact a follow-up email containing a copy of the NSTCS-14 flyer and requested confirmation of field test participation. Confirmed participants were sent an email containing an electronic version of the appropriate survey (see Attachment 20). The piloted versions of the survey contained directions for completion as well as contact information for the NSTCS-14 Help Desk at KAI.

Debriefing interviews were held with each field test participant following submission of the completed survey. Field testers were asked to describe time needed to complete each survey, questions or sections in need of clarification, and overall impression of the survey participation experience. Some of the findings from the pilot testing phase suggest that:

• Average survey completion times varied by survey type:

  • NSTCS-14L48 = 2 hours, 15 minutes

  • NSTCS-14AK = 2 hours

  • NSTCS-14CFR = 1 hour, 30 minutes

• With the exception of one NSTCS-14AK field test participant, all points of contact required assistance or information from colleagues/other tribal court departments to provide answers for some survey questions.

• No survey questions or sections were recommended for deletion. Participants reported that each section of each survey was relevant to uncovering details regarding tribal court system operation and administration.

KAI compiled findings from the pilot test into a summary report, which was reviewed with BJS during an in-person meeting on August 20, 2013 (see Attachment 21). Based on the initial feedback from the pilot test, BJS significantly reduced the number of questions in each survey version. Also to ensure a high tribal court response rate, modifications to the wording of questions to improve clarity and the inclusion of additional directions guiding points of contact to consult with sources (e.g., law enforcement or finance department officials) in order to obtain information needed to complete the survey in its entirety.

5. Contact Information for Statistical Design Consultants and Data Collection Agency

a. BJS staff contact:

Steven Perry
Prosecution and Adjudication Statistics Unit
Bureau of Justice Statistics
810 7th Street, NW
Washington, DC 20531
202-307-0777

b. Persons consulted on data collection and analysis:

Adrienne Wiley, MSW, MID
Project Manager

Kauffman & Associates, Inc.
1100 Wayne Avenue, Suite 1010
Silver Spring, MD 20910
240-863-0360

Susan Silberman, Ph.D.

Director, Research & Evaluation

Kauffman & Associates, Inc.
1100 Wayne Avenue, Suite 1010
Silver Spring, MD 20910
240-863-0359

c. Persons consulted on statistical methodology:

Jeanette Hassin, Ph.D.
Senior Researcher
Kauffman & Associates, Inc.
1100 Wayne Avenue, Suite 1010
Silver Spring, MD 20910
240-863-0366

Bijal Shah, PhD.

Program Evaluator

Kauffman & Associates, Inc.
1100 Wayne Avenue, Suite 1010
Silver Spring, MD 20910

240-863-0372

Susan Silberman, Ph.D.

Director, Research & Evaluation

Kauffman & Associates, Inc.
1100 Wayne Avenue, Suite 1010
Silver Spring, MD 20910
240-863-0359

Kristen Hudgins, Ph.D.

Research Associate

Kauffman & Associates, Inc.
1100 Wayne Avenue, Suite 1010
Silver Spring, MD 20910
240-863-0366

d. Substantive experts:

Jerry Gardner

Executive Director

Tribal Law and Policy Institute

8235 Santa Monica Blvd., Suite 211

West Hollywood, CA 90046

323-650-5467

Heather Valdez-Singleton

Program Director

Tribal Law and Policy Institute

8235 Santa Monica Blvd., Suite 211

West Hollywood, CA 90046

323-650-5467

Lauren Frinkman

Tribal Law Specialist

Tribal Law and Policy Institute

8235 Santa Monica Blvd., Suite 211

West Hollywood, CA 90046

323-650-5467

Joseph Little

Associate Judge

Isleta Tribal Court

117 Tribal Road 40

Isleta Village Proper, NM 87022

505-869-9699

Thomas P. Keefe, JD

General Council

Kauffman & Associates, Inc.501
S. 165 Howard St., Suite 200

Spokane, WA 99201

509-747-4994

ATTACHMENTS:

1. National Survey of Tribal Court Systems – Lower 48 Survey Instrument

2. National Survey of Tribal Court Systems – Alaska Survey Instrument

3. National Survey of Tribal Court Systems – Code of Federal Regulations Court (CFR Court) Survey Instrument

4. Expert Panel List & Meeting Agenda

5. Tribal Court Judges Panel List & Meeting Agenda

6. Dear Tribal Leader Letter

7. Contact Verification Form

8. Letter to Tribal Courts

9. Screen Shot of Website

10. NSTCS-14 Flyer

11. Follow-up Verification Email

12. Pre-notification Letter and Email

13. Survey Preview Brochures

14. Announcement Letter and Email

15. Tribal Law and Policy Institute Letter of Support

16. Association of Probation and Parole Letter of Support

17. Bristol Bay Native Association Letter of Support

18. 1st Follow-up Script

19. 2nd Mailing Letter and Email

20. 3rd Follow-up Letter and Email

21. Telephone Retrieval and Voicemail Script

22. Email to Field Test Participants

23. 2014 NSTCS-14 Field Test Summary Report

1 Public Law 106-559

2 Tribal justice system, as defined by 25 USC § 3653(8)

3 BJS Census of Tribal Justice Agencies, 2002, see http://www.bjs.gov/content/pub/pdf/ctjaic02.pdf

4 Schwartz, April and Mary Jo Hunter (2011). United States Tribal Court Directory. William S. Hein & Co., Inc., New York. See https://www.wshein.com/catalog/72277

5 Alaska Tribal Court Directory, 2012, see http://alaskatribes.org/uploads/2012-tc-directory.pdf

6 Tribal Court Clearing House, see http://www.tribal-institute.org/lists/tlpi.htm

7 http://www.naicja.org/directory

11