OMB Memo on CTLEA Cognitive Interviews

MEMORANDUM

MEMORANDUM TO: Robert Sivinski

Office of Information and Regulatory Affairs (OIRA)

Office of Management and Budget

THROUGH: Jeffrey H. Anderson, Director

Bureau of Justice Statistics

Jeri M. Mulrow

Principal Deputy Director

Gerard Ramker

Deputy Director

Planning, Policy and Operations Division

FROM: Steven W. Perry, Statistician

Prosecution and Adjudication Statistics Unit

DATE: April 25, 2018

SUBJECT: BJS request for OMB Clearance to conduct cognitive interviewing for the 2018 Census of Tribal Law Enforcement Agencies (CTLEA-18), under the OMB generic clearance agreement (OMB Number 1121-0339).

Introduction

BJS requests approval for a generic clearance to test contact strategies and conduct cognitive testing of the 2018 Census of Tribal Law Enforcement Agencies (CTLEA) questionnaire. The CTLEA is a new BJS data collection proposed to gain a better understanding of the administration and operational characteristics of tribal law enforcement agencies operating in the U.S.

The CTLEA has two primary goals; first, to fulfill the statutory mandate directing BJS to improve its tribal statistical data collections at the federal, state, local and tribal levels,¹ and second, to generate accurate and reliable statistics about tribal law enforcement agencies and the challenges faced in the administration of justice in Indian country or on tribal lands.

Tribal Law Enforcement Challenges and Empirical Data Needs

Tribal justice systems in Indian country vary across tribal nations and regions of the country. Criminal jurisdiction in Indian country—federally recognized reservations, tribal communities, and identified trust lands - varies by the type of crime committed, whether the offender or victim is a tribal member, and by the state in which the offense occurred. Due to the sovereign status of federally recognized tribes in the United States, crimes committed in Indian country are often subject to concurrent jurisdiction among multiple criminal justice agencies, including Federal, State and local law enforcement. Tribal law enforcement agencies enforce tribal laws established under tribal constitutions, codes, or statutes among American Indians, as well apprehend and detain non-Indians for the appropriate Federal, State or local law enforcement agencies.

There are a total of 567 federally recognized American Indian and Alaska Native (AIAN) tribes in the contiguous 48 states and in Alaska, with about 334 federally and state-recognized American Indian reservations or villages.² In 2010, according to the Census, about 4.6 million people lived on American Indian reservations or in Alaska Native villages, with 76% (3.5 million) being described as non-Indians.³ An estimated 1.1 million residents on tribal lands in 2010 were identified as American Indians.

The 567 federally recognized sovereign tribes vary in size both in population and the geographic area of their reservation, trust lands or tribal communities. Tribal communities are as small as the Augustine Band of Cahuilla Indians, with just 11 members on a one-square-mile block of land or as large as the Navajo Nation, with a population of more than 330,000 and a land area of more than 27,000 square miles crossing into four states. Some tribes do not operate their own tribal law enforcement agencies, while others have tribal law enforcement agencies that coordinate their activities with state or federal law enforcement authorities.

The challenges and complexities faced by tribal law enforcement agencies are impacted not only by American Indians and Alaska Natives who reside on tribal lands, but also by the many non-Indian residents on tribal lands. Tribal law enforcement agencies and officers deal with issues of overlapping jurisdiction with Federal and State or local governments; limited financial and staffing resources; the responsibility for patrolling and responding to crimes over large geographic areas and rough terrains; and the disproportionate rates of violence and substance abuse among residents on reservations. ⁴

More than 300 tribes in the United States are located in mandatory or optional Public Law 83-280 jurisdictions. Public Law 83- 280 (commonly referred to as Public Law 280 or P.L. 280) either

transferred mandatory jurisdiction over major crimes in Indian country to states or allowed states to acquire optional jurisdiction in whole or in part over Indian country within their boundaries. In the states where P.L. 280 does not apply, the federal government retains criminal jurisdiction for major felony crimes committed on tribal lands. Tribal law enforcement agencies, retaining concurrent criminal jurisdiction in all circumstances involving American Indian offenders and are generally the first responders to crimes occurring on tribal land, whether the incidents involve American Indians or non-Indians.

In addition, the responsibilities of tribal law enforcement agencies have expanded in recent decades due to changes in violent crime rates and legislative mandates; however, data on their contextual, structural and operational characteristics have remained extremely limited or nonexistent. Several key pieces of legislation that were passed have had significant impact on the attention placed on Indian country violent crime, victimization of women and children, and the performance of law enforcement services. These pieces of legislation granted new authority to tribal justice agencies with respect to domestic violence offenders and jurisdiction over non-Indians; have called attention to sex offenders and rape or sexual assault of Native American women and children; and called attention to tribal crime data deficiencies and lack of reporting regarding crime and law enforcement activities on tribal lands.

The Tribal Law and Order Act of 2010 (TLOA), directed BJS to improve its tribal statistical data collection at the federal, state, local and tribal levels.⁵ The act further requires BJS to consult with Indian tribes to establish and implement a data collection system and to report to Congress within one year of enactment, and annually thereafter, the data collected and analyzed in accordance with the act.

TLOA also created the Indian Law and Order Commission, an independent advisory group to help examine the challenges to securing equal justice for Native Americans living and working on Indian lands. In 2013, the Indian Law and Order Commission in a report to the President and Congress noted that “Indian Tribes and nations throughout the country would benefit if tribal law enforcement agencies were staffed at force levels comparable to similarly situated communities off the reservations.”⁶ No current or publically available data exist about the total number of tribal, federal or state law enforcement officers working on tribal lands. The CTLEA will collect information on total staffing, as well as information on tribal law enforcement agency partnership and coordination with state and local law enforcement agencies, access to and submission of criminal history records to regional and national criminal justice databases, the availability of IT departments or staff, capacity to issue AMBER Alerts, and access to other mobile electronics.

The Violence Against Women Reauthorization Act of 2013 (VAWA) sought to bridge the gaps that occur in Indian country when a non-Indian commits domestic violence against an American Indian or Alaska Native woman.⁷ To help understand the impact initiated by VAWA, the CTLEA will collect information on whether tribal law enforcement agencies made arrests for domestic violence and/or violation of protection orders.

The Sex Offender Registration and Notification Act (SORNA), which is Title I of the Adam Walsh Child Protection and Safety Act of 2006 (Public Law 109-248), provides a comprehensive set of minimum standards for sex offender registration and notification in the US. The CTLEA will collect information about SORNA-related responsibilities that tribal law enforcement provide in tribal communities. The CTLEA will specifically ask whether tribal agencies duties include SORNA registration or have databases to monitor offenders residing on tribal lands.

The Office of the Inspector General at the Department of Justice, recently reported that crime data in Indian country remains unreliable and incomplete, limiting the Department’s ability to engage in performance based management of its effort to implement its Tribal Law and Order Act responsibilities.⁸ The CTLEA will help provide important information to help federal, state and tribal governments and policymakers understand the capacity and needs of tribal law enforcement agencies in order to more effectively improve public safety in Indian country. The CTLEA survey includes questions related to staffing; funding sources; officer training; inter-governmental agreements and coordination with other justice agencies; workloads and duty functions; juvenile and domestic violence matters; and criminal justice information system needs, access and use.

CTLEA Questionnaire Development

To accomplish the CTLEA project goals, BJS implemented a multi-tiered approach to developing the draft questionnaire that will be cognitively tested through this generic clearance. Given the unique nature of this data collection effort and that much of the information of interest was not available from any existing sources, it was necessary to develop new survey questions to collect data on topics most critical to tribes, tribal justice experts, tribal law enforcement agencies and federal agencies. The content, concepts and topics for the CTLEA were developed and prioritized through on-going discussions between various representatives from Federal, State and Tribal agencies that provided input and feedback.

First, during FY 2016, meetings were conducted with grant making components of the Office of Justice Programs and other offices at DOJ, including the Office of Juvenile Justice and Delinquency Prevention (OJJDP), Bureau of Justice Assistance (BJA), National Institute of Justice (NIJ), Office for Victims of Crime (OVC), the Office for Violence against Women (OVW), and Office of Tribal Justice (OTJ). These offices provided suggestions on the type of information needed about tribal law enforcement agencies that could help inform program planning and awards.

Second, during FY 2017, BJS established a tribal justice and law enforcement panel to ensure that tribal governments and their law enforcement agencies had a central role in the development, design, and implementation of the census. BJS hosted a two day panel meeting in Phoenix, Arizona, that included participants from OTJ, the FBI’s Indian Country Crimes Unit, and several US Attorney’s Offices that represent Indian territory in their districts; the Department of Interior’s (DOI) Bureau of Indian Affairs (BIA) Office of Justice Services; the Alaska State Troopers; tribal law enforcement agencies, and Village Public Safety Office coordinators.

Third, during FY 2017, BJS submitted the CTLEA draft survey to the tribal justice panel representatives from Federal, State and Tribal agencies for review. In addition, from October through December 2017, nine eligible tribal law enforcement agencies were selected for the survey development pilot test. Agencies were selected based on geographic location and size of population over which the agency had jurisdiction. Eight of the nine agencies completed the survey and five agencies participated in a debriefing interview. As a result of the initial pilot test, revisions were made to the tribal law enforcement agency survey including the removal of several burdensome questions, specifically types of arrests and counts of incidents investigated by offense category. In addition, instead of requesting information directly from the individual BIA law enforcement agencies, BJS decided to gather this information from the BIA at the Department of Interior.

The CTLEA survey is now ready for expanded cognitive testing among various tribal law enforcement agencies across the country. The current version of the CTLEA survey include sections covering jurisdiction, administration, operational characteristics, agency functions, and information technology. The survey was developed to collect data on the characteristics of tribal law enforcement agencies, including a number of key estimates, such as the number of officers by race and ethnic heritage; sources of funding; officer employee benefits, safety equipment, and training requirements; number of calls for service, arrests and juvenile arrests; types of matters handled including domestic violence, human trafficking and opioid distribution; and agency access to and submission of criminal history information to regional and national criminal justice databases.

Testing Contact Strategies and Cognitive Testing of the CTLEA Survey

BJS is requesting approval to test the contact and follow-up strategies for the CTLEA respondents and conduct cognitive interviews on the CTLEA questionnaire form (CTLEA-18) content under the BJS generic clearance (OMB Number 1121-0339). The federally recognized tribes often have different cultural norms and different languages, therefore it is important to assess the reliability and validity of the CTLEA questionnaire content.

The data collection agent, NORC at the University of Chicago and its partner, the International Association of Chiefs of Police (IACP), Indian Country Law Enforcement section, will coordinate the testing of the CTLEA survey (Attachment 1).⁹ Unlike surveys that have been in the field on numerous occasions or that use standardized questions that have been previously tested, the CTLEA will gather information from tribal law enforcement agencies using questions that have not been presented to tribal law enforcement agencies previously

Specifically, through the cognitive testing BJS aims to: 1) test the CTLEA respondent contact and follow-up strategies; 2) establish an accurate estimate of the respondent burden time to complete the CTLEA; 3) test the reliability of the questions to avoid response error; 4) test the content validity of the questions and verify completeness of the response categories; 5) verify the instruction materials are clear and easy to follow; and 6) document the respondent feedback from the cognitive interviews (Attachment 2) to help make improvements and revisions to the CTLEA, where needed, and establish best practices for future data collection involving tribal justice agencies.

Twenty (20) tribal law enforcement agencies will be selected to participate in the testing of the contact strategy and cognitive interviews for the CTLEA survey. The intended respondent for the CTLEA survey is the agency’s chief law enforcement officer or another designated staff member who is knowledgeable about the administration and operations of the agency.

CTLEA Cognitive Test Methodology

The proposed methodology for the CTLEA respondent contact and cognitive test is divided into four phases: respondent selection, pilot survey collection and cognitive interview, nonresponse follow-up; and cognitive interviews and feedback.

Respondent recruitment and selection process

The project team developed a preliminary list of agencies for contact and cognitive testing based on a set of criteria to maximize the potential variance of agency characteristics. A total of 20 tribal law enforcement agencies will be selected for the CTLEA cognitive test. Agencies selected for the cognitive test will be included based on the following criteria: (1) Public Law 280 and non-PL 280 status; (2) size of residential population served; and (3) geographic area – (North, South, East, or West). The table below presents the characteristics of the 20 tribal law enforcement agencies selected for the pilot activities.

Table 1. CTLEA Cognitive test and interview sample size, 2018

Tribal land resident population size	PL 280	Geographic region	Total agencies
Small (0 – 6000 persons)	Yes	West (2 agencies) North (1 agency)	3
Medium (6001-20,000)	Yes	North (2 agencies) West (1 agency)	3
Large (20,001 – 60,000)	Yes	North (2 agencies)	2
Mega (60,000 and above)	Yes	North (1 agency) West (1 agency)	2
Small (0 – 6000 persons)	No	East (1 agency) South (1 agency) West (1 agency)	3
Medium (6001-20,000)	No	North (1 agency) West (1 agency) South (1 agency)	3
Large (20,001 – 60,000)	No	South (2 agencies)	2
Mega (60,000 and above)	No	West (1 agency) North (1 agency)	2

Survey contact and cognitive interview procedures

To begin, the CTLEA invitation letter (Attachment 3) will be mailed to the twenty sites selected by the IACP Indian Country Law Enforcement Section. This letter will request the agency’s participation in the test, describe the requested information, and stress the importance of participation and benefits of the project to all tribes. The packet will also include the CTLEA questionnaire (Attachment 1) and a pre-paid business-reply envelope (BRE) for respondents to return the completed hard copy questionnaire to the data collection agent. The letter will provide the telephone and email contact information for the BJS Project Officer. The letter and copy of the survey will also be e-mailed to the agency contact.

Nonresponse follow-up procedures and process

Two weeks after the initial mailing, NORC and IACP will conduct one round of telephone outreach to all non-responding agencies to confirm both receipt of the CTLEA form and that the correct agency representative was contacted. Telephone interviewers will also offer respondents the opportunity to complete the survey over the telephone. A copy of the prompting script and frequently asked question is included (Attachment 4). If an agency cannot be reached by telephone during this outreach, NORC will send a reminder e-mail (Attachment 5).

During week 4 of the data collection, a reminder email, fax, or mailing with a replacement copy of the survey will be sent to all non-responding agencies. This mailing will include the follow-up letter (Attachment 6), duplicate CTLEA survey (Attachment 1) and a postage-paid return envelope. If an agency contact has an email address, an email reminder will be sent. If only a fax number is available, a fax will be sent to the agency. The fax/email will provide a different form of contact to gain the respondent’s attention, stress the importance of the study, and prompt for the return of the questionnaire.

As the completed surveys are returned, NORC will schedule telephone calls with the selected sites to conduct the cognitive interviews. The cognitive interviews are expected to last about one hour and will capture feedback on the question clarity, response category options and respondent burden (Attachment 2).

Summarize pilot findings

NORC and IACP will review the pilot survey responses and draft a summary memorandum for BJS which will detail the interview item responses by question, survey burden estimates and any recommendations from the cognitive interview respondents regarding revisions to the survey questions, response choices, question order or presentation.

Respondent Burden Hours

Based on estimates provided by the initial pilot test, BJS estimates that a sample of 20 tribal law enforcement agencies will require a total of 31 hours of respondent burden to complete the CTLEA pilot and cognitive interviews (Table 2).

The estimated respondent burden includes an estimated 30 minutes to complete the pilot survey and 60 minutes for cognitive interview for each of the 20 respondent (20 x 90 minutes = 1800/60 = 30 hours), and 15 minutes for non-response follow-up or validation by 20% of the sample (4 x 15 minutes = 1 hour). More specifically, the estimated burden hours were calculated as follows:

Table 2. CTLEA cognitive test respondent burden

Agency task	Number of agencies	Estimated burden (in minutes)	Total burden hours
CTLEA Pilot survey	20	30	10 hours
Conduct cognitive interview	20	60	20 hours
Nonresponse follow-up and verification (20% of sample)	4	15	1 hour
Total burden			31 hours

Institutional Review Board

The project team obtained a determination of non-human subjects research from NORC’s IRB to ensure the testing protocol is compliant with informed consent and data confidentiality standards.

Contact Information

Questions regarding any aspect of this project can be directed to:

Steven W. Perry

Statistician

Bureau of Justice Statistics

U.S. Department of Justice

810 7th Street, NW

Washington, DC 20531

Office Phone: (202) 307-0777

E-mail: [email protected]

Attachments

Attachment 1: CTLEA Pilot Survey

Attachment 2: CTLEA Cognitive Interview Script

Attachment 3: CTLEA Pilot Survey Invitation Letter

Attachment 4: CTLEA Pilot Survey Phone, Voicemail and Frequently Asked Questions Script

Attachment 5: CTLEA Pilot Survey Follow Up Letter

Attachment 6: CTLEA Pilot Survey Reminder Letter

1 Tribal Law and Order Act, Pub.L. 111–211, H.R. 725, 124 Stat. 2258, enacted July 29, 2010.

2 Bureau of Indian Affairs, “Indian Entities Recognized and Eligible to Receive Service from the United States” Federal Register, Vol. 82, No. 10, January 17, 2017.

3 Census Bureau, “The American Indian and Alaska Native Population,” 2010 (C2010BR-10), January 2012. (Retrieved from Census.gov website, June 7, 2017).

4 Luna-Firebaugh, Eileen. Tribal Policing: Asserting Sovereignty, Seeking Justice. The University of Arizona Press: Tucson, (2007).

5 Tribal law and Order Act, Pub.L. 111–211, H.R. 725, 124 Stat. 2258, enacted July 29, 2010.

6 Tribal Law and Order Commission, “A Roadmap for Making Native America Safer: Report to the President and Congress of the United States” November 2013.

7 VAWA 2013 recognizes tribes' inherent power to exercise "special domestic violence criminal jurisdiction" (SDVCJ) over certain defendants, regardless of their Indian or non-Indian status, who commit acts of domestic violence or dating violence or violate certain protection orders in Indian country. (See http://www.gpo.gov/fdsys/pkg/PLAW-113publ4/pdf/PLAW-113publ4.pdf)

8 Department of Justice, Office of the Inspector General. “Review of the Department’s Tribal Law Enforcement Efforts Pursuant to the Tribal Law and Order Act of 2010.” December 2017. see https://oig.justice.gov/reports/2017/e1801.pdf

9 Willis, Gordon B. Cognitive Interviewing: A Tool for Improving Questionnaire Designs. Sage Publications, London, (2005).

7