Supplelment

MEMORANDUM

MEMORANDUM TO: Chad Lallemand

Official of Statistical and Science Policy

Office of Management and Budget

THROUGH: Lynn Murray

Clearance Officer

Justice Management Division

Amy C. Blasher

Chief of Crime Statistics Management Unit

Federal Bureau of Investigation

FROM: Cynthia Barnett-Ryan

Survey Statistician

DATE: June 23, 2014

SUBJECT: FBI Request for OMB Clearance for cognitive testing of change in mode of collection, the interpretation of “mixed racial group” bias motivation category, and additional ethnic and religion bias motivation categories in the UCR Hate Crime Data Collection under the OMB generic clearance agreement (OMB Number 1110-0057).

The Federal Bureau of Investigation (FBI) is requesting clearance for the cognitive testing of changes to an existing instrument used in the Hate Crime Data collection in order to capture data about hate crimes that come to the attention of law enforcement agencies. The Hate Crime Data Collection is a component of the Uniform Crime Reporting (UCR) Program, which collects statistical information on the counts and characteristics of crime from law enforcement agencies on an annual basis. The results of the cognitive interviews will be used to assess the impact of changing the current paper-based instrument to a web-based instrument along with comprehension of newly introduced categories on anti-Arab bias, anti-Hindu bias, and anti-Sikh bias. In addition, the cognitive interviews will allow for the assessment of the respondent’s comprehension of the long-standing category of bias motivation against a mixed-race group.

The Hate Crime Statistics Program seeks to capture information about the types of biases that motivate crimes, the nature of the offenses, and some information about the victims and offenders. Hate Crimes are not recognized as separate, distinct crimes; instead, they are traditional offenses motivated by the offender’s bias or biases (for example, an offender assaults a victim because of a bias against the victim’s race). Hate Crime data is derived by capturing the additional element of bias in those offenses already being reported to the UCR Program. Attaching the collection of hate crime statistics to the established UCR data collection procedures fulfills the directives of the Hate Crime Statistics Act without placing an additional undue reporting burden on law enforcement.

The current request for approval, under the FBI UCR Generic Clearance (1110-0057), is for cognitive testing of the Hate Crime Data Collection incident report with twenty law enforcement personnel. The cognitive testing will require 20 burden hours.

For this clearance, the UCR Program will conduct cognitive interviews to identify issues with the application of new bias motivation categories for ethnicity (i.e., anti-Arab) and religion (anti-Sikh and anti-Hindu). In addition, the cognitive interview will allow for the opportunity to reveal whether the transition from a paper-based collection instrument to a web-based collection instrument has introduced any problems with the general understanding of the requested information or confusion with the wording of instructions. Finally, the cognitive interviews will allow for an assessment of the current understanding of the application of the bias motivation specified as anti-Multiple Races, Group that has been a part of the data collection since its beginning. The results of the cognitive interviews will allow for the UCR Program to fine tune the language and question structure to be used with the introduction of new categories, as well as the new mode of collection. The information collected on the understanding of the anti-Multiple Races, Group bias motivation will be used for possible future refinements depending upon the findings.

Purpose of the Research

In response to a growing concern about hate crimes, on April 23, 1990, Congress passed the Hate Crime Statistics Act. This law required the Attorney General to collect data “about crimes that manifest evidence of prejudice based on race, religion, sexual orientation, or ethnicity.” The Attorney General delegated the responsibilities of developing the procedures for implementing, collecting, and managing hate crime data to the Director of the FBI, who in turn assigned the tasks to the FBI UCR Program. Under the direction of the Attorney General and with the cooperation and assistance of many local and state law enforcement agencies, the FBI UCR Program created a hate crime data collection system to comply with the congressional mandate.

Lawmakers then amended the Hate Crime Statistics Act to include bias against persons with disabilities by passing the Violent Crime Control and Law Enforcement Act of 1994 in September of that year. The FBI started gathering data for the additional bias type on January 1, 1997. In October 2009, the Matthew Shepard and James Byrd, Jr. Hate Crime Prevention Act amended the Hate Crime Statistics Act under Division E of P.L. 111-84, the National Defense Authorization Act for Fiscal Year 2010. As a result, the FBI UCR Program now captures statistics on hate crimes based on gender and gender identity prejudices, as well as hate crimes committed by or directed against juveniles. In 2012, the UCR Hate Crime Statistics Program made additional system modifications to comply with the Office of Management and Budget’s requirements for the collection of race and ethnicity. In addition, the Criminal Justice Information System (CJIS) Advisory Policy Board (APB) recommended and the Director of the FBI approved that the program allow agencies to report up to four additional bias motivations per offense type. Subsequently, in June 2013, the CJIS APB approved a motion and the FBI Director approved a modification of the UCR Hate Crime Data Collection to add the religions of Mormon, Jehovah’s Witness, Greek Orthodox, Other Christian, Buddhist, Hindu and Sikh as well as an anti-Arab bias motivation.

In April 2012, staff from the FBI Uniform Crime Reporting (UCR) Program and the Census Bureau conducted nine qualitative interviews testing the usability of the UCR Program’s Hate Crime Incident Report at the Maryland State Program Office and the Washington, DC Metropolitan Police Department. Participants included reporting clerks, supervisors, and detectives. Usability issues were discovered throughout the cognitive interviews, which appeared to limit both the validity of certain sections of the form and the overall reliability of the form. These issues were addressed on the most recent version of the Hate Crime Incident Report.

The goals of this cognitive interview phase are to:

• Identify whether the change from a paper-based collection to a web-based Excel Workbook collection introduces any unforeseen issues with comprehension for the respondents;

• Assess comprehension issues associated with the understanding of anti-Arab bias motivation, including whether respondents interpret questions consistently

• Assess comprehension issues associated with anti-Sikh and anti-Hindu bias motivations, including whether respondents interpret questions consistently;

• Assess the current comprehension of the anti-Multiple Races, Group bias motivation, including whether respondents interpret questions consistently;

• Assess the impact of presenting race and ethnicity information in separate lists or in a combined list to aid comprehension;

• Sharpen the instructions regarding the definitions of new categories in the collection final data collection.

Design of Revisions to Current Hate Crime Data Collection

The decision to add bias motivations for anti-Arab, anti-Sikh, and anti-Hindu was based upon the requests of external advocacy groups and legislators that were originally received in 2010. A detailed accounting of the history and groups involved in the development of the new bias motivation categories is provided below:

1. September 10, 2010 – The first letter to the Criminal Justice Information Services (CJIS) Division was received from the Sikh American Legal Defense and Education Fund (SALDEF)

2. October 12, 2010 – CJIS sent a response letter informing SALDEF about the CJIS Advisory Policy Board (APB) Process for policy change and approval.

3. November 9-10, 2010 – SALDEF’s Jasjit Singh addressed the Association of State Uniform Crime Reporting Programs (ASUCRP) and the Uniform Crime Reporting (UCR) Subcommittee providing information on the Sikh religion and requested a motion that anti-Sikh be added to Hate Crime Statistics (HCS).

UCR Subcommittee made a motion to accept the presentation as “information-only.”

4. March 29, 2012 – A letter was received from Rep. Wally Herger (R) California, to Director Mueller requesting crimes against Sikh-Americans be separately identified as a category in the UCR HCS.

April 17, 2012 – Shortly after the receipt of the letter from Rep. Herger, another letter was received which was signed by an additional 93 Congressional Representatives (including Rep. Crowley) containing the same request.

5. June 12, 2012 – The Department of Justice (DOJ) provided an official response stating DOJ’s desire to continue a productive dialogue and learn more from the Congressional staff about their constituents’ concerns.

6. July 16, 2012 – Members of FBI Headquarters and CJIS met with Congressional staffers to discuss CJIS processes and limitations for adding new categories to the HCS. Those are listed as follows:

   1. CJIS APB Process

   2. Process for implementation upon CJIS APB approval

   3. Unfunded legislative mandates

   4. Cost of updating Records Management Systems

   5. Importance of state and local agency buy-in in order to gather adequate data

   6. Incidents are reported based on the perception of the offender

7. August 10, 2012 – FBI, CJIS, FBI Headquarters, DOJ’s Office of the Deputy Attorney General (ODAG), and the White House Domestic Policy Council held a teleconference to discuss Hate Crime data collection procedures for adding Sikh as a religious bias motivation indicator to HCS. This meeting was a result of the shootings at the Sikh gurdwara in Oak Creek, Wisconsin. Included in the discussion were the following topics:

   1. CJIS APB Process

   2. Efficacy of Congressional mandates vs. the perception of unfunded mandates

   3. Cost of updating Records Management Systems for law enforcement data contributors

   4. Importance of buy-in for local, state, and tribal law enforcement data contributors

   5. Collection of the bias as the perception of the offender vs. the self-identification of the victim

   6. Sikh community’s position that the offender is biased against Sikhs, not Muslims

8. August 2012 – The Anti-Defamation League (ADL), the co-presidents of the Seattle Chapter of the Japanese American Citizens League (JACL), 20 Senators, including Senator Harry Reid also drafted letters to the U.S. Attorney General asking the FBI UCR Program to adapt the HCS to include an anti-Sikh category.

9. September 2012 – The FBI Director was provided talking points for congressional testimony on the APB Process and the UCR Process for Implementation. SALDEF drafted a letter to the Assistant Attorney General Perez and the Director of the DOJ’s Community Relations Service (CRS) requesting an anti-Sikh addition to HCS.

10. October 3, 2012 – A teleconference was held with the FBI, the ODAG, and the staff of Senator Richard Durbin-Illinois to aide understanding of Hate Crimes. They conveyed Senator Durbin’s suggestion that anti-Sikh, anti-Hindu, and anti-Arab be added to HCS.

11. October 2012 – The ODAG conducted a town hall meeting with religious groups to determine if additional anti-religious biases should be included in HCS, and if so, which religions should be collected. The Hindu American Foundation (HAF) provided a letter to the Attorney General requesting an anti-Hindu addition to HCS.

12. October 9, 2012 – A letter from the American-Arab Anti-Discrimination Committee (ADC) to the Attorney General requesting crimes against Arab-Americans be tracked as a separate category in HCS.

13. October 15, 2012 – A letter from the Hate Crime Coalition to the Attorney General was received, requesting hate crimes directed against Sikhs, Arabs, and Hindus be added to HCS.

14. October 17, 2012 – A letter was received by CJIS Designated Federal Officer, R. Scott Trent from eight Representatives encouraging the APB Regional Working Groups (WG) and the UCR Subcommittee to amend HCS to include a separate anti-Sikh religion bias.

15. October 18, 2012 – The FBI held a discussion concerning the Sikh and other religion biases with the APB’s UCR Subcommittee. DOJ provided the subcommittee with a summary of the issues identified in their October 3, 2012, teleconference. Guest speakers from the ADC and the SC urged subcommittee members to add Arab to the race and/or ethnicity bias motivations and Sikh to the religion bias motivation. Based on the recommendations from the UCR Subcommittee, the UCR Program prepared two topic papers for the Spring 2013 WG meetings. These papers presented options for additional religion, race, and/or ethnicity bias motivations in the UCR Program.

16. October 2012-February 2013 – The UCR Program also received additional letters to expand the bias motivation categories from Groundswell at Auburn Seminary, the Hate Crime Coalition, the HAF, JACL, the National Network for Arab American Communities, SALDEF, Senator Durbin & 26 additional Senators, Senator Stabenow, the Sikh Coalition (SC), South Asian American Policy and Research Institute, South Asian Americans Leading Together, and the South Asian Bar Association of New York.

17. March 5-6, 2013 – The WGs were presented with options to add anti-Sikh, anti-Hindu, and anti-Arab bias codes. Other options, to add all religious affiliations listed in the U.S. Census Bureau’s Statistical Abstract (2012) publication and the Pew Research Center’s Forum on Religious and Public Life (2008), and the addition of a custom text field to allow any and all religious groups to be reported in the UCR, were also discussed. The WG meeting discussions also included the potential for adding anti-Arab as a race or ethnicity in the UCR Program.

18. April 19, 2013 – A letter was received by the Attorney General and FBI Director from the U.S. Commission on Civil Rights to expand HCS to report anti-Sikh, anti-Hindu, and anti- Arab bias motivations.

19. April 23, 2013 –During the UCR Subcommittee meeting, a motion was passed to collect all religious affiliations listed in the Pew Research Center’s Forum on Religion and Public Life (2008) and the U.S. Census Bureau’s Statistical Abstract (2012) publications in HCS. The subcommittee also motioned to modify HCS to include an anti-Arab bias motivation.

20. May 17, 20, and 28, 2013 – Letters were received by CJIS Designated Federal Officer, R. Scott Trent, from the American Sikh Congressional Caucus, the National Criminal Justice Association, and the Hate Crime Coalition requesting the anti-Sikh, anti-Hindu, and anti-Arab additions to HCS.

21. May 31, 2013 – A teleconference was held with the U.S. Department of Justice (ODAG, Office of Legislative Affairs, CRS and the Civil Rights Division) to express concern regarding the UCR Program’s Hate Crime Statistics Program Religion Bias Motivations topic to be considered before the APB on June 5, 2013. The proposal was to “modify the Hate Crime to include all self-identified religions in the U.S. as listed in the Pew Research Center’s Forum on Religion and Public Life (2008) and the Statistical Abstract (2012) approved by the U.S. Census Bureau.” DOJ was concerned the total listing in the Pew Forum and Census Bureau’s Statistical Abstract would create an excessive training burden on law enforcement. DOJ mistakenly arrived at the conclusion that the FBI was only proposing the addition of Sikh, Hindu, and Arab bias motivations. While those were an option in the APB proposal, the FBI’s UCR Program also provided other options. Because the Advisory Process wanted to finalize the issue, they chose the option to add the self-identified religions as listed in the Pew Forum and Statistical Abstract. While compiling the list, the UCR Program and Mr. Scott Trent, Designated Federal Officer (DFO) to CJIS APB, spoke to UCR Subcommittee Chair, Mr. Lawrence A. Stelma, to ensure that this combined list (eliminating the subcategories) would reflect the spirit of the motion.

22. June 5, 2013 – Mr. Stelma agreed with the above-mentioned motion and the following categories were presented and passed by the APB: Catholic, Protestant, Mormon, Jehovah’s Witness, Eastern Orthodox (Russian, Greek, Other), Other Christian, Jewish (Judiasm), Islamic (Muslim), Buddhist, Hindu, Sikh, Other Religions, Multiple Religions-Group, and Atheist/Agnostic.

Selection of Law Enforcement Participants in Cognitive Interviews

The law enforcement community is typically organized and identified based upon the type of community they serve. Within the contributor community of the UCR Program, these community types are identified as those which serve incorporated cities, the unincorporated communities of counties, state police, campus police, and federal law enforcement. These broad categories can be further divided based upon the size of the populations they serve. In order to capture the different dynamics that might emerge in the response to the application of the bias motivation categories, participants were selected from Morgantown Police Department (a small, city police department), Pittsburgh Bureau of Police (a large, urban police department), Monongalia County Sheriff’s Office, the West Virginia State Police, West Virginia University Police Department, and the FBI Police Department.

In addition to selecting a set of agencies that represent the variety of viewpoints based upon agency type, participants were also selected based on the role they play within their agency. The participants were selected from both civilian and sworn employee populations, which include record clerks, detectives, patrol officers, and Chiefs.

Cognitive Interview Procedures

Cognitive interviewing is proposed to begin on June 26, 2014, and conclude on June 27, 2014. The cognitive interviews will assess two aspects of the revised collection. The first component will focus on whether or not new web-based collection in the Excel Workbook introduces problems with general usability through the transition from one mode of collection to another. The second component will specifically target usability and possible comprehension problems associated with the introduction of new bias motivation categories as well as the use of anti-Multiple Races, Group bias motivation.

The twenty law enforcement personnel will participate in the cognitive interviews on one of two possible days. The method involves intensive, one-on-one interviews in which the participant will be asked to "think aloud" as he or she answers a series of scenarios for classification within the Hate Crime Data Collection. A number of different techniques may be involved, such as asking respondents to paraphrase questions or asking probing questions to determine how respondents came up with their answers. In addition to the think-aloud, concurrent and retrospective probes may be used to expand upon a respondent’s remarks or actions. The objective is to identify problems of ambiguity or misunderstanding, identify potential improvements on form appearance, flow, and instructions, or highlight other difficulties respondents have answering questions. Participants will be provided a series of scenarios of five hate crime incidents. The first scenario will serve as a “warm-up” exercise from the prior round of cognitive testing and will be used to evaluate the impact of changing from a paper-based instrument to a web-based instrument. The remaining four scenarios will address the comprehension of the three new bias motivations (anti-Arab, anti-Sikh, and anti-Hindu) and the anti-Multiple Races, Group bias motivation. Each interview will take approximately one hour.

While all participants will be using the same five scenarios, the twenty participants will be split into two groups of ten to assess the best format for presenting anti-Arab bias motivation in the list of possible options. The current form provides a list of all possible bias motivations under larger categorical designations. They are race, ethnicity, religion, sexual orientation, disability, gender, and gender identity. Under each of these larger categories, there are further detailed examples of each of these types of bias motivations. For example, the detailed bias motivations of anti-White, anti-Black or African American, Anti-American Indian or Alaska Native, Anti-Asian, Anti-Multiple Races, Group, and anti-Native Hawaiian or Other Pacific Islander are considered part of the race category. The cognitive interviews will assess whether participants better understand the concept of anti-Arab depending upon its presentation.

The first group of ten participants will be given the web-based data collection that presents the anti-Arab bias motivation as an additional category under Ethnicity. This is similar to the current method on the hate crime incident form that presents the list of bias motivations by race and by ethnicity in separate categories. The second group of ten participants will use a web-based form that combines all racial, ethnic, and ancestry-based bias motivations in a combined category. This method has been successfully used by other agencies in their collections on individual race and ethnicity by observer data. Using a split-sample design on this one particular question will allow for a better assessment on the impact of the presentation of race and ethnicity information as it relates to bias motivation.

A team of two individuals from the UCR Program will conduct the cognitive interviews with each participant individually. This will allow for both cognitive interviewers to take detailed notes of responses provided by the participant and record behavioral cues. One cognitive interviewer will be directly viewing how the participant is filling out the data collection on a laptop computer and will primarily be responsible for recording any behavioral cues that would indicate difficulty with the mode of collection. Examples of this behavior may include hesitation, hovering over specific elements, or typographic problems. The second cognitive interviewer will be primarily responsible for recording the verbal information provided by the participant as a part of the “think aloud” aspect of the cognitive interview.

Language

The cognitive interviews will be conducted in English.

Burden Hours for Cognitive Testing

We request a total of 20 burden hours for 20 law enforcement personnel (sixty minutes per respondent).

Analysis Plan

The cognitive interviews will take place over the course of two days where participants will be asked to “think aloud” as they fill out an electronic Hate Crime Incident form based upon five scenarios. Due to the limited number of participants, the analysis will be qualitative rather than quantitative. The notes taken by the cognitive interviewers will be analyzed for patterns of problems associated with the usability and possible comprehension of certain bias motivation categories, as well as any indication of difficulties with the web-based data collection.

Responses will be categorized by whether there was an error in classification by the participant, which would indicate a possible problem in validity, as well as verbal and behavioral indications of comprehension problems and confusion. Particular attention will be paid to differences that emerge from the different types of law enforcement agencies, as well as differences between sworn and civilian law enforcement employees.

The FBI will produce a final report summarizing overall indications of problems with validity and comprehension. Raw counts of correct versus incorrect classification of scenarios based upon UCR standard definitions will be provided in the report as an indication of problems with validity. In addition, a question-by-question summary will be provided of any difficulties encountered by participants based upon the new web-based mode of collection and the two specific presentations of race and ethnicity bias motivations. Finally, the report will specifically address any complications in the understanding of the anti-Arab, anti-Sikh, anti-Hindu, or anti-Multiple Races, Group bias motivations. Recommended changes to questions, instructions, or training materials will also be provided.

Informed Consent, Data Confidentiality and Data Security

The telephone invitations, introduction to the Hate Crime Data Collection, and a description of the cognitive interview that was given to the participants previously on the telephone provide the elements of informed consent. The telephone invitation provided the purpose of the survey, the voluntary nature of the study, how the participants were selected, and a number to call with questions about the study. The telephone invitation and introduction to the survey announced the estimated length of the interview in advance, allowing the participant an opportunity to decline if the burden would be unacceptable.

Once the participants arrive at the location for the cognitive interview, the cognitive interviewers will reiterate the points delivered during the telephone invitation on the voluntary nature of the cognitive interview and the purpose of the research. In addition, the participants will be assured that their response will be protected to the extent that we are legally allowed. This information will be provided to the participants in hard-copy for their signature along with a copy for their own records should they wish.

4