Technical Report of Cognitive Testing

Attach4_Technical Report of Cognitive Testing.docx

National Use of Force Data Collection

Technical Report of Cognitive Testing

OMB: 1110-0071

Document [docx]
Download: docx | pdf

Technical Report of Cognitive Testing of National Use-of-Force Data Collection

February 14, 2017

Since the shooting of Michael Brown in Ferguson, Missouri, in 2014, law enforcement uses of force have called public attention to the need for more information on these types of encounters. To provide a better understanding of the incidents of use of force by law enforcement, the Uniform Crime Reporting (UCR) Program is proposing a new data collection to include information on incidents where a use of force by a law enforcement officer as defined by the Law Enforcement Officers Killed and Assaulted (LEOKA) Program has led to the death or serious bodily injury of a person, as well as when a law enforcement officer discharges a firearm at or in the direction of a person. The definition of serious bodily injury will be based, in part, on 18 United States Code (USC) Section 2246 (4), to mean “bodily injury that involves a substantial risk of death, unconsciousness, protracted and obvious disfigurement, or protracted loss or impairment of the function of a bodily member, organ, or mental faculty.”

Goal of the Proposed Data Collection on Law Enforcement Use of Force

The goal of the FBI’s data collection on law enforcement officer use of force is to produce a national picture of the trends and characteristics of use of force by a law enforcement officer, as defined by the LEOKA Program, to the FBI. The collection and reporting would include use of force that results in the death or serious bodily injury of a person, as well as when a law enforcement officer discharges a firearm at or in the direction of a person. The data collected by the UCR Program would include information on circumstances surrounding the incident itself, the subjects, and the officers. The data collection would focus on information that is readily known and obtainable by law enforcement with the initial investigation following an incident rather than any assessment of whether the officer acted lawfully or within the bounds of department policies. Publications and releases from the data collection will provide for the enumeration of fatalities, nonfatal encounters that result in serious bodily injury, and firearm discharges by law enforcement. In addition, targeted analyses could potentially identify those law enforcement agencies with “best practices” in comparison with their peers as an option for further study.

This FBI data collection will facilitate important conversations with communities regarding law enforcement actions in relation to decisions to use force and works in concert with recommendations from the President’s Task Force on 21st Century Policing. Given a growing desire among law enforcement organizations to increase their own transparency and embrace principles of procedural justice, this collection will expand the measure to a broader scope of incidents of use of force to include nonfatal instances as well.

Purpose of the Research

The purpose of the research is to investigate the understanding of the language and wording of the questions in the proposed data collection on law enforcement use of force, as well as their associated instructions by the law enforcement community. The ultimate goal for the development research activities is to ensure that participants have a clear understanding of what information is requested even in complex law enforcement situations. This will aid the UCR Program in its efforts to increase the overall validity and reliability of its data collections. The cognitive testing is a first step to understand the extent to which the law enforcement community has a common understanding of key concepts in the data collection. In addition, there are questions that ask participants to indicate what records are readily available on certain key pieces of information such as time and location.

Background Research

Criminal Justice Information Services (CJIS) Advisory Policy Board (APB)—

Spring 2015

The FBI’s initial proposal was to pursue the addition of nonfatal officer-involved shootings to the existing information on justifiable homicide. Beginning in March 2015, the FBI CJIS Division worked with its APB to consider the question of whether to pursue the collection.

The CJIS APB is a committee comprised of representatives from the law enforcement and criminal justice communities who advise the FBI Director on matters related to the criminal justice information systems the CJIS Division manages. The APB meets semiannually and provides recommended actions on policy and technical issues, to include the UCR Program. While this body does not have the expertise to provide advice on matters of statistical methodology, the APB does provide two important functions. First, its membership is a geographically diverse group of experts on criminal justice records maintained by law enforcement agencies at all levels of government: local, state, tribal, and federal. These representatives can provide important feedback on the potential impact of any change or addition to UCR data collections on the law enforcement community. Secondly, the APB represents the community from which UCR data are collected. Through the APB process, the FBI is able to engage in critical discussions that inform decisions about content and scope of law enforcement statistics.

On June 3, 2015, the APB received the recommendation to approve the data collection and passed the following amended motion:

“The UCR Program to develop a method to collect information on nonfatal/fatal shootings by law enforcement in the line-of-duty. The UCR Program will work with local law enforcement agencies and the five major national law enforcement organizations to develop what information to collect and the best method to do so and bring the topic back through the APB Process.”

Meeting of Major Law Enforcement Organizations

Based upon the direction of the CJIS APB, the FBI also began efforts to solicit input from the major law enforcement organizations. Representatives from the major law enforcement organizations including the International Association of Chiefs of Police (IACP), the Major Cities Chiefs Association (MCCA), the National Sheriffs’ Association (NSA), the Major County Sheriffs’ Association (MCSA), the Association of State Criminal Investigative Agencies (ASCIA), and the Police Executive Research Forum (PERF) met with representatives from the CJIS APB, the FBI, and the Department of Justice (DOJ) to discuss the next steps regarding the creation of the new data collection on officer-involved shootings. The result of the meeting was a unanimous endorsement of a data collection system, to include the following definition and content.

The participants at the meeting on September 18, 2015, proposed the following definition:

“Law Enforcement Officer’s as defined by the Law Enforcement Officer Killed and Assaulted (LEOKA) Program Use of Force (LEOUF) that results in the death or serious physical injury to a person, or when the law enforcement officer discharges a firearm at or in the direction of a person.”

The effects of this definition were threefold. First, it expanded the collection to include the use of force that results in serious physical injury, rather than the original focus of officer-involved shootings. Secondly, the inclusion of the definition of law enforcement as set by the LEOKA Program would allow for some flexibility to expand the scope of the use of force collection in parallel with the LEOKA data collection. Finally, the definition does not require a law enforcement agency to determine that the use of force was justified.

In addition to the proposed definition for the types of incidents eligible for reporting in the UCR use-of-force collection, the representatives from the major law enforcement organizations proposed content to be collected on every incident that meets the criteria of the definition. The proposed data elements included basic information on any officers involved, the subjects of the use of force, and circumstances related to the use of force, which served as the basis for the final recommendation approved by the CJIS APB. These data should be collected by the FBI as a part of its UCR Program but collected separately from the rest of the criminal incident and offense information the FBI currently maintains. These proposals were brought to the CJIS APB for consideration.

CJIS APB—Fall 2015

After the meeting with the law enforcement representatives in September 2015, the FBI introduced a topic to address the question of whether a new data collection on law enforcement officer use of force would be recommended to the FBI Director. On December 3, 2015, the CJIS APB approved four motions that effectively established the scope and minimum content of the new data collection.

The scope of the data collection was defined in the following language:

“The APB recommends the collection and reporting of use of force by a law enforcement officer (as defined by LEOKA) to the FBI. The collection and reporting would include use of force that results in the death or serious bodily injury of a person, as well as when a law enforcement officer discharges a firearm at or in the direction of a person. The definition of serious bodily injury will be based, in part, upon 18 USC Section 2246 (4). The term ‘serious bodily injury’ means bodily injury that involves a substantial risk of death, unconsciousness, protracted and obvious disfigurement, or protracted loss or impairment of the function of a bodily member, organ, or mental faculty.”

The minimum content was identified to include basic information on the characteristics of the incident, demographics and actions of the subject, and demographics and actions of the officer. In addition to delineating the minimum content, the APB moved to establish a Use of Force Task Force that would review this information and provide recommendations for changes or additions to the data elements.

Task Force

As specified in the CJIS APB Recommendations, the FBI moved to establish a Use of Force Task Force to make the decision on the additional content of the new data collection in January 2016. The Use of Force Task Force is comprised of representatives from major law enforcement organizations and local, tribal, and federal law enforcement representatives. Specifically, the following organizations are represented on the Use of Force Task Force:

  • IACP

  • NSA

  • MCCA

  • MCSA

  • PERF

  • ASCIA

  • National Organization of Black Law Enforcement Executives

  • Association of State UCR Programs

In addition to representatives from these major organizations, the Task Force welcomed observers from the Office of the Deputy Attorney General, the Community Oriented Policing Services Office, the DOJ, and the Bureau of Justice Statistics (BJS).

The Task Force met on four occasions (January 27, 2016; March 17, 2016; May 4-5, 2016; and August 3, 2016). The final set of data elements recommended by the Use of Force Task Force was based upon discussions from these four meeting and, importantly, the input of the BJS on their experiences and research that formed the basis of the Arrest-Related Death (ARD) Collection. Many of the data elements and concepts in the National Use-of-Force Data Collection were mirrored from the BJS’s ARD Collection. Because the ARD Collection has been through significant testing, this particular cognitive test focuses upon areas that are new (such as in the case of interpreting the concept of serious bodily injury) or is captured in a different fashion than the ARD (e.g., the concept of active aggression).

Data Elements of the National Use-of-Force Data Collection

Incident Information

  • Date and time of the incident

  • Total number of officers who applied actual force during time of incident

  • Number of officers from your agency who applied actual force during time of incident

  • Location of the incident [physical location acceptable through address or latitude and longitude]

  • Location type of the incident [location codes from the National Incident-Based Reporting System (NIBRS)]

  • Did the officer(s) approach the subject[s]? [Yes/No/Pending/Unknown]

  • Was this an ambush incident? [Yes/No/Pending/Unknown]

  • Was a supervisor or a senior officer acting in a supervisory capacity present or consulted at any point during the incident? [Yes/No/Pending/Unknown]

  • Reason for initial contact between subject and officer [response to unlawful or suspicious activity/medical, mental health, or welfare assistance/routine patrol other than traffic stop/traffic stop/warrant service/service of a court order/mass demonstration/follow up investigation/ other/unknown]

    • If this was due to “unlawful or criminal activity,” what were the most serious reported offenses committed by the subject prior to or at the time of the incident? [NIBRS offense codes drop down, 3 boxes]

    • If applicable, NIBRS (or local) incident number of report detailing criminal incident information on subject or assault or homicide of law enforcement officer.

  • If incident involved multiple law enforcement agencies, case numbers for the local “use of force reports” at the other agencies.

Subject Information

  • Age, sex, race, ethnicity, height, and weight (with range of values) of the subject(s)

  • Injury/Death of subject(s) [gunshot wound/apparent broken bones/possible internal injury/severe laceration/loss of teeth/other major injury/unconsciousness/death/ pending/unknown]

  • Type(s) of force used connected to serious bodily injury or death [firearm/electronic control weapon (Taser)/explosive device/pepper or OC (oleoresin capsicum) spray/baton/impact projectile/blunt instrument/hands-fists-feet/canine/other/pending/unknown]

  • Subject(s) resisted? [Yes/No/Pending/Unknown]

  • Was the threat by the subject(s) directed to the officer or to another party? [Officer/Other party/Pending/Unknown]

  • Type(s) of subject resistance/weapon involvement [threatened officer/threatened others/threatened self/active aggression/edged weapon/firearm/vehicle/chemical/ electronic/verbal/passive resistance/resist being handcuffed or arrested/attempt to escape or flee from custody/none]

  • Apparent or known impairment/physical conditions of subject? [Yes/No/Pending/Unknown]

    • If Yes, indicate which [mental health/alcohol/drugs/unknown]

    • At any time during the incident, was the subject(s) armed or believed to be armed with a weapon? [Yes/No/Pending/Unknown]



Officer Information

  • Age, sex, race, ethnicity, height, and weight of the officer(s)

  • Officer’s years of service as a law enforcement officer (total tenure) [Number of years]

  • Full-time? [Yes/No]

  • Was the officer readily identifiable? [Yes/No]

  • Was the officer on duty at the time of the incident? (Yes/No)

  • Did the officer discharge a firearm? [Yes/No]

  • Officer(s) injured [Yes/No]

  • Officer injury type [gunshot wound/apparent broken bones/possible internal injury/severe laceration/loss of teeth/other major injury/unconsciousness/death]

Overview of Comprehensive Testing Plan

The FBI acknowledges that managing the scope of this collection and providing good guidance will be a challenge. In order to manage this effort, the FBI completed a Comprehensive Testing Plan to outline a series of activities to help inform the FBI on the decisions impacting scope, content, and participation levels. This document was forwarded to the OMB on September 2, 2016, and sets forth the expected activities to occur both before data collection commences and at the onset of data collection with a pilot study.

The pre-testing activities occurring before data collection consists of three primary efforts all of which build upon each other for planning the pilot study. The first is a cognitive testing effort to further research some concepts connected to data elements included in the National Use-of-Force Data Collection at the request of the law enforcement community. The second activity during pre-testing is a canvass of state UCR Program managers and state CJIS System Officers to gather information on programmatic and technical capabilities of the states in anticipation of the launch of the National Use-of-Force Data Collection. The final pre-testing activity is a small-scale assessment of the usability of the data collection application.

The pre-testing activities will provide critical information that will allow for the FBI to finalize plans to conduct a pilot study over the course of the first six months of data collection using a targeted group of law enforcement agencies. The goal of the pilot study is to assess the interpretation of questions used in the National Use-of-Force Data Collection and any guidance or instructions included in the data collection. This assessment will be based on a comparison of the original law enforcement record to the submitted responses to the questionnaire for pilot agencies. In addition to the record comparison, an on-site review of records for a sample of agencies will be conducted to assess the extent of nonresponse for in-scope incidents for participating agencies. The pilot study provides the best path to assess the data collection in the context of complex law enforcement decisions.

Background Research on Cognitive Testing Instrument

The cognitive testing instrument was developed with input from the law enforcement community (through the Use of Force Task Force membership), the BJS, and William Bozeman, M.D. Dr. Bozeman is a physician in the Department of Emergency Medicine at Wake Forest University. He has been extensively published in the research area of injury and law enforcement use of force and is a member of the IACP Police Physicians Section. We anticipate continuing our collaboration with all three parties for both pre-testing activities and the pilot study. Based upon input from all parties, draft questions were revised to reflect the final version attached to this document.



Methodological Plan

The cognitive testing was primarily focused upon the language and construction of the response categories rather than the usability of the Web-form that is under construction or other questions on mode of collection. These usability tests will be conducted separately as a part of system development. The purpose of the cognitive test is to identify key concepts that may have the potential for a high amount of variability in their interpretation. These areas will require thorough explanation to promote the reliability of the information measured.

The FBI solicited participation from the 280 participants in the FBI National Academy in residence at the FBI training facility in Quantico, VA on November 30, 2016. The FBI National Academy is a 10-week training program of leaders and managers of state, local, county, tribal, military, federal, and international law enforcement agencies. These 280 potential participants represent the total roster of the current FBI National Academy class.

The questions on the cognitive testing instrument were developed to identify areas where there might not be a common understanding of the same terminology. In essence, the results of the test will provide a general “yes or no” response to the question of whether there is an existing normative understanding of some concepts in the National Use-of-Force Data Collection. This questionnaire is not going to be used to understand how the terminology may be applied on complex law enforcement scenarios. The FBI plans to do further analysis on the application of definitions and guidance during the upcoming pilot study as described in the Comprehensive Testing Plan provided to the Office of Management and Budget via email on September 2, 2016.

The areas that will be addressed in the cognitive testing include the following:

  • The assignment at the time of the incident

  • The selection of the location and location type (because many location types are not mutually exclusive)

  • Further exploration on the request identifying aggression

  • The application of the legal definition of serious bodily injury

On each of these particular concepts, the participants will be presented with a series of questions. Some of the questions will involve a simple “yes” or “no” response based on how information is recorded by law enforcement. Other questions will present an array of responses for their ranking or interpretation. For example, on the question of serious bodily injury, a list of potential injuries will be offered to participants. Each participant will indicate the injuries that he or she understands to be “serious” based upon the definition provided. The testing instrument will be paper-based and emailed to participants once consent has been obtained. This phase of the cognitive testing will be concluded by early December 2016.

The Cognitive Testing was submitted to the FBI Institutional Review Board (IRB) with a request for an expedited review under 28 CFR 46.110 to determine if the protocol would be considered exempt from the IRB process under 28 CFR 46.101(b)(2). The FBI IRB approved the overall protocol with minor provisions. Those provisions primarily addressed concerns over the privacy of the participant given that some questions called for a judgment on whether certain actions considered “aggressive.” The FBI IRB required that information on the participants’ agency type, region, and current position be separated from the answers provided on the remaining questions in order to reduce the likelihood that a participant’s identity may be discovered.

Selection and Response of Participants in Cognitive Testing

The FBI National Academy classes are drawn from a variety of law enforcement agencies to include local, county, state, tribal, federal, and international law enforcement agencies, as well as military law enforcement. There were 149 participants that completed and returned the survey instrument.


Table 1. Participants by Agency Type, Counts and Percentages

Agency Type


Frequency

Percent

Valid Percent

Cumulative Percent

Valid

City police

89

58.9

61.4

61.4

County sheriff

18

11.9

12.4

73.8

State police

15

9.9

10.3

84.1

Campus police

2

1.3

1.4

85.5

Other state agency

1

.7

.7

86.2

Tribal

1

.7

.7

86.9

Federal

7

4.6

4.8

91.7

Other

12

7.9

8.3

100.0

Total

145

96.0

100.0


Missing

Did not answer

6

4.0



Total

151

100.0




The majority of respondents are sworn personnel at municipal police departments. County sheriffs and state police are the next most comment (12.9 percent and 10.3 percent, respectively). (See Table 1.) Of those that identified a region of the United States, there was slightly more representation from the South with the remaining three regions receiving approximately equal representation. (See Table 2.) In addition, the preponderance of the respondents (93.8) were at least mid-level managers if not higher ranked. The widest representation was from mid-level managers (72.2 percent). (See Table 3.)







Table 2. Participants by Region, Counts and Percentages

Region


Frequency

Percent

Valid Percent

Cumulative Percent

Valid

Northeast

27

17.9

18.9

18.9

Midwest

29

19.2

20.3

39.2

South

38

25.2

26.6

65.7

West

28

18.5

19.6

85.3

Other

21

13.9

14.7

100.0

Total

143

94.7

100.0


Missing

Did not answer

8

5.3



Total

151

100.0





Table 3. Participants by Current Position, Counts and Percentages

Current Position


Frequency

Percent

Valid Percent

Cumulative Percent

Valid

Chief Executive

11

7.3

7.6

7.6

Assistant Executive

20

13.2

13.9

21.5

Mid-level management

104

68.9

72.2

93.8

Supervisor/Sergeant

6

4.0

4.2

97.9

Investigator/Detective/Special Agent

3

2.0

2.1

100.0

Total

144

95.4

100.0


Missing

Did not answer

7

4.6



Total

151

100.0




Time

In order to better discern the availability of information on the time of a law enforcement use of force, the participants were asked to provide whether or not information on particular points in time would be available based upon their agencies typical practice. The question differentiated among three points in time. The first was the time that officers arrived at the scene of the incident. The second point in time was when the officer(s) first made contact with the subject(s). Finally, participants were asked about the time that force was used by law enforcement. For each of these three points in time, participants were asked whether that information would be available as an actual recorded value in a database, provide as an actual value in a narrative, estimated from information available in a narrative, or not typically available.

Results


Table 4. Information on Time Available, Counts and Percentages

Time Information Available Frequencies


Responses

Percent of Cases

N

Percent

Time

Information Availablea

Time-Arrived at the scene-Actual value in form

130

19.2%

90.3%

Time-Arrived at the scene-Actual value in narrative

80

11.8%

55.6%

Time-Arrived at the scene-Estimated from narrative

54

8.0%

37.5%

Time-Arrived at the scene-Not typically available

5

0.7%

3.5%

Time-Contacted the subject-Actual value in form

44

6.5%

30.6%

Time-Contacted the subject-Actual value in narrative

59

8.7%

41.0%

Time-Contacted the subject-Estimated from narrative

89

13.2%

61.8%

Time-Contacted the subject-Not typically available

14

2.1%

9.7%

Time-Used force-Actual value in form

42

6.2%

29.2%

Time-Used force-Actual value in narrative

57

8.4%

39.6%

Time-Used force-Estimated from narrative

92

13.6%

63.9%

Time-Used force-Not typically available

10

1.5%

6.9%

Total

676

100.0%

469.4%

a. Dichotomy group tabulated at value 1.


Based upon responses, agencies commonly record the time that officers arrived at the scene (90.3 percent) as an actual value in a database or form more than any other time. After that category, a little over 60 percent of respondents indicated that their agencies can estimate the time that officers came in contact with the subject(s) and when the officer(s) used force from information in the narrative. (See Table 4). However, when asked about the time that would “best” represent when a use-of-force event occurred, 72.2 percent indicated that it would be when the officer used force rather than when the officer arrived at the scene or made contact with the subject. (See Table 5.) Upon further analysis, 85.9 percent of respondents indicated that they could at least estimate the time that force was used based upon information in the narrative (see Table 6).




Table 5. Time Information that Best Represents a Use-of-Force Event, Counts and Percentages

Time that best represents


Frequency

Percent

Valid Percent

Cumulative Percent

Valid

When the officer arrived at the initial scene

15

10.1

10.4

10.4

When the officer first contacted the subject

19

12.8

13.2

23.6

When the officer used force

104

69.8

72.2

95.8

Other

6

4.0

4.2

100.0

Total

144

96.6

100.0


Missing

Unusable answer

2

1.3



Did not answer

2

1.3



System

1

.7



Total

5

3.4



Total

149

100.0






Table 6. Is the Time of the Use-of-Force Event Available?, Counts and Percentages

Time-Used Force


Frequency

Percent

Valid Percent

Cumulative Percent

Valid

Information can at least by estimated from narrative

128

85.9

85.9

85.9

No information typically available

21

14.1

14.1

100.0

Total

149

100.0

100.0





Discussion

Respondents favored recording the time that force was used, and a strong majority indicated that the time fore was used could at least be estimated from existing records. Given these findings, the following recommendation was made to amend the instructions to the participants in the National Use-of-Force Data Collection as follows:

Instructions:

Q2. Time of the incident

The time of the incident should reflect your best estimate of when the interaction between law enforcement and the subject occurred. If the exact time is not known, round the time to the nearest hour.

The format of the time should be 24-hour military time of HHMM.

Location and Spatial Information

Much like the question set for time, participants were asked about the availability of information on the location of where a use of force by law enforcement may have occurred. Participants were asked to respond to twelve different combinations of the three points in an incident (the place the officer(s) were when they first arrived at the scene, the place where the officer(s) first made contact, and the place where officer(s) used force). Again, participants indicated whether those three locations would be available as a record as an actual value on a form or database, an actual value in a narrative, estimated from information in a narrative, or not typically available.

Results


Table 7. Information on Location Available, Counts and Percentages

Location Available Frequencies


Responses

Percent of Cases

N

Percent

Location

Information

Availablea

Location-Arrived at the scene-Actual value in form

109

15.7%

76.8%

Location-Arrived at the scene-Actual value in narrative

70

10.1%

49.3%

Location-Arrived at the scene-Estimated from narrative

65

9.4%

45.8%

Location-Arrived at the scene-Not typically available

8

1.2%

5.6%

Location-Contacted the subject-Actual value in form

50

7.2%

35.2%

Location-Contacted the subject-Actual value in narrative

72

10.4%

50.7%

Location-Contacted the subject-Estimated from narrative

87

12.5%

61.3%

Location-Contacted the subject-Not typically available

11

1.6%

7.7%

Location-Used force-Actual value in form

52

7.5%

36.6%

Location-Used force-Actual value in narrative

68

9.8%

47.9%

Location-Used force-Estimated from narrative

91

13.1%

64.1%

Location-Used force-Not typically available

11

1.6%

7.7%

Total

694

100.0%

488.7%

a. Dichotomy group tabulated at value 1.


The majority of the responses (76.8 percent) indicated that the location where officers arrived at the scene is recorded by law enforcement. However, additional potential locations can often be estimated from information provided in the incident narrative. (See Table 7.) Much like the concept of time, respondents also provided that the location of where force was used by law enforcement “best” represented the event (79.2 percent). (See Table 8.) Additional analysis of location and spatial information available in law enforcement records, indicate that address information could be provided or estimated. (See Table 9 and 10).





Table 8. Location Information that Best Represents a Use-of-Force Event, Counts and Percentages

Location that best represents


Frequency

Percent

Valid Percent

Cumulative Percent


Valid

Where the officer arrived at the initial scene

17

11.4

11.8

11.8



Where the officer first contacted the subject

12

8.1

8.3

20.1



Where the officer used force

114

76.5

79.2

99.3



Other

1

.7

.7

100.0



Total

144

96.6

100.0




Missing

Unusable answer

2

1.3





Did not answer

3

2.0





Total

5

3.4





Total

149

100.0





Table 9. Is the Location of the Use-of-Force Event Available?, Counts and Percentages

Location-Used Force


Frequency

Percent

Valid Percent

Cumulative Percent

Valid

Information can at least by estimated from narrative

135

90.6

90.6

90.6

No information typically available

14

9.4

9.4

100.0

Total

149

100.0

100.0



Table 10. Spatial Information Available by Type of Record, Counts and Percentages

Spatial Information Available Frequencies


Responses

Percent of Cases

N

Percent

Spatial

Information

Availablea

Address information on Calls for Service

142

23.2%

96.6%

Geocoded information on Calls for Service

89

14.5%

60.5%

Address information in Records Management System

138

22.5%

93.9%

Geocoded information in Records Management System

69

11.3%

46.9%

Address information on Use of Force

124

20.2%

84.4%

Geocoded information on Use of Force

51

8.3%

34.7%

Total

613

100.0%

417.0%

a. Dichotomy group tabulated at value 1.



Discussion

The following recommendation was made to amend the instructions to the participants in the National Use-of-Force Data Collection as follows:

Instructions:

Q4. Location of the incident

Please identify your best estimate of the location of the event causing injury, death, or the location of firearm discharge either by its address, approximate location (i.e., street intersection, neighborhood), or by geographic coordinates (latitude/longitude). The agency can provide one or the other. Both are not necessary.

When providing the geospatial data in longitude and latitude (geographic coordinates), please provide or transform the data in the NAD83 coordinate system.

If information is unknown because the investigation is still incomplete, record pending further investigation.

If the information is not known and is unlikely to ever be known, record unknown and is unlikely to be known.

Q5. Location type of the incident

Law enforcement agencies (LEAs) should use this data element to report the type of location/premises where the use of force took place.

The FBI UCR Program recognizes that for many incidents, there is more than one possible choice for reporting a location. The FBI UCR Program recognizes that for many incidents, there is more than one possible choice for reporting a location. For the purposes of this collection, law enforcement should use their best estimate for the location of where the use of force took place.

Because the geographic location of an incident is not always the same as the functional location of the incident, the FBI UCR Program relies on LEAs to report the most appropriate location type. For example, if an incident occurred at an elementary school playground during school hours, the location can be classified as School – Elementary/Secondary. But, if the incident occurred at the same physical location on a Saturday afternoon when the school was not operating and the public was allowed to use the facility for recreational purposes, LEAs would be equally correct in classifying the location as Park/Playground.

Sometimes, LEAs can determine the location by the offender’s intent during the commission of the crime. For example, if the offender chose to commit a robbery during a church service held at a public facility routinely used for basketball games, LEAs can choose to classify the location as Church/Synagogue/Temple/Mosque since the building was being used for a public religious activity at the time the crime was committed.



Armed with a Weapon

Respondents were asked the following question about the concept of “armed” to assess what types of weapons may be considered if left loosely defined.

If a subject is described as “armed,” which of the following could the subject be in possession of?

Respondents could select multiple weapon types.

Results


Table 11. Subject could be "Armed" with this Weapon, Counts and Percentages

Armed with a Weapon Frequencies


Responses

Percent of Cases

N

Percent

Armed

with a

Weapona

Firearm

148

12.7%

100.0%

BB or pellet gun

128

11.0%

86.5%

Knife

145

12.5%

98.0%

Other cutting instrument or edged weapon

142

12.2%

95.9%

Electronic control weapon

130

11.2%

87.8%

Explosive device

143

12.3%

96.6%

Blunt instrument

131

11.3%

88.5%

Pepper or OC Spray

114

9.8%

77.0%

Motor vehicle

82

7.1%

55.4%

Total

1163

100.0%

785.8%

a. Dichotomy group tabulated at value 1.



Based upon frequency of selection, a firearm would always be considered a weapon when “armed”. Other weapon types that were consistently chosen as a potential weapon (greater than 90 percent of the time) include knife, other cutting instrument or edged weapon, and explosive device. Three additional types of weapons that were selected over 80 percent of the time include BB or pellet gun, electronic control weapon, and blunt instrument. Finally pepper or OC (oleoresin capsicum) spray and motor vehicle were less consistently considered weapons when “armed.” (See Table 11.)

Discussion

Based upon the findings, the following changes were made to the language in the question and its instructions:

Questionnaire:

Q22. At any time during the incident, was the subject armed or believed to be armed with a weapon (other than hands, fists, or feet)?

  • Yes

  • No

  • Pending further investigation

  • Unknown and is unlikely to ever be known

Instructions:

Q22. At any time during the incident, was the subject(s) armed or believed to be armed with a weapon (other than hands, fists, or feet)?

The purpose of this item is to indicate if the subject was in possession of a weapon, regardless of whether the subject used the weapon in a threatening or assaultive way against either law enforcement officers or other persons.

A weapon can generally include, but are not limited to, firearm; BB or pellet gun; knife; other cutting instrument or edged weapon; electronic control weapon; explosive device; blunt instrument; chemical agent (e.g. acid, gasoline, pepper or OC (oleoresin capsicum) spray, etc.). Under certain circumstances motor vehicles or other objects could also be considered weapons if used or displayed in a threatening manner. Please mark all categories that apply.

If information is unknown because the investigation is still incomplete, record pending further investigation.

If the information is not known and is unlikely to ever be known, record unknown and is unlikely to be known.



Injury

The following question was asked on the cognitive testing instrument:

According to the National Use-of-Force Data Collection, the definition of serious bodily injury includes, “bodily injury that involves a substantial risk of death, unconsciousness, protracted and obvious disfigurement, or protracted loss or impairment of the function of a bodily member, organ, or mental faculty.”

While this question does not take into account situations where multiple injuries may increase the overall severity of injury to the person, we are interested in understanding which injuries listed below would you consider to meet the definition of serious bodily injury in of itself?

The question was used to begin to understand the extent of agreement among respondents on the application of a provided definition of serious bodily injury. Respondents were provided an array of potential injuries to determine if it meets or does not meet the definition of serious bodily injury. Each variable was subjected to dummy coding with a value of “1” being assigned if the respondent had indicated that the injury would meet the criteria for serious bodily injury according to definition.

Results



Table 12. Injuries Considered "Serious," Counts and Percentages

Injury Frequencies

 

Responses

Percent of Cases

N

Percent

Injurya

Brain damage

145

6.2%

98.0%

Penetrating gunshot wound

144

6.2%

97.3%

Paralysis

141

6.0%

95.3%

Loss of arm or leg

140

6.0%

94.6%

Internal bleeding

136

5.8%

91.9%

Coma

136

5.8%

91.9%

Loss of a finger or toe

128

5.5%

86.5%

Loss of part of a finger or toe

120

5.1%

81.1%

Eye damage

119

5.1%

80.4%

Apparent broken bones

116

5.0%

78.4%

Burns

107

4.6%

72.3%

Heart attack/cardiac arrest/cardiac event

105

4.5%

70.9%

Unconscious for more than a minute

104

4.5%

70.3%

Neck injury

102

4.4%

68.9%

Puncture wounds

99

4.2%

66.9%

Loss of teeth

88

3.8%

59.5%

Grazing gunshot wound

84

3.6%

56.8%

Lacerations that do require stitches

84

3.6%

56.8%

External head injury

75

3.2%

50.7%

Unconscious for less than a minute

65

2.8%

43.9%

Canine bite

50

2.1%

33.8%

Lacerations that do not require stitches

20

0.9%

13.5%

Bruising/contusions

15

0.6%

10.1%

Abrasions/scratches

13

0.6%

8.8%

Total

2,336

100.0%

1578.4%

a. Dichotomy group tabulated at value 1.


Based upon frequencies, the majority of respondents found nineteen different injuries were seen to be potentially serious according to the provided definition. The remaining five types of injury—unconscious for less than a minute, canine bit, lacerations that do not require stitches, bruising/contusions, and abrasions/scratches—did not satisfy the criteria of serious bodily injury for the majority of respondents. (See Table 12.)

In addition to basic frequencies, a factor analysis was conducted on the twenty-four variables to determine what underlying structure might exist. Principal components analysis using a Varimax rotation was conducted against the twenty-four variable set resulting in the initial retention of six components. The six components explained 61.8 percent of the variance in total. Based upon the criteria of eigenvalue and variance, five components were explored using an abbreviated list of variables that excluded two variables associated with the sixth component (grazing gunshot wound and cardiac event). The five component model showed an improved fit that explained 65.8 percent of the variance in total and reduced the number of residuals exceeding the .05 criteria from 100 (36.0 percent of non-redundant values) to 56 (32.0 percent of non-redundant values). All five components also exhibited sufficient inter-item reliability with a Cronbach alpha greater than .6. (See Table 13.)

Table 13. Component Loadings and Inter-item Reliability for Injuries


Factor Loading

Component 1: Physical trauma (Cronbach alpha = .823)


Lacerations that require stitches

.590

Loss of teeth

.637

Loss of a finger or toe

.737

Loss of a part of a finger or toe

.765

Apparent broken bones

.650

Component 2: Internal injury (Cronbach alpha = .826)


Penetrating gunshot wound

.586

Internal bleeding

.711

Brain damage

.737

Coma

.811

Paralysis

.838

Component 3: Not serious bodily injury (Cronbach alpha = .793)


External head injury

.445

Abrasions/scratches

.833

Bruising/contusions

.889

Lacerations that do not require stitches

.821

Component 4: Unconsciousness (Cronbach alpha = .731)


Unconscious for less than a minute

.797

Unconscious for more than a minute

.761

Component 5: Other Major Injury (Cronbach alpha = .692)


Neck injury

.632

Eye damage

.797

Burns

.700



Four injuries did not appear to clearly load on any component—cardiac event, puncture wound, grazing gunshot wound, and canine bite. It appears that respondents did not share a consistent interpretation of these four injuries that would align with existing categories.

Discussion

The five components were assessed for common themes and used to provide additional recommendations and guidance. Identified themes include physical trauma, internal injury, not serious bodily injury, unconsciousness, and other major injury. These detected components were compared against the initial data element and data values for injury. Injury is collected on both the subject and officer in the National Use-of-Force data collection. In order to incorporate some need to clarify the coding of certain types of injury, three data values (cardiac event, canine bite, and loss or partial loss of digit or limb) were added as options and additional information (puncture wound) was added to one data value (severe laceration) in order to provide a clear category to assign these injuries. In addition to the modification to data values, preliminary instructions were modified to provide guidance on how to code some of the other injury values according to the two broad categories of possible internal injury or other major injury.

Questionnaire

Q24. What were the subject’s injuries received as a direct consequence of the use of force by law enforcement? (Select all that apply)

Q35a. What were the officer’s injuries during the incident that precipitated the use of force? (select all that apply)

  • Apparent broken bones

  • Gunshot wound

  • Loss of teeth

  • Loss or partial loss of finger, toe, arm, leg, etc.

  • Possible internal injury

  • Severe laceration/puncture wound

  • Canine bite

  • Unconsciousness

  • Cardiac event

  • Other major injury

  • Death

  • None1

  • Pending further investigation

  • Unknown and is unlikely to ever be known

Instructions:

The purpose of this question is to record the injuries sustained by the subject as a result of the use of force by law enforcement.

If the subject/officer sustained multiple injuries, please mark all that apply. If the subject/officer died, death should be the only value recorded.

Please record all gunshot wounds regardless of whether they are penetrating or grazing as gunshot wound.

Examples for possible internal injury

  • Internal bleeding

  • Brain damage

  • Concussion

  • Coma

  • Paralysis

Examples for Other Major Injury

  • Neck injury

  • Eye damage

  • Burns

If information is unknown because the investigation is still incomplete, record pending further investigation.

If the information is not known and is unlikely to ever be known, record unknown and is unlikely to be known.

Resistance/Weapon

The National Use-of-Force Data Collection will ask participants to classify the actions of subjects into categories of resistance and weapons use on a consistent basis. Early in the discussions by the Use of Force Task Force, the concept of active aggression and passive resistance was introduced. In order to ascertain the types of actions that could be considered aggressive by law enforcement, respondents were asked the following question during cognitive testing:

Aggression is a complex concept to identify—especially in law enforcement scenarios. While this question does not take into account situations where multiple actions may escalate a situation between an officer and a subject, we are interested in understanding which of the following activities would you consider to be an act of active aggression in of itself?

The question was used to begin to understand the extent of agreement among respondents on the concept of active aggression and the behaviors that could be classified as aggressive. Respondents were provided an array of potential actions to determine if he or she would judge it to be “active aggression.” Each variable was subjected to dummy coding with a value of “1” being assigned if the respondent had indicated that the action could be considered aggressive.

Results

Based upon the frequency of responses, the majority of respondents found all but seven actions to be actively aggressive from an array of thirty-four possible choices. Those seven include: making verbal threats again another person; bleeding on an officer; bleeding on another person; display of feet; display of hands; yelling or using foul language at an officer; yelling or using foul language at another person. (See Table 14.)

In addition to these frequencies, a factor analysis was conducted to explore the underlying structure of the thirty-four actions and their relationship to the concept of active aggression. Principal components analysis using a Varimax rotation was conducted against the thirty-four variable set resulting in the retention of nine components. The nine components explained 78.2 percent of the variance in total, and 114 non-redundant residuals exceeded the .05 criteria (20 percent). All nine components exhibite high inter-item reliability with Cronbach alphas greater than .6. (See Table 15.)



Table 14. Resistance and Weapons Considered "Aggressive," Counts and Percentages

Active Aggression Frequencies

 

Responses

Percent of Cases

N

Percent

Active

Aggressiona

Attempting to gain possession of officer's weapon

145

3.9%

98.0%

Using a firearm against an officer

145

3.9%

98.0%

Using a blunt object against another person

145

3.9%

98.0%

Using a chemical agent against an officer

144

3.8%

97.3%

Using an edged weapon against an officer

144

3.8%

97.3%

Using a blunt object against an officer

144

3.8%

97.3%

Using a chemical agent against another person

143

3.8%

96.6%

Using an electronic control weapon against an officer

143

3.8%

96.6%

Using a firearm against another person

143

3.8%

96.6%

Use of hands, fists, or feet against an officer

143

3.8%

96.6%

Using an edged weapon against another person

142

3.8%

95.9%

Using an electronic control weapon against another

141

3.8%

95.3%

Use of hands, fists, or feet against another person

140

3.7%

94.6%

Directing a vehicle at an officer

139

3.7%

93.9%

Directing a vehicle at another person

138

3.7%

93.2%

Displaying an edged weapon

132

3.5%

89.2%

Displaying a firearm

132

3.5%

89.2%

Displaying a chemical agent

128

3.4%

86.5%

Displaying an electronic control weapon

123

3.3%

83.1%

Resisted being handcuffed or arrested

114

3.0%

77.0%

Displaying a blunt object

113

3.0%

76.4%

Display of fists

107

2.9%

72.3%

Spitting on an officer

99

2.6%

66.9%

Spitting on another person

87

2.3%

58.8%

Barricading self

81

2.2%

54.7%

Attempted to escape or flee from custody

78

2.1%

52.7%

Making verbal threats against an officer

75

2.0%

50.7%

Making verbal threats against another person

68

1.8%

45.9%

Bleeding on an officer

63

1.7%

42.6%

Bleeding on another person

51

1.4%

34.5%

Display of feet

49

1.3%

33.1%

Display of hands

39

1.0%

26.4%

Yelling or using foul language at an officer

37

1.0%

25.0%

Yelling or using foul language at another person

35

0.9%

23.6%

Total

3750

100.0%

2533.8%

a. Dichotomy group tabulated at value 1.



Factor Analysis

Table 15. Component Loadings and Inter-item Reliability for Resistance and Weapons


Factor loading

Component 1: Using a weapon 1 (Cronbach alpha = .951)


Attempting to gain possession of officer’s weapon

.643

Using a chemical agent against an officer

.641

Using an edged weapon against an officer

.901

Using an edged weapon against another

.825

Using an electronic control weapons against an officer

.827

Using a firearm against an officer

.803

Using firearm against another

.719

Using a blunt object against an officer

.790

Using a blunt object against another

.667

Using hands, fists, or feet against an officer

.851

Use of hands, fists, or feet against another

.713

Component 2: Displaying a weapon or fists (Cronbach alpha = .858)


Display a chemical agent

.709

Displaying an edged weapon

.759

Displaying an electronic control weapon

.693

Displaying a firearm

.772

Displaying a blunt object

.660

Display of fists

.564

Component 3: Verbal threats and yelling (Cronbach alpha = .900)


Making verbal threats against an officer

.817

Making verbal threats against another person

.829

Yelling or using foul language at an officer

.866

Yelling or using foul language an another

.874

Component 4: Vehicle related (Cronbach alpha = .909)


Directing a vehicle at an officer

.817

Director a vehicle at another

.791

Component 5: Using a weapon 2 (Cronbach alpha = .797)


Using a chemical agent against another person

.703

Using a electronic control weapon against another person

.733

Component 6: Display of hands or feet (Cronbach alpha = .887)


Display of hands

.888

Display of feet

.889

Component 7: Spitting (Cronbach alpha = .896)


Spitting on an officer

.899

Spitting on another

.848

Component 8: Bleeding (Cronbach alpha = .914)


Bleeding on an officer

.917

Bleeding on another

.913

Component 9: Resisting/evading arrest (Cronbach alpha = .634)


Attempted to escape or flee from custody

.763

Resisted being handcuffed or arrested

.653

Barricading self

.501





Other acts of aggression:

When participants were asked for other potential acts of active aggression that may have been encountered during the line of duty, only 14 participants provided a response. From those responses, the following situations were provided:

  • Biting or attempting to bite

  • Head butting

  • Charging at/closing with officer

  • Crowd gathering around officers in aggressive manner

  • Display Anti Govt. Sign/Flag; Display Anti Police Sign(s)

  • Failure to comply with an order

  • Initial threat to dispatch that they intend to fight with police, or resist arrest

  • Social media comments (negative/threatening)

  • Strangling

  • Refusal to maintain distance/separation

  • Any touching under adverse conditions

  • Throwing articles/objects at police

  • Using aggressive dogs

  • Warning to hurt himself



Discussion

The nine components detected were assessed for common themes. These themes could be characterized as: Using a weapon 1; displaying a weapon or fists; verbal threats and yelling; vehicle-related; using a weapon 2; display of hands or feet; spitting; bleeding; resisting/evading arrest. Based upon the results of the factor analysis, several recommendations emerged. First, there did not appear to be a necessity to separate acts against officers from acts against other present since none of the nine components were delineated by the recipient of the resistance involved. Second, there were clearly some types of actions that were viewed as distinct from other actions involved weapons (e.g., intentionally spitting on an officer) that a category should be present to capture those actions. Finally, a definition for passive resistance was needed to ensure that participants in the data collection clearly understood what actions were considered passive. Based upon those findings, the following changes were made to the questionnaire and instructions:

Questionnaire:

Q21a. What resistance or weapon was or believed to be involved? (Select all that apply)

  • Attempted to escape or flee from custody

  • Resisted being handcuffed or arrested

  • Barricading self

  • Using a chemical agent (acid, gasoline, pepper or OC (oleoresin capsicum) spray, etc.) against an officer or another

  • Using an edged weapon against an officer or another

  • Using an electronic control weapon against an officer or another

  • Using a firearm against an officer or another

  • Using hands/fist/feet against an officer or another

  • Displaying a weapon at an officer or another

  • Directing a vehicle at an officer or another

  • Intentionally spitting or bleeding on an officer

  • Throwing an article or object at an officer

  • Making verbal threats

  • Failing to comply to verbal commands

  • Other types of passive resistance

  • Pending further investigation

  • Unknown and is unlikely to ever be known

  • None

Instructions:

Q21a What resistance or weapon was involved?

A weapon can generally include, but are not limited to, firearm; BB or pellet gun; knife; other cutting instrument or edged weapon; electronic control weapon; explosive device; blunt instrument; chemical agent (e.g. acid, gasoline, pepper or OC (oleoresin capsicum) spray, etc.). Under certain circumstances motor vehicles or other objects could also be considered weapons if used or displayed in a threatening manner. Please mark all categories that apply.

Passive Resistance is indicated when the subject is not complying with an officer’s commands and is uncooperative, but is taking only minimal physical action to prevent an officer from placing the subject in custody and taking control. Examples include: standing stationary and not moving upon lawful direction, falling limply and refusing to use their own power to move (becoming “dead weight”), holding onto a fixed object, or locking arms to another during a protest or demonstration.

If information is unknown because the investigation is still incomplete, record pending further investigation.

If the information is not known and is unlikely to ever be known, record unknown and is unlikely to be known.

Implications for the Pilot Study

While the cognitive testing provided guidance to the FBI UCR Program on areas that needed to be more clearly defined, it cannot be used to as the final word on issues related to the collection of information on time, location, injury, and resistance/weapon. The six-month pilot study will provide an important opportunity to confirm the guidance provided in the questions and instructions of the National Use-of-Force Data Collection through the comparison of original law enforcement records to the statistical reports. Beyond the larger goals of assessing data quality and data completeness, the following questions will be specifically addressed during the pilot study:

  • Do law enforcement records confirm that the time of the use of force be acquired or estimated?

  • Do law enforcement records confirm that the location of the use of force be acquired or estimated?

  • Are the current instructions for the coding of possible internal injury and other major injury confirmed by the law enforcement records?

  • How reliably are law enforcement agencies applying the definition of serious bodily injury?

  • Are there additional categories that could be added to the array of injuries that would improve reporting?

  • Are there additional categories of that could be added to the array of resistance or weapons that would improve reporting?

After the conclusion of the pilot study, the FBI UCR Program will detail the findings of the pilot study and any recommended changes that would result from those findings.







1 The category, “none,” is collected only on the subject. It is not included in the array of choices for officer because there is a screener question (Q35) that asks whether the officer received any injuries before Q35a is asked.

27


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
AuthorCindy Barnett-Ryan
File Modified0000-00-00
File Created2021-01-22

© 2024 OMB.report | Privacy Policy