Generic Clearance Memo

MEMORANDUM

MEMORANDUM TO:

Jennifer Park
Official of Statistical and Science Policy
Office of Management and Budget (OMB)

THROUGH:

Lynn Murray
Clearance Officer
Justice Management Division
Christopher A. Nicholas
Section Chief, Law Enforcement Support Section
Federal Bureau of Investigation (FBI)
Amy C. Blasher
Unit Chief
Crime Data Modernization (CDM) Team
FBI

FROM:

Cynthia Barnett-Ryan
Survey Statistician
CDM Team
FBI

DATE:

November 14, 2016

SUBJECT:

FBI Request for OMB Clearance for developmental activities
including a cognitive testing of a new National Use-of-Force data
collection on law enforcement use of force under the OMB generic
clearance agreement (OMB Number 1110-0057).

Since the shooting of Michael Brown in Ferguson, Missouri, in 2014, law enforcement uses of force have
called public attention to the need for more information on these types of encounters. To provide a
better understanding of the incidents of use of force by law enforcement, the Uniform Crime Reporting
(UCR) Program is proposing a new data collection to include information on incidents where a use of
force by a law enforcement officer as defined by the Law Enforcement Officers Killed and Assaulted
(LEOKA) Program has led to the death or serious bodily injury of a person, as well as when a law
enforcement officer discharges a firearm at or in the direction of a person. The definition of serious
bodily injury will be based, in part, on 18 United States Code (USC) Section 2246 (4), to mean “bodily
injury that involves a substantial risk of death, unconsciousness, protracted and obvious disfigurement,
or protracted loss or impairment of the function of a bodily member, organ, or mental faculty.”

1

Goal of the Proposed Data Collection on Law Enforcement Use of Force
The goal of the FBI’s data collection on law enforcement officer use of force is to produce a national
picture of the trends and characteristics of use of force by a law enforcement officer as defined by the
LEOKA Program to the FBI. The collection and reporting would include use of force that results in the
death or serious bodily injury of a person, as well as when a law enforcement officer discharges a
firearm at or in the direction of a person. The data collected by the UCR Program would include
information on circumstances surrounding the incident itself, the subjects, and the officers. The data
collection would focus on information that is readily known and obtainable by law enforcement with the
initial investigation following an incident rather than any assessment of whether the officer acted
lawfully or within the bounds of department policies. Publications and releases from the data collection
will provide for the enumeration of fatalities, nonfatal encounters that result in serious bodily injury,
and firearm discharges by law enforcement. In addition, targeted analyses could potentially identify
those law enforcement agencies with “best practices” in comparison with their peers as an option for
further study.
This FBI data collection will facilitate important conversations with communities regarding law
enforcement actions in relation to decisions to use force and works in concert with recommendations
from the President’s Task Force on 21st Century Policing. Given a growing desire among law
enforcement organizations to increase their own transparency and embrace principles of procedural
justice, this collection will expand the measure to a broader scope of incidents of use of force to include
nonfatal instances as well.

Purpose of the Research
The purpose of the research is to investigate the understanding of the language and wording of the
questions in the proposed data collection on law enforcement use of force, as well as their associated
instructions by the law enforcement community. The ultimate goal for the development research
activities is to ensure that participants have a clear understanding of what information is requested even
in complex law enforcement situations. This will aid the UCR Program in its efforts to increase the
overall validity and reliability of its data collections. The cognitive testing is a first step to understand
the extent to which the law enforcement community has a common understanding of key concepts in
the data collection. In addition, there are questions that ask participants to indicate what records are
readily available on certain key pieces of information such as time and location.

Background Research
Criminal Justice Information Services (CJIS) Advisory Policy Board (APB)—
Spring 2015
The FBI’s initial proposal was to pursue the addition of nonfatal officer-involved shootings to the existing
information on justifiable homicide. Beginning in March 2015, the FBI CJIS Division worked with its APB
to consider the question of whether to pursue the collection.

2

The CJIS APB is a committee comprised of representatives from the law enforcement and criminal
justice communities who advise the FBI Director on matters related to the criminal justice information
systems the CJIS Division manages. The APB meets semiannually and provides recommended actions on
policy and technical issues, to include the UCR Program. While this body does not have the expertise to
provide advice on matters of statistical methodology, the APB does provide two important functions.
First, its membership is a geographically diverse group of experts on criminal justice records maintained
by law enforcement agencies at all levels of government: local, state, tribal, and federal. These
representatives can provide important feedback on the potential impact of any change or addition to
UCR data collections on the law enforcement community. Secondly, the APB represents the community
from which UCR data are collected. Through the APB process, the FBI is able to engage in critical
discussions that inform decisions about content and scope of law enforcement statistics.
On June 3, 2015, the APB received the recommendation to approve the data collection and passed the
following amended motion:
“The UCR Program to develop a method to collect information on nonfatal/fatal shootings by
law enforcement in the line-of-duty. The UCR Program will work with local law enforcement
agencies and the five major national law enforcement organizations to develop what
information to collect and the best method to do so and bring the topic back through the APB
Process.”

Meeting of Major Law Enforcement Organizations
Based upon the direction of the CJIS APB, the FBI also began efforts to solicit input from the major law
enforcement organizations. Representatives from the major law enforcement organizations including
the International Association of Chiefs of Police (IACP), the Major Cities Chiefs Association (MCCA), the
National Sheriffs’ Association (NSA), the Major County Sheriffs’ Association (MCSA), the Association of
State Criminal Investigative Agencies (ASCIA), and the Police Executive Research Forum (PERF) met with
representatives from the CJIS APB, the FBI, and the Department of Justice (DOJ) to discuss the next steps
regarding the creation of the new data collection on officer-involved shootings. The result of the
meeting was a unanimous endorsement of a data collection system, to include the following definition
and content.
The participants at the meeting on September 18, 2015, proposed the following definition:
“Law Enforcement Officer’s as defined by the Law Enforcement Officer Killed and Assaulted
(LEOKA) Program Use of Force (LEOUF) that results in the death or serious physical injury to a
person, or when the law enforcement officer discharges a firearm at or in the direction of a
person.”
The effects of this definition were threefold. First, it expanded the collection to include the use of force
that results in serious physical injury, rather than the original focus of officer-involved shootings.
Secondly, the inclusion of the definition of law enforcement as set by the LEOKA Program would allow
for some flexibility to expand the scope of the use of force collection in parallel with the LEOKA data

3

collection. Finally, the definition does not require a law enforcement agency to determine that the use
of force was justified.
In addition to the proposed definition for the types of incidents eligible for reporting in the UCR use-offorce collection, the representatives from the major law enforcement organizations proposed content to
be collected on every incident that meets the criteria of the definition. The proposed data elements
included basic information on any officers involved, the subjects of the use of force, and circumstances
related to the use of force, which served as the basis for the final recommendation approved by the CJIS
APB. These data should be collected by the FBI as a part of its UCR Program but collected separately
from the rest of the criminal incident and offense information the FBI currently maintains. These
proposals were brought to the CJIS APB for consideration.

CJIS APB—Fall 2015
After the meeting with the law enforcement representatives in September 2015, the FBI introduced a
topic to address the question of whether a new data collection on law enforcement officer use of force
would be recommended to the FBI Director. On December 3, 2015, the CJIS APB approved four motions
that effectively established the scope and minimum content of the new data collection.
The scope of the data collection was defined in the following language:
“The APB recommends the collection and reporting of use of force by a law enforcement officer
(as defined by LEOKA) to the FBI. The collection and reporting would include use of force that
results in the death or serious bodily injury of a person, as well as when a law enforcement
officer discharges a firearm at or in the direction of a person. The definition of serious bodily
injury will be based, in part, upon 18 USC Section 2246 (4). The term ‘serious bodily injury’
means bodily injury that involves a substantial risk of death, unconsciousness, protracted and
obvious disfigurement, or protracted loss or impairment of the function of a bodily member,
organ, or mental faculty.”
The minimum content was identified to include basic information on the characteristics of the incident,
demographics and actions of the subject, and demographics and actions of the officer. In addition to
delineating the minimum content, the APB moved to establish a Use of Force Task Force that would
review this information and provide recommendations for changes or additions to the data elements.

Task Force
As specified in the CJIS APB Recommendations, the FBI moved to establish a Use of Force Task Force to
make the decision on the additional content of the new data collection in January 2016. The Use of
Force Task Force is comprised of representatives from major law enforcement organizations and local,
tribal, and federal law enforcement representatives. Specifically, the following organizations are
represented on the Use of Force Task Force:
•
•
•

IACP
NSA
MCCA
4

•
•
•
•
•

MCSA
PERF
ASCIA
National Organization of Black Law Enforcement Executives
Association of State UCR Programs

In addition to representatives from these major organizations, the Task Force welcomed observers from
the Office of the Deputy Attorney General, the Community Oriented Policing Services Office, the DOJ,
and the Bureau of Justice Statistics (BJS).
The Task Force met on four occasions (January 27, 2016; March 17, 2016; May 4-5, 2016; and August 3,
2016). The final set of data elements recommended by the Use of Force Task Force was based upon
discussions from these four meeting and, importantly, the input of the BJS on their experiences and
research that formed the basis of the Arrest-Related Death (ARD) Collection. Many of the data elements
and concepts in the National Use-of-Force Data Collection were mirrored from the BJS’s ARD Collection.
Because the ARD Collection has been through significant testing, this particular cognitive test focuses
upon areas that are new (such as in the case of interpreting the concept of serious bodily injury) or is
captured in a different fashion than the ARD (e.g., the concept of active aggression).

Data Elements of the National Use-of-Force Data Collection
Incident Information


Date and time of the incident



Total number of officers who applied actual force during time of incident



Number of officers from your agency who applied actual force during time of incident



Location of the incident [physical location acceptable through address or latitude and longitude]



Location type of the incident [location codes from the National Incident-Based Reporting System
(NIBRS)]



Did the officer(s) approach the subject[s]? [Yes/No/Pending/Unknown]



Was this an ambush incident? [Yes/No/Pending/Unknown]



Was a supervisor or a senior officer acting in a supervisory capacity present or consulted at any
point during the incident? [Yes/No/Pending/Unknown]



Reason for initial contact between subject and officer [response to unlawful or suspicious
activity/medical, mental health, or welfare assistance/routine patrol other than traffic
stop/traffic stop/warrant service/service of a court order/mass demonstration/follow up
investigation/ other/unknown]
o

If this was due to “unlawful or criminal activity,” what were the most serious reported
offenses committed by the subject prior to or at the time of the incident? [NIBRS
offense codes drop down, 3 boxes]

5

o



If applicable, NIBRS (or local) incident number of report detailing criminal incident
information on subject or assault or homicide of law enforcement officer.
If incident involved multiple law enforcement agencies, case numbers for the local “use of force
reports” at the other agencies.

Subject Information
•

Age, sex, race, ethnicity, height, and weight (with range of values) of the subject(s)

•

Injury/Death of subject(s) [gunshot wound/apparent broken bones/possible internal
injury/severe laceration/loss of teeth/other major injury/unconsciousness/death/
pending/unknown]

•

Type(s) of force used connected to serious bodily injury or death [firearm/electronic control
weapon (Taser)/explosive device/pepper or OC (oleoresin capsicum) spray/baton/impact
projectile/blunt instrument/hands-fists-feet/canine/other/pending/unknown]

•

Subject(s) resisted? [Yes/No/Pending/Unknown]

•

Was the threat by the subject(s) directed to the officer or to another party? [Officer/Other
party/Pending/Unknown]

•

Type(s) of subject resistance/weapon involvement [threatened officer/threatened
others/threatened self/active aggression/edged weapon/firearm/vehicle/chemical/
electronic/verbal/passive resistance/resist being handcuffed or arrested/attempt to escape or
flee from custody/none]

•

Apparent or known impairment/physical conditions of subject? [Yes/No/Pending/Unknown]
•

If Yes, indicate which [mental health/alcohol/drugs/unknown]

•

At any time during the incident, was the subject(s) armed or believed to be armed with
a weapon? [Yes/No/Pending/Unknown]

Officer Information
•

Age, sex, race, ethnicity, height, and weight of the officer(s)

•

Officer’s years of service as a law enforcement officer (total tenure) [Number of years]

•

Full-time? [Yes/No]

•

Was the officer readily identifiable? [Yes/No]

•

Was the officer on duty at the time of the incident? (Yes/No)

•

Did the officer discharge a firearm? [Yes/No]

•

Officer(s) injured [Yes/No]

•

Officer injury type [gunshot wound/apparent broken bones/possible internal injury/severe
laceration/loss of teeth/other major injury/unconsciousness/death]

6

Overview of Comprehensive Testing Plan
The FBI acknowledges that managing the scope of this collection and providing good guidance will be a
challenge. In order to manage this effort, the FBI completed a Comprehensive Testing Plan to outline a
series of activities to help inform the FBI on the decisions impacting scope, content, and participation
levels. This document was forwarded to the OMB on September 2, 2016, and sets forth the expected
activities to occur both before data collection commences and at the onset of data collection with a pilot
study.
The pre-testing activities occurring before data collection consists of three primary efforts all of which
build upon each other for planning the pilot study. The first is a cognitive testing effort to further
research some concepts connected to data elements included in the National Use-of-Force Data
Collection at the request of the law enforcement community. This memo reflects the methodology for
the cognitive testing. The second activity during pre-testing is a canvass of state UCR Program managers
and state CJIS System Officers to gather information on programmatic and technical capabilities of the
states in anticipation of the launch of the National Use-of-Force Data Collection. The final pre-testing
activity is a small-scale assessment of the questionnaire design that will use volunteers from the Use of
Force task force to use the questionnaire and provide feedback during a focus group.
The pre-testing activities will provide critical information that will allow for the FBI to finalize plans to
conduct a pilot study over the course of the first six months of data collection using a targeted group of
law enforcement agencies. The goal of the pilot study is to assess the interpretation of questions used
in the National Use-of-Force Data Collection and any guidance or instructions included in the data
collection. This assessment will be based on a comparison of the original law enforcement record to the
submitted responses to the questionnaire for pilot agencies. In addition to the record comparison, an
on-site review of records for a sample of agencies will be conducted to assess the extent of nonresponse
for in-scope incidents for participating agencies. The pilot study provides the best path to assess the
data collection in the context of complex law enforcement decisions.

Background Research on Cognitive Testing Instrument
The cognitive testing instrument was developed with input from the law enforcement community
(through the Use of Force Task Force membership), the BJS, and William Bozeman, M.D. Dr. Bozeman is
a physician in the Department of Emergency Medicine at Wake Forest University. He has been
extensively published in the research area of injury and law enforcement use of force and is a member
of the IACP Police Physicians Section. We anticipate continuing our collaboration with all three parties
for both pre-testing activities and the pilot study. Based upon input from all parties, draft questions
were revised to reflect the final version attached to this document.

Methodological Plan
The cognitive testing will be primarily focused upon the language and construction of the response
categories rather than the usability of the Web-form that is under construction or other questions on
mode of collection. These usability tests will be conducted as a part of system development. The
7

purpose of the cognitive test is to identify key concepts that may have the potential for a high amount of
variability in their interpretation. These areas will require thorough explanation to promote the
reliability of the information measured. This planning for this initial cognitive test will commence in
November 2016.
Participants will be recruited through regular points of contact available to the FBI. The FBI will seek to
recruit up to 400 participants with representation from each of the four regions of the United States, as
well as the major types of law enforcement agency (e.g., municipal police department, sheriff’s office,
state bureau of investigation, campus law enforcement, and federal agencies). The three main sources
for participants include the Use of Force Task Force, state UCR Program managers, and FBI National
Academy attendees. The 13 Task Force members and 50 state UCR Program managers or state points of
contact will be asked to solicit two participants from their own agency or roster of agencies under their
management for a potential maximum of 112 participants. In addition, the FBI will solicit participation
from the 280 participants in the FBI National Academy. These 280 potential participants represent the
total roster of the current FBI National Academy class. The FBI National Academy is a 10-week training
program of leaders and managers of state, local, county, tribal, military, federal, and international law
enforcement agencies.
The content and scope of the National Use-of-Force Data Collection was constructed based upon the
consensus of representatives from the law enforcement community. Through the CJIS APB and the
work of the Use of Force Task Force, the law enforcement community has indicated that this
information is valuable for understanding the circumstances surrounding a use of force by law
enforcement and that the information exists in local records on these events. However, there is a lack
of information to understand the extent to which the law enforcement community applies certain terms
on a consistent basis.
The purpose of these questions on the cognitive testing instrument to identify areas where there might
not be a common understanding of the same terminology. In essence, the results of the test will
provide a general “yes or no” response to the question of whether there is an existing normative
understanding of some concepts in the National Use-of-Force Data Collection. This questionnaire is not
going to be used to understand how the terminology may be applied on complex law enforcement
scenarios. The FBI plans to do further analysis on the application of definitions and guidance during the
upcoming pilot study as described in the Comprehensive Testing Plan provided to the Office of
Management and Budget via email on 09/02/2016.
The areas that will be addressed in the cognitive testing include the following:





The assignment at the time of the incident
The selection of the location and location type (because many location types are not mutually
exclusive)
Further exploration on the request identifying aggression
The application of the legal definition of serious bodily injury

8

On each of these particular concepts, the participants will be presented with a series of questions.
Some of the questions will involve a simple “yes” or “no” response based on how information is
recorded by law enforcement. Other questions will present an array of responses for their ranking or
interpretation. For example, on the question of serious bodily injury, a list of potential injuries will be
offered to participants. Each participant will indicate the injuries that he or she understands to be
“serious” based upon the definition provided. The testing instrument will be paper-based and emailed
to participants once consent has been obtained. This phase of the cognitive testing will be concluded by
early December 2016.

Selection of Participants in Cognitive Testing
The cognitive testing will seek to ensure that participants are representative of the varieties of law
enforcement agencies in the United States, as well as some geographic diversity. The FBI National
Academy classes are drawn from a variety of law enforcement agencies to include local, county, state,
tribal, military, federal, and international law enforcement agencies.
The recruitment of additional participants will be done by the members of the Use of Force Task Force
and the points of contact with each state that is used by the FBI for its criminal justice information. The
points of contacts will be instructed to identify one volunteer that represent someone who would be a
typical user of the National Use-of-Force Data Collection from two different types of law enforcement
for a total of two per state.
While the inclusion of two participants from each of the states increase the size of the respondent pool,
these potential participants were included at the request of the law enforcement community. The FBI
has been engaging the law enforcement community throughout this process in order to increase overall
participation in the upcoming National Use-of-Force Data Collection. The FBI wishes to open this
cognitive testing to each state point of contact in order for stakeholders to voluntarily provide assistance
in the establishment of the new data collection and as a result, increase the sense of ownership over the
data collection.
In order to monitor the general distribution of the pool of participants, there are questions on the
cognitive testing instrument that will allow for the participant to select the type of agency that he or she
represents, as well as the geographic region of his or her agency. The FBI will include frequency tables
for these variables in the final report of the cognitive test.

Cognitive Testing and Developmental Activity Procedures
Language
The cognitive testing will be conducted in English.

9

Burden Hours for Developmental Activities
The FBI is requesting 200 burden hours for this activity based upon 400 participants to complete a
survey lasting 0.5 hours for each administration. The burden hours were estimated based upon the
timed responses from a small set of former law enforcement officers currently employed by the FBI.

Analysis Plan
Initial analysis of the questions will focus on basic descriptive statistics to provide some indication of the
variability of the data. This initial analysis may be sufficient for the early questions about time and
location in order to make recommendations for final instruction and guidance on those particular
concepts. Of primary importance is to understand the extent of time and location information beyond
what is customarily recorded through most law enforcement dispatch systems. If responses indicate
that law enforcement has records to support both time and location beyond those associated with the
initial dispatch information, the FBI would have the possibility of requesting time and location
associated with other aspects of the use-of-force incident.
For the questions related to active aggression, the concept of “armed,” and interpretation of serious
bodily injury, there may be a benefit for further analysis on inter-item reliability (e.g., Cronbach’s Alpha
or exploratory factor analysis to identify additional conceptual constructs). If the results of the cognitive
testing questions indicate very poor inter-item reliability on these concepts indicating that there is a lack
of a tacit understanding within the law enforcement community, the FBI will provide explicit guidance
on the consistent coding of these particular questions. These responses to these questions will also be
monitored carefully during the initial pilot study that will take place at the beginning of overt data
collection in order to assess the consistent application of these definitions by the law enforcement
community.
At a minimum, the results of this analysis will be documented in a report available to requesters.
However, the results of this analysis will more ideally be presented before interested groups, such as
academic and law enforcement conferences, or published in peer-reviewed journals.

Informed Consent, Data Confidentiality, and Data Security
Participation is voluntary for participants in the cognitive testing. Any potential participant who does
not wish to participate would be free to refrain from participating as stated in the introductory
paragraph on the survey instrument. A cover sheet detailing the purpose of the survey and its voluntary
nature along with contact information for questions will be provided to every participant. No identifying
information will be requested from the participants on the instrument itself. Any demographic
information that is collected on the respondents will be separated from the respondents’ responses in
order to lessen the likelihood that individuals could potentially be linked back to their identities. Data
files will be housed on an internal FBI computer system and are subject to its data security policies.

10