2024 LEMAS Part B 9 23 24

Part B. Collection of Information Employing Statistical Methods

1. Universe and Respondent Selection

The universe for the 2024 Law Enforcement Management and Administrative Statistics (LEMAS) survey will consist of all general-purpose law enforcement agencies (LEAs) within the United States. to represent the population of active, publicly funded primary state agencies; sheriffs’ offices; and municipal, county, or regional police departments. They are distinct from special-purpose agencies, sheriffs’ offices with jail and court duties only, and federal LEAs. The target population for the 2024 LEMAS survey is all general-purpose agencies that employ the equivalent of at least one full-time equivalent (FTE) sworn personnel.¹

Sampling Frame

The 2024 LEMAS universe is obtained using the Law Enforcement Agency Roster (LEAR), an enumeration of all publicly funded law enforcement agencies operating in the United States that RTI International (RTI) maintains for the Bureau of Justice Statistics (BJS). It includes all general purpose (i.e., local/county/regional police departments, sheriffs’ offices, and the 49 primary state and highway patrol agencies) and special purpose LEAs (e.g., park police, transit, tribal, campus, and independent school districts) in the United States. The LEAR was first developed in 2016 and has since been updated with data from public membership listings and other BJS data collections, including the 2016 LEMAS core, 2016 LEMAS Body-Worn Camera Supplement, 2018 Census of State and Local Law Enforcement Agencies (CSLLEA), 2020 LEMAS core, 2021 Survey of Campus Law Enforcement Agencies (SCLEA), 2022 Census of State and Local Law Enforcement Agencies (CSLLEA), and the 2023 LEMAS Post-Academy Training and Officer Wellness Survey (LEMAS PATOW).

While data from the 2022 CSLLEA that has been merged into the LEAR will be the primary source used to identify eligibility, additional non-BJS data sources have also been integrated into the LEAR and will be used to further verify and supplement the existing data. These include the 2023 FBI’s Police Employee Data, data from state Peace Officer Standards and Training (POST) commissions or offices, and state chief and sheriffs’ association lists. Information from all of these various sources result in the most comprehensive and up-to-date list of law enforcement agencies known.

As the first step in frame construction for the 2024 LEMAS survey, RTI will extract all cases from the current LEAR that meet the eligibility criteria. Based on data from the current LEAR and prior iterations of the LEMAS, approximately 16,000 general-purpose agencies will be eligible for the 2024 LEMAS.

Next, RTI will review the addresses, phone numbers, and agency head names of cases in the preliminary frame to identify possible duplicates. Any matches will be reviewed to identify true duplicates from non-matches with similar characteristics. For example, small police departments and county sheriff’s offices are often located at the same address. Duplicates will then be dropped from the frame.

Status issues (i.e., whether the agency is still an operational agency), potential duplicates, and other eligibility issues that are not easily resolved will be investigated by reviewing information from previous BJS data collections and through online searches. Research will be done online using publicly sourced information. Sources will include (1) records from prior surveys, (2) law enforcement and government webpages, (3) city-, county-, or organization-level budgets, (4) official governmental reports, and (5) news reports/articles from reputable sources. All updated data will be entered in the Agency Record Management System, which includes LEAR, and then tracked and approved through the standard verification process before becoming part of LEAR.

In addition, the following actions will be taken to reduce the possibility of including out-of-scope agencies from the LEAR:

1. Potentially non-publicly funded LEAs (e.g., those serving private universities) will be vetted to reduce out-of-scope agency participation.

2. Publicly available information will be reviewed to determine if agencies do not have general sworn law enforcement authority.

3. Agency size will be checked across a variety of sources, including the 2020 LEMAS core and prior waves of the CSLLEA. Agencies will also be checked against the 2022 CSLLEA in order to attempt to minimize the number of agencies in the frame with less than one FTE sworn personnel.

Once verification is complete, this list of unique LEAs will become the frame from which the 2024 LEMAS sample will be drawn.

2024 LEMAS Sample Selection

The 2024 LEMAS core survey will use the same sampling methodology as the 2020 LEMAS core and 2023 LEMAS PATOW-- a stratified simple random sample design in which LEAs are stratified by agency type and agency size. Agency type has three categories: (1) local police, (2) sheriff’s offices, and (3) state police. To obtain a representative sample of all agency sizes, agency type is further stratified by agency size. Agency size is split into seven categories: (1) 1-1.5 FTE, (2) 2 – 4.5 FTEs (3) 5 – 9.5 FTEs, (4) 10 – 24.5 FTEs, (5) 25 – 49.5 FTEs, (6) 50 – 99.5 FTEs, and (7) 100 or more FTEs.

As with previous LEMAS data collections, a survey sample of 3,500 general-purpose agencies with at least one FTE sworn personnel will be selected from the universe (Table 1). All primary state police agencies (N=49)² and agencies with 100 or more full-time equivalent sworn officers (approximately N=1,030) are considered self-representing (SR) and will be selected with certainty. All other agencies will be placed into non–self-representing (NSR) strata defined by agency type and number of officers. The sample will be selected such that the anticipated respondents are proportionally allocated across NSR strata, which will result in more cases allocated to strata with historically lower response rates.

Table 1: Sample size allocation based on the proportion to number of agencies by stratum, 2024 LEMAS

	Agency Type	Full-time Equivalent Sworn Officers	Sample Size
Self-representing	Local police	100+	656
	Sheriff office	100+	361
	State police	All	49
Non–self-representing	Local police	50–99.5	157
		25–49.5	308
		10–24.5	587
		5–9.5	483
		2–4.5	331
		1–1.5	101
	Sheriff office	50–99.5	71
		25–49.5	122
		10–24.5	163
		5–9.5	81
		2–4.5	27
		1–1.5	3
			3,500

^a Number of full-time equivalent officers.

Starting with the 2020 LEMAS core, RTI developed a new strategy to reduce burden on smaller agencies over time. NSR LEAs were assigned a permanent random number (PRN) and sorted by PRN within strata. The PRN is a random number selected uniformly between 0 and 1. After sorting the frame by the PRN, the first agencies in each stratum were selected for the 2020 LEMAS, where is the sample size for each stratum.

For the 2024 LEMAS survey, the following steps will be done to select the NSR sample:

1. Any recently identified new agencies will be added to the frame and be assigned a PRN.

2. Closed or otherwise ineligible agencies will be removed from the frame.

3. If needed, strata assignments will be updated based on new size information.

4. A new sample will be selected by beginning after the maximum PRN (following selections used in the 2020 LEMAS core and 2023 LEMAS PATOW) in each stratum and selecting the next agencies in the stratum where is the sample size for the stratum.

5. The sampling weight is calculated as where is the population size of the stratum at the time of sampling.

This reduces the probability NSR agencies have of being selected in more than one of the next five waves of LEMAS core or supplement administrations.

The LEMAS has traditionally experienced a high response rate; the 2020 LEMAS had an overall response rate of 78.4 percent. However, as seen in Table 2, the response rate varied by agency type and agency size. As a result, we will assume a response rate that differs by agency type and size. Lower response rates are assumed for sheriffs and smaller agencies, and these assumed response rates are presented in Table 3. Based on the response rate assumptions, the design calls for an overall sample size of 3,500 with 2,835 complete questionnaires expected.

Table 2: 2020 LEMAS survey response rates, by agency type and size

	Agency Type	Full-time Equivalent Sworn Officers
			Response Rate
Self-representing	Local police	100+	88.5
	Sheriff office	100+	75.5
	State police	All	98.0
Non–self-representing	Local police	50–99.5	83.7
		25–49.5	80.3
		10–24.5	80.7
		5–9.5	77.2
		2–4.5	66.7
		1–1.5	66.0
	Sheriff’s office	50–99.5	60.0
		25–49.5	67.0
		10–24.5	76.1
		5–9.5	71.0
		2–4.5	58.6
		1–1.5	N/A*
	Overall		78.4

^a Number of full-time equivalent sworn officers.

*Note: All three sampled agencies were found to be ineligible during LEMAS 2020 data collection.

Table 3: Assumed response rates for the 2024 LEMAS, by agency type and self-representation status

Agency Type	Self-Representation Status	Response Rate
Local Police	Self-representing	90%
	Non-self-representinga	80%
Sheriff’s Office	Self-representing	80%
	Non-self-representinga	75%
State Police	Self-representing	90%
All agencies		81%

^a Non-self-representing agencies are comprised of agencies with less than 100 full-time equivalent sworn personnel.

Sampling Error

Although the allocation of sample size will be made with consideration for the overall national estimates, it is not the only domain of interest. Other domains of interest include:

• Local police departments – all sizes

• Sheriff’s offices – all sizes

• State police departments – all sizes

• Local police departments – non-self-representing

• Sheriff’s offices – non-self-representing

The estimates for random sampling error (RSE) for each domain, based on 2020 LEMAS data, are presented in Table 4. RSE is used as the measure of precision in the sampling design where the RSE is the ratio of a measure and its standard error. As such, RSE is a standardized measure of precision regardless of estimate value and is useful in comparing different populations. While there are no absolute limits on acceptable RSE values, a lower RSE is often desirable which is shown in the table below. The domain with the highest RSEs is the state agencies domain. Since all agencies in this group are sampled with certainty, no allocation could improve the RSEs for this domain. Because the prevalence of female officers is lower than the other proportions of interest, the associated RSE is higher than those for the other selected statistics.

Table 4: RSEs, by selected statistic and domain, 2020 LEMAS

Domain	Budget*	BWC Usage	Community Policing	Number of FTS Officers	Number FTNS Staff	Female Officers
National	2.49%	1.53%	0.71%	2.26%	3.04%	2.59%
Local Police	2.91%	1.74%	0.77%	3.03%	4.52%	3.10%
Sheriff’s Office	5.30%	3.27%	1.82%	3.79%	4.89%	4.13%
State Police	6.83%	4.92%	1.82%	5.17%	6.95%	2.56%
Non-Self-Representing	1.65%	1.67%	0.77%	0.84%	3.39%	2.90%
Self-Representing	3.36%	0.78%	0.24%	3.41%	3.95%	0.59%
Local Police – Non-Self-Representing	1.79%	1.86%	0.82%	0.90%	2.65%	3.38%
Sheriff’s Office – Non-Self-Representing	3.62%	3.71%	2.11%	1.96%	7.04%	5.08%

*NOTE: 5 outliers from 2020 LEMAS were removed from this analysis. RTI and BJS made attempts to resolve these outliers during data collection but were unable to resolve.

2. Procedures for Collecting Information

Data Collection Procedures

The 2024 LEMAS core survey data collection effort will mirror the protocols used for the 2020 LEMAS core and 2023 LEMAS PATOW surveys with some enhancements (Table 5). There are two data collection instruments, one designed for local police departments and state police (Attachment A) and another for sheriff’s offices (Attachment B), which also mirrors the protocols used for the 2020 LEMAS core survey. Additionally, a robust nonresponse follow-up strategy will be employed to ensure an 81% response rate. This comprehensive series of mailings and non-response follow-up activities is designed to facilitate high response rates and data quality. Data collection will begin with an invitation letter (mailed via USPS), on BJS letterhead and signed by the BJS Acting Director to LEA agency heads announcing the survey (Attachment F). This letter will explain the purpose and significance of the survey. It will include the survey web address and agency-specific log-in credentials. The survey invitation letter will provide a toll-free telephone number and project-specific email address for the survey Help Desk should the POC have any questions. Accompanying this letter will be a letter of support from the International Association of Chiefs of Police (IACP) (Attachment G) and an enclosed flyer that explains the design of the LEMAS—use of tailored questionnaires for police departments and sheriff’s offices (Attachment H).

Table 5. Outreach schedule for 2024 LEMAS

Survey Communication	Project Week	Mode	Attachment
Invitation, Letter of Support, and Informational Flyer	1	Mail	F, G, H
Invitation	2	Email	I
First Reminder	3	Mail	J
Second Reminder	4	Email	K
Third Reminder	6	Mail	L
Fourth Reminder	8	Email	M
Fifth Reminder	11	Mail	N
Telephone Reminders	12-16	Telephone	O
Sixth Reminder	17	Mail	P
Seventh Reminder	19	Email	Q
Eighth Reminder	21	Mail	R
Ninth Reminder	22	Email	S
Tenth Reminder	24	Mail	T
Eleventh Reminder	26	Mail	U
End of Study Notice	30	Email, Mail	V
Thank You Notice	4-34	Mail	W

Approximately 1 week after sending the survey invitation letter, RTI will send an invitation email message that is identical to the survey invitation letter to those recipients for whom an email address is available to confirm receipt of the study materials. The email invitation will introduce LEMAS, provide a link to the web survey, login credentials for the agency, and Help Desk contact information (Attachment I).

Two weeks after sending the survey invitation letters, the first reminder will be sent via USPS to nonrespondents (Attachment J); this mailing will include a paper copy of the 2024 LEMAS instrument (Attachment A or B) and a business reply return envelope. One week later, a second reminder will be sent to all nonrespondent POCs via email (Attachment K). The reminder letter and email will express the importance of the LEMAS to the LEA community and encourage response via the online survey (or paper copy, if preferred).

Two weeks after those reminders are sent, RTI will mail a postcard third reminder to POCs (Attachment L) that will include the website information and instructions on how to download another paper copy of the questionnaire if needed. Two weeks after sending the postcard reminder, we will send a fourth reminder via email (Attachment M).

A fifth reminder (Attachment N) and a second copy of the paper questionnaire will be sent three weeks after the fourth reminder. Telephone follow-up with all non-responding LEAs will begin one week after the fifth reminder is sent (Attachment O).

A sixth reminder (Attachment P) will be sent via UPS or FedEx to all nonresponding LEAs six weeks after the fifth reminder. The sixth reminder will include instructions on how to access and complete LEMAS. Two weeks after the sixth reminder, a seventh reminder (Attachment Q) email will be sent to all nonresponding agencies with an email address encouraging them to complete LEMAS for their agency.

The eighth reminder postcard (Attachment R) will be sent to all nonresponding agencies two weeks after the seventh reminder. The eighth reminder will encourage response and include information on how to complete LEMAS. One week after the eighth reminder, the ninth reminder email (Attachment S) will be sent to all nonresponding agencies and encourage response to LEMAS.

A tenth reminder (Attachment T) and a third copy of the paper questionnaire will be sent two weeks after the ninth reminder. An eleventh reminder (Attachment U) will be sent via USPS and include a letter from an International Association of Chiefs of Police subsection (e.g., the Smaller Departments Section) encourage response. The eleventh letter will be tailored based on agency type and size. The eleventh reminder will be sent two weeks after the tenth reminder.

An end of study reminder letter (sent via USPS) and email (Attachment V) will be sent to all nonresponding agencies four weeks after the eleventh reminder. The end of study reminders will notify agencies of the upcoming due date and provide details on completing their survey.

Within three weeks of receiving a survey, the respondent will receive a thank you email or letter depending on the mode of completion (Attachment W). The thank you will acknowledge receipt of the survey and state that the agency may be contacted for clarification once their survey responses are processed.

The 2024 LEMAS survey will employ a multi-mode approach that relies primarily on web-based data collection with hardcopy surveys provided in reminder outreach, in line with standard web-first protocols. When the 2020 LEMAS core survey ended data collection, there was an overall response rate of 78.4% (about 2,700 agencies). Of these, 2,380 agencies (88%) responded via the web. Due to increased web-based capabilities of LEAs and the project’s strong encouragement to respond using the web-based data collection tool, BJS expects that most of the agencies responding to the 2024 LEMAS core survey will use the web-based option.

Data Processing

Upon receipt of a survey (web or hardcopy), data will be reviewed and edited, and if needed, the respondent will be contacted to clarify answers or provide missing information. Respondents who submit via web will be prompted with real-time validation checks when submitting missing or inconsistent data. Any unresolved items that remain after the respondent submits will result in recontact by RTI staff to the respondent to attempt to resolve these issues.

The hardcopy survey will be developed and keyed using TeleForm, which will allow the surveys to be scanned and the data read directly into the same database containing the web survey data. This will ensure that the same post-collection data quality review procedures, which mirror and expand upon the web validation checks, are applied to all survey data, regardless of response mode. The following is a summary of the data quality assurance steps that RTI will take during the data collection and processing period:

Data Editing. RTI will attempt to reconcile missing or erroneous data through automated and manual edits of each questionnaire. In collaboration with BJS, RTI will develop a list of edits that can be completed by referring to other data provided by the respondent on the survey instrument. For example, if a screening question was left blank, but the follow-up questions were completed, a manual edit could be made to indicate the intended positive response to the screening question. Through this process, RTI can quickly identify which hardcopy cases require follow-up and indicate the items that need clarification or retrieval from the respondent.

Data Retrieval. When it is determined that data retrieval is needed, an Agency Liaison will contact the respondent for clarification. Throughout the data retrieval process, RTI will document the questions needing retrieval (e.g., missing or inconsistent data elements), request clarification on the provided information, obtain values for missing data elements, and examine any other issues related to the respondent’s submission.

Data Entry. Respondents completing the survey via the web instruments will enter their responses directly into the online instrument. For those respondents returning the survey via hardcopy (mail or fax), data will be scanned once received and determined complete. Once the data have been entered into the database, they will be made available to BJS via an SFTP site. To confirm that editing rules are being followed, RTI will review frequencies for the entered data after the first 10% of cases are received. Any issues will be investigated and resolved. Throughout the remainder of the data collection period, RTI staff will conduct regular data frequency reviews to evaluate the quality and completeness of data captured in both the web and hardcopy modes.

3. Methods to Maximize Response Rates

The LEMAS core has historically achieved high survey response rates, with earlier administrations achieving above an 80% response rate. However, the 2020 LEMAS core achieved a 78% response rate after being in the field 8 months. When the response rate fell below the OMB standard of 80%, a nonresponse bias analysis was conducted to evaluate the bias levels. BJS and RTI will undertake various procedures to maximize the likelihood that the 2024 LEMAS core survey reaches a similar level of participation. For example, RTI will work with BJS to adapt survey outreach protocols based on response patterns and tailor messages by agency type similar to what was done for the 2022 CSLLEA and LEMAS PATOW.

BJS will use a web-based instrument supported by several online help functions to maximize response rates. For convenience, respondents will receive the survey link in an email invitation and a mailed hardcopy copy invitation.³ A Help Desk will be available to provide both substantive and technical assistance. BJS will supply the Help Desk with the survey flyer to help answer respondent questions (Attachment H). In addition, the web survey interface is user-friendly, which encourages response and ensures more accurate responses. Because online submission is such an important response method, close attention will be paid to the formatting of the web survey instrument. The online application will be flexible so it can adapt to meet the needs of multiple device types (e.g., desktop computer and tablet), browser types (e.g., Internet Explorer and Google Chrome), and screen sizes.

Other features in the instrument will include the following, each intended to enhance respondent experience and reduce burden, resulting in continued respondent cooperation:

• Respondents’ answers will be saved automatically, and they will have the option to leave the survey partway through and return later to finish.

• The online instruments will be programmed with data consistency checks and automatic prompts, thereby reducing the amount of item nonresponse and necessary follow-up.

• The online consistency checks will be tailored based on previously reported data or on the type and size of agency, providing the respondent helpful information in resolving any validation errors and reducing the likelihood of breakoff.

• LEAs may also download and print a hardcopy survey from the website or request one from the Help Desk.

To obtain higher response rates and to ensure unbiased estimates, multi-stage survey administration and follow-up procedures have been incorporated into BJS’s response plans. Ensuring adequate response (not just department response rates, but also item responses) begins with introducing agencies to the survey. This will be accomplished through the initial invitation letter and accompanying documents (Attachments F, G, and H). Resources available to help the respondent complete the survey (e.g., telephone- or email-based Help Desk support) will be described in detail. We will provide LEAs with online and fax methods to identify respondents and change the agency point-of-contact assignment if needed.

Nonresponse Adjustments

Unit nonresponse. With any survey, it is typically the case that some of the selected units (in this case, LEAs) will not respond to the survey request (i.e., unit nonresponse) and some will not respond to particular questions (i.e., item nonresponse), despite best efforts made to collect all the data. Using agency data from the LEAR, weighting will be used to adjust for unit nonresponse in the 2024 LEMAS. To determine which factors to use in the agency nonresponse weight adjustments, a procedure available in RTI’s SUDAAN software based on the Generalized Exponential Model will be used to model the response propensity using information from the LEAR (e.g., agency characteristics such as geography, operating budget, whether sworn personnel have arrest powers) within sampling strata.⁴ SUDAAN is a statistical software package that analyzes complex survey data. Ideally, only variables highly correlated with the outcomes of interest will be included in the model used to reduce potential bias. Given the expected differential response rates by agency type and size, the weighting adjustment procedures will attempt to minimize the bias in the estimates within these domains.

Item nonresponse will be addressed using imputation. First, BJS and RTI will identify key variables and if the level of item nonresponse is acceptable based on how the variable is used (e.g., is it used for weighting). RTI will then calculate the level of item nonresponse for each variable that is a candidate for imputation. Variables will be sorted using the below hierarchy to determine if imputation is necessary:

1. Variable used for weighting

2. Variable has relatively high amount of missing data

3. Variable may have more uncertainty

4. Variable is a continuous variable

Categorical variables that will be used for weighting will be imputed using single imputation. The weighting process cannot incorporate multiply imputed data, and therefore require a single imputation procedure. For categorical variables hot deck imputation will be used. For continuous variables, if any are used for weighting, linear regression will be used.

As previously stated, an overall response rate of 81% is expected (see Table 3). To ensure that nonresponding agencies are not fundamentally different than those that participate, a nonresponse bias analysis will be conducted if the agency-level response rate obtained in the 2024 LEMAS core survey falls below 80%. Administrative data on agency type, size, census region or division, and population served will be used in the nonresponse bias analysis. For each agency characteristic, RTI will compare the distribution of respondents to nonrespondents. A Cohen’s Effect Size statistic will be calculated for each characteristic. If any characteristic has an effect size that falls into the “medium” or “high” category, as defined by Cohen, then there is a potential for bias in the estimates.⁵ Each estimate will be included in a nonresponse model to adjust weights to minimize the potential for bias in the estimates. In addition to estimating effect sizes, an examination of early and late responders will be conducted. If late responders (i.e., those that take more contact attempts before responding) are significantly different on the key outcomes of interest, that is also an indication of potential bias. Comparison will be made to determine if the potential for bias varies by agency type and size.

4. Final Testing of Procedures

The 2024 LEMAS survey instruments are built upon previous waves of the LEMAS survey to ensure data comparability. The 2024 instruments are almost identical to the 2020 instruments, with the 2024 instruments having six fewer questions than the previous iteration and no new questions added. The 2020 LEMAS was cognitively tested before fielding, successfully administered, and reports produced based on the findings. As a result, there are no significant changes to the sampling approach, questionnaire, or data collection methodology and the success of past LEMAS iterations serves as the testing of the approach.

5. Contacts for Statistical Aspects and Data Collection

1. BJS contacts include:

Elizabeth Davis

LECS Program Manager and Statistician

202-305-2667

[email protected]

Alexia Cooper

Law Enforcement Statistics Unit Chief

202-307-0582

[email protected]

Sean Goodison

LEMAS survey Program Manager and Statistician

202-532-5148

[email protected]

2. Persons consulted on statistical methodology:

Harley Rohloff

RTI International

3. Persons consulted on data collection and analysis:

Dustin Williams

RTI International

Mark Pope

RTI International

Mac Ayers

RTI International

1 For reporting purposes, two part-time officers are treated as equivalent to one full-time officer.

2 Hawaii does not have a primary state police agency.

3 Remaining nonrespondents will receive a total of two copies of the paper questionnaire during the course of data collection.

4 Folsom, R.E., & Singh, A.C. (2000). The generalized model for sampling weight calibration for extreme values, nonresponse, and poststratification. In Proceedings of the American Statistical Association’s Survey Research Methods Section, 598-603.

5 Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Routledge, 2013.

15