SS Part B - OMB 1121-0314

Supporting Statement – Part B. Statistical Methods

1. Universe and Respondent Selection

The FIST program is unique in the varied nature of the checking agencies to be surveyed. This uniqueness is a product of federal law (notably the Brady Handgun Violence Prevention Act of 1993), plus a variety of state statutes that govern how background checks for firearm transfers and permits are conducted within certain states.

As described in Part A, a state may operate as a partial POC that conducts some NICS transfer checks, or as a full POC that conducts all transfer checks. Alternatively, a state may decide to operate as what FIST refers to as a “NICS FBI state” that is entirely reliant on the FBI to conduct firearm background check activities. All fifty states and the District of Columbia fall into one of these basic NICS transfer check categories. Further, state and local checking agencies in many states are authorized to conduct background check activities in addition to the NICS transfer checks, as required by state statutes. For example, a state may rely on the FBI to conduct transfer checks while a state agency reporter or local agencies are authorized by state law and are responsible for conducting an exempt carry permit. In these instances, FIST obtains information from the FBI and from the state or local checking agencies responsible for conducting the additional checks.

For the purposes of FIST, BJS uses data from three general sources to generate the national FIST estimates of firearm applications and denials:

• FBI – includes the states that rely exclusively on the FBI NICS to conduct firearm background check activities for handgun and long gun transfers, plus the partial POC states that rely on the FBI for long gun checks only. BJS will obtain counts of background check activities directly from the FBI for these states.

• State agency reporters – include full or partial NICS POC states, agencies that conduct checks required by state law, and those states where local agencies conduct background checks but report their activity to a state-level entity. BJS will administer the FIST survey or use state-issued reports to collect complete statewide counts of applications for firearm transfers or permits, denials, and reasons for denial (as applicable) from these state-level entities.

• Local agencies – include local agencies in twelve states that are authorized to conduct firearm background check activities and do not report counts to a state agency. BJS will administer the FIST survey to obtain data on applications for firearm transfers or permits, denials, and reasons for denial (as applicable) from local agencies, either through an enumeration or a sample of local agencies within the state.

Attachment VII details the FIST data reporting sources and provides additional explanation about how FIST obtains information from each state.

Eligibility

BJS and REJIS worked extensively in 2012 to revise the FIST methodology, including developing a new FIST sampling plan, redesigning the FIST survey, and assessing the criteria for respondent eligibility. BJS determined that eligible agencies should be defined as those state and local agencies that are authorized to conduct and are known to conduct background checks. Such agencies are accounted for in the universe because these are the agencies that collect or maintain data on the critical FIST data elements: applications, denials, and (as available) reasons for denial.

For the FIST program, there is an important distinction to be made between agencies authorized by statute to conduct background checks and those that actually do the checks. Although in certain states local agencies are legally authorized to conduct background checks for firearm transfers or permits, these agencies are not required to do so. As part of the 2012 efforts to develop a new FIST sampling plan, REJIS identified a sizable number of instances where a local agency (usually a municipal police department) that was legally authorized by state statute to conduct a background check had never actually conducted background check activities and was unlikely to ever do so. Instead, transfer and permit applicants who might use such a local agency are directed to another local authority (usually the county sheriff) with jurisdiction to conduct a transfer check or issue a permit. For the purpose of FIST, BJS determined that these agencies should be considered out of scope because they do not actually conduct firearm background check activities and/or track information on such activities, which are the critical data collection items on the survey.

Thus, the local agencies that have delegated background check functions but do not actually conduct background checks are not included in the frame. Accordingly, only those agencies that are known to conduct background checks or have some involvement in or information on the application process for a firearm transfer or permit are represented.

Frame generation and frame maintenance activities

State statutes determine which agencies conduct background checks for a firearm permit or transfer, and there are typically few changes from year to year. REJIS routinely researches these laws to identify changes that may impact the FIST frame. Additionally, BJS and REJIS have developed specialized knowledge of the background check processes and practices by state, and have used this information to maintain and update the FIST universe. For example, based on REJIS’s extensive research to maintain the FIST universe each year, BJS has determined that —

• In most states where the authorized background check agency is a county-level agency, it is assumed that all of these agencies (often sheriff’s offices) are eligible to be in the FIST frame, unless information about a particular agency’s checking or permitting status is known from past FIST collections; and

• In certain states, the authorized checking agency or reporting agency is not a law enforcement agency at all, but rather another county government authority. For example, in Georgia, the authorized checking agencies are the county probate courts, and in New York, most of the agencies reporting to FIST are county clerks’ offices, depending on the county.

BJS proposes to use the 2015 FIST frame to create the 2016 frame, and REJIS has reviewed applicable state laws, ATF decisions, and responses to the screener questions on the 2015 FIST survey to update the frame as appropriate.

REJIS, under BJS’s direction, used multiple data sources combined with a large known pool of past FIST responders to generate and verify the 2015 FIST universe, including —

• The 2013-2014 FIST frame, updated with information from the 2014 collection on agencies eligible to participate in FIST;

• Published agency lists from professional officer associations such as state sheriffs’ and police chiefs’ association lists and officer standards and training office lists; and

• FIST respondents’ answers to the screener questions on the 2015 survey related to firearm background check functions and responsibilities.

The 2016 universe will be updated to include the following additions that reflect ATF’s determination that the carry permits in these two states are qualified as alternatives to the NICS point-of-transfer check:

• Sixty-seven Alabama county sheriffs will be added; and

• One state agency reporter (the Ohio State Attorney General) will be added.

Table 4 shows the projected 2016 FIST universe based on REJIS’s ongoing frame maintenance activities and details how BJS will obtain information by data source and jurisdiction.

Table 4. 2016 FIST Universe
Data Collection Method	Reporting Agency Type
Data reported to BJS by the FBI	FBI NICS (n=31*)		AL, AK, AR, AZ, DC, DE, GA, ID, IN, KS, KY, LA, MA, ME, MI, MN, MO, MS, MT, ND, NM, NY, OH, OK, RI, SC, SD, TX, VT, WV, WY
Data collected by BJS for FIST	State agency reporters (n=34)		AK, AZ, AR, CA, CO, CT, DC, FL, HI, IL, KS, KY, LA, MD, MA, MI, MN, MS, NE, NV, NH, NJ, ND, OH, OR, PA, RI, SC, TN, TX, UT, VA, WI, WY
	Local agencies (n=1,333)	Sampled states	Population Category	GA	MN	WA	Total
			1) 1-9,999	33	235	81	349
			2) 10,000-99,999	101	126	86	313
			3) 100,000-199,999	14	2	5	21
			4) 200,000+	11	2	6	19
			Totals	159	365	178	702
		Census states	Population Category	AL	IA	ID	MT	NC	NE	NV	NY	WV	Totals
			1) 1-9,999	2	24	16	35	3	105	7	2	11	205
			2) 10,000-99,999	53	69	24	19	70	25	8	33	41	342
			3) 100,000-199,999	5	4	2	2	17	1	0	11	3	45
			4) 200,000+	7	2	2	0	10	2	2	14	0	39
			Totals	67	99	44	56	100	133	17	60	55	631

*BJS also obtains data from the FBI on applications and denials for long gun transfers only in IO, MD, NC, NE, NH, WA, and WI.

BJS and REJIS will continue to complete similar frame building and maintenance activities annually to review and update the frame for future FIST collections, and will continue to assess the availability of other methods to ensure accuracy and completeness.

2. Procedures for Collecting Information

The 2016 FIST data collection¹ will be administered through a survey to thirty-four state agency reporters that serve an entire state population, including Washington, D.C., a statewide census of local checking agencies in nine states, and a sample of local checking agencies in 3states. REJIS will combine the data obtained from state and local agencies with FBI NICS transaction data² to create an estimate of the total number of applications received and denied for the year. The 2016 FIST collection will use a multi-mode design to allow respondents to submit data via web-form, email, paper survey, phone, or fax. Recognizing that this is a voluntary survey, BJS and REJIS will make all reasonable efforts to make the data collection process as convenient and least burdensome as possible for respondents.

State agency reporters

REJIS has routinely obtained data for several state agency reporters from state-issued reports on criminal justice statistics or publicly available sources, such as state websites. REJIS maintains a list of agencies that provide data via these methods and will obtain the data directly from the state reports or websites, where practical, in order to reduce the respondent burden. In these circumstances, REJIS will verify the accuracy of the data with the agency. There will be no duplication of effort required for the state agencies that choose to submit information through these methods, and the associated burden will consist of the time spent emailing the reports to REJIS and/or verifying data. For agencies that do not provide data via state reports or publically available websites, REJIS will send the survey notification letter request to the agency POC with a link to the FIST web-form and REJIS’s contact information.

Local checking agencies

REJIS maintains a comprehensive list of local agencies, agency POCs, and the type of background check each agency is responsible for conducting. REJIS will use this information to tailor the language in the survey correspondence to each agency’s function to reduce respondent burden.

FBI NICS and ATF data

REJIS will obtain relevant data from the FBI NICS section and ATF Denial Enforcement and NICS Intelligence (DENI) Branch to complete the analysis and preparation of the FIST estimates. REJIS will obtain data from the FBI NICS Section on federal transactions and denials for the states for which the FBI conducts all or part of the NICS checks. REJIS will also obtain data from the FBI’s NICS Operations report and other publically available FBI data on reasons for denials and NICS Index totals, and from the ATF DENI Branch on FBI denials that are screened by DENI and referred to the ATF field offices for investigation to compile the relevant data on post-denial activities.

2016 Sampling Plan

For the 2016 FIST collection, BJS is requesting OMB approval to use a similar sample design and sampling plan implemented for the two most recent collections (2014 and 2015). The vast majority of FIST data is comprised of counts provided by the FBI and state agency reporters, with a relatively small proportion (less than 10%) of the FIST national estimate derived from the local agency population. Table 5 shows the proposed 2016 FIST sampling frame.

Table 5. Proposed 2016 FIST Sampling Plan
Reporting Agency Type													n
State agency reporter		AK, AZ, AR, CA, CO, CT, DC, FL, HI, IL, KS, KY, LA, MD, MA, MI, MN, MS, NE, NV, NH, NJ, ND, OH, OR, PA, RI, SC, TN, TX, UT, VA, WI, WY											34
Local agencies	Sampled states	Population Category	GA	MN	WA	Total
		1) 1-9,999	23	94	49	166
		2) 10- 99,999	71	50	52	173
		3) 100,000-199,999	14	2	5	21
		4) 200,000+	11	2	6	19
		Totals	119	148	112	379							379
	Census states	Population Category	AL	IA	ID	MT	NC	NE	NV	NY	WV	Totals
		1) 1-9,999	2	24	16	35	3	105	7	2	11	205
		2) 10,000-99,999	53	69	24	19	70	25	8	33	41	342
		3) 100,000-199,999	5	4	2	2	17	1	0	11	3	45
		4) 200,000+	7	2	2	0	10	2	2	14	0	39
		Totals	67	99	44	56	100	133	17	60	55	631	631
Total Agencies for Potential FIST Collection													1044

BJS proposes to again employ an enumeration of local agencies in nine states (AL, ID, IA, MT, NE, NV, NY, NC, and WV) after determining that a census will produce more reliable results than sampling due to the small number of agencies overall that operate within the state, and within stratum, as well as the uniqueness and diversity of the FIST agency respondents. This approach will also enable BJS to produce more reliable state-level estimates.

BJS will sample agencies responsible for conducting background checks in three states (GA, MN, and WA) due to the greater number of agencies in these states compared to the other nine. In these states, a stratified sample will be created based on population size that roughly equates to:

• Category 1) rural – places of less than 10,000 population (556 agencies)

• Category 2) small cities – places of between 10,000 and 99,999 population (655 agencies)

• Category 3) small metropolitan areas – places of between 100,000 and 199,999 population (66 agencies)

• Category 4) large metropolitan areas – places of 200,000 or more population (58 agencies)

These population categories were chosen to be consistent with definitions of various census place levels. Additionally, they allow the collection to maintain some consistency with portions of the methodology used for previous FIST collections. During the 2012 FIST methodology redesign, BJS and REJIS took several steps to determine if correlates could be found between population size and the number of applications, including several variables (such as the number of sworn officers) from BJS’s 2008 Census of Local Law Enforcement Agencies. BJS determined that using the four standard Census Place population categories correlated better with the number of applications than the other variables.

Sampling will be done for agencies in Categories 1 and 2, while the agencies in Categories 3 and 4 will be kept in the sample with certainty. BJS will use a random sample, proportionate to size, for each sampled state. The sampling design was determined based on the appropriate precision needed to calculate a reliable national estimate and individual state-level estimates, and to improve the overall robustness of the sample. Attachment VIII provides the formula used to determine the sample size and weight allocation for the stratified sample of the three states taken as a group as well as the formula used to determine the sample size of a random sample in each state. Appropriate sampling weights will be assigned such that state-level estimates may be made in addition to a national estimate.

Changes to 2016 sampling methodology

For the past three collection years (2012, 2014, and 2015), BJS created reserve samples in the sampled states. The reserve samples were randomly selected and were deployed if the response rate within the state was less than 85% after the data collection for the main sample was complete.

BJS and REJIS reviewed the survey paradata on response burden from prior year collections and determined increasing the sample size would more effectively mitigate nonresponse and realize time efficiencies in the data collection schedule. The survey paradata revealed that the reserve samples used in prior year collections did not successfully increase the response rates in MN and WA. Additionally, the reserve samples were deployed in GA in all three years because the initial response rate never reached the 80% target. Further, the use of the reserve samples added up to eight weeks to the data collection schedule, which created timeliness issues for data analysis and publication.

The allocation of reserve samples has proven to be inefficient for the FIST data collection and unnecessarily extends the data collection period. BJS proposes to increase the 2016 FIST sample to 1,044 to include what previously had been reserve samples and update the frame to include newly eligible agencies. This sample size will be sufficient to address potential variance within each state. This change will improve the data collection timeframe.

Prior year FIST response rates

As shown in Table 6, the overall 2015 FIST response rate was about 81%. All (100%) of state agency reporters and about 81% of local agencies provided data. REJIS will continue to employ similar strategies for the 2016 collection to maximize the response rate and minimize the respondent burden, including employing multi-modal submission options, following a rigorous contact schedule, and tailoring FIST correspondence to individual agencies to include language specific to the types of permits and checks that the agency is responsible for conducting.

Table 6. FIST Response Rates, 2012-2015

	2012	2014	2015
Overall	74% (n=643/868)	80% (n=679/845)	81%(n=777/963)
State agency reporters	100% (n=32/32)	100% (n=32/32)	100% (n=33/33)
Local agencies	73% (n=611/836)	80% (n=647/813)	81% (n=744/930)

2016 FIST questionnaire

BJS proposes to retain the current questionnaire to obtain 2016 data (Attachment II). The survey includes questions implemented in 2015 (with OMB’s approval) that were designed to obtain more detailed information on how agencies track denials, why agencies are unable to track denials (if applicable), and what resources would enable them to begin or improve tracking summary statistics of denials (if applicable).

These questions obtain more complete information on processes and resources, and do not require respondents to query or tabulate new or additional information. Through the addition of these questions, BJS’s goal is to better understand how agencies track and report on reasons for denial, and identify ways that BJS may potentially be able to provide assistance to improve the reporting capabilities or improve how the data are reported. BJS and REJIS are currently analyzing the 2015 responses and have found that local agencies reported the following as key barriers to collecting these data: 1) a lack of understanding that reasons for denial are legally allowed to be collected, 2) lack of a tracking mechanism, and 3) lack of an agency policy requiring the data to be maintained.

BJS is currently assessing the responses to the questions to determine if and what type of assistance can be provided through FIST, or potentially through the NCHIP or NARIP programs, to enhance agencies’ abilities to collect and share data on reasons for denial. For example, BJS may be able to create some type of data collection mechanism, such as an Excel spreadsheet for agencies to use to tally reasons for denial, or use NCHIP or NARIP funds to support FIST data collection and tracking efforts.

New agencies will be added to the 2016 frame and the sample of local checking agencies will be redrawn. BJS will also redraw the FIST sample for the 2017 and 2018 collections. This will enable BJS to collect information from a new subset of agencies, as well as determine if agency needs change from the prior year. BJS proposes to retain these questions on the 2016 FIST survey to obtain more information on needs that can be used to inform decisions and on potential ways that BJS can provide assistance.

Estimation procedures

BJS and REJIS propose to employ similar estimation procedures in 2016 that were used for the 2014 and 2015 collections. BJS will generate the national FIST estimates of annual firearm purchase applications and denials by adding the state agency counts and the total local agency counts and estimates to the FBI NICS transaction totals. As addressed previously, the FBI electronically sends REJIS a report with data on transactions and denials for monthly and year-end aggregate totals (the FBI refers to these data as Total Federal Inquiries and Federal Denials). REJIS will apply weighting factors (described below) to integrate these data with the FIST survey data to generate a national estimate of applications and denials.

Weighting and nonresponse weighting adjustment

The 2016 FIST data collection will provide for two basic weighting structures for respondent agencies: a weight applied to self-representing (SR) agencies and a weight applied to non-self-representing (NSR) agencies.

Self-representing agencies (enumerated)

Each checking agency within the nine states (AL, IA, ID, MT, NC, NE, NV, NY, and WY) in which all known eligible agencies were contacted will receive a base weight of 1 (w₁ = 1). In addition to the base weight, a nonresponse adjustment will be applied to responding agencies to compensate for those agencies who did not respond (w₂). Because bias may be introduced with a nonresponse adjustment, BJS will mitigate response bias by controlling adjustments to population size. Therefore, the nonresponse adjustment will consist of a ratio adjustment of the sum of all agencies’ populations served in the universe (within each state and population size category) to the sum of the populations’ served by all respondent agencies (again, within each state and population size category).

Non self-representing agencies (sampled)

The process for calculating weights for NSR agencies³ will be similar to that for SR agencies with the exception that these agencies (in strata 1 and 2) will receive a base weight greater than 1.00 according to the population-based stratum and the state in which they reside. Weights will be adjusted for any agencies that are identified to be out of scope in the sampled states. Attachment IX shows the base weights that were created in 2015 for the three NSR states from strata 1 and 2 (GA, MN, and WA).

Weighting strategy

The final weights applied to each FIST case will be the product of a base weight applied to each agency and a nonresponse adjustment weight (w₁ x w₂ = Fw). For the purposes of the FIST collection, agencies considered to be out of scope (ineligible) will be those that have indicated they do not currently have a role in the issuance of firearm permits and transfers or are not actively conducting background checks for them.

Item nonresponse imputation

For the 2014 FIST collection, REJIS determined that there were a negligible number of cases where information on applications for firearm transfers or permits (a critical data element) was missing. There were more cases of missing data for denials but still very few compared to other missing data (e.g. reasons for denial).

REJIS is currently assessing item nonresponse for the 2015 data. Findings so far suggest that a similar pattern of item nonresponse will be determined.

To address cases of missing data for the 2015 FIST collection, REJIS will conduct a mean value imputation within each state by population-based stratum. The use of imputed data may result in a slight deflation of estimated standard errors; however, there have been relatively few instances of imputations of applications and denials relative to actual responses and nearly 90% of the FIST estimate is derived from actual counts. The impact of imputation will be minimal.

Estimating missing state-provided information for annual applications and denials

To address missing data for state agency reporters, REJIS will apply a linear interpolation or similar procedure to the data to estimate the number of applications and denials by state. The exact nature of the method used will need to be determined at the time of analysis based on presentation of the data in the context of prior years. BJS anticipates that there will minimal impact on the standard errors for the overall FIST estimates.

Degree of required accuracy

REJIS will calculate standard errors for the sampled states and national estimates. As noted, three states will be sampled (GA, MN, and WA). The national estimate will be composed of totals from the sampled states, complete counts from nine states with local checking agencies and the thirty-four of state agency reporters, and the data obtained from the FBI for the FBI NICS states. The thirty-four state agency reporters and the data reported by the FBI represent the vast majority of respondents. About 90% of the national estimate comes from FIST reporting on complete counts of background check activity, which means that standard error calculations will based on less than10% of the national estimate. Nonresponse adjustments will be made to account for error from nonresponse.

REJIS will also calculate confidence intervals for the estimated portion and the overall national FIST estimate at the 95% confidence level.

2014 and 2015 FIST standard errors⁴

For the 2014 collection, REJIS computed standard error calculations for the estimates of total applications, total denials, and the ratio of denials to total applications for purchase, transfer and concealed carry permits across states, and size of region for local agencies. The standard error computations take into account several aspects of the FIST design, including the stratification of data collection by a combination of state and population served categories and finite population sampling (without replacement) across the states and population categories of interest, as well as weighting adjustments for nonresponse. Attachment X provides the 2014 FIST standard errors (BJS and REJIS are still in the process of analyzing the 2015 data).

BJS is following the same process to compute standard errors for the 2015 data, and proposes to follow similar procedures to calculate standard errors for the 2016 collection. BJS will publish the standard error tables in the Background Checks for Firearm Transfer, 2016 – Statistical Tables.

Issues unique to the FIST data collection

The functions that checking agencies in the FIST universe are responsible for conducting are based on federal law and a variety of state statutes that govern how background checks for firearms transfers and permits operate within each state. Moreover, the terminology used for background check functions varies across jurisdictions and using terms different from what an agency is accustomed to can be a source of confusion for respondents, notably for local agencies representing smaller and rural areas that are familiar with only their state-specific terminology as it relates to firearm transfers and permits. While most of the FIST local agency respondents are law enforcement agencies, other agency types are also included (such as probate court or county clerk offices). These agencies may not be familiar with the jargon used by law enforcement agencies or the terms may not be relevant to the work they conduct.

Beginning in 2014, REJIS used modified language in the FIST correspondence and on the survey instruments to match the terms that each state is familiar with (e.g., referring only to the permit types that the agency is responsible for conducting and tailoring permit type names to match those used within the state) in order to improve the accuracy and reliability of the FIST data, as well as to lessen burden and maximize the response rate. Since the revised correspondence was implemented, REJIS reported that they fielded fewer questions, which suggests that the revised language was effective. Therefore, BJS proposes to continue this strategy for the 2016 collection. Attachment XI provides a list of terminology used for firearm checks and permits, by jurisdiction.

Sampling of specific state and local issues

Through its extensive history working with the FIST collection, REJIS has developed a comprehensive list of issues unique to each state and has accounted for these in the FIST methodology, including:

Georgia

The local POCs in Georgia for the FIST survey are Probate Courts (the only probate court responders in the FIST collection), which provide information on exempt carry permit applications and denials. Administration of the FIST survey has shown that these permits are most commonly known within the Georgia court system as “Weapons Carry Licenses.”

Minnesota

There are both state level and local FIST POCs in Minnesota. The Minnesota Department of Public Safety – Bureau of Criminal Apprehension is contacted to confirm data they provide in an annual report of data on the state’s Permit to Carry (an exempt carry permit). Local police departments and county sheriffs are asked for data they collect on Permits to Purchase/Transfer (in the FIST category of purchase permit). Not all local police departments issue permits or conduct background checks for the permits; rather, some local police departments contract with the county sheriff or another police department for their residents to obtain permits with the contracted agency. This varies by police department and county. Counts of the populations served must then be adjusted to reflect this situation. Finally, there is the option for a Minnesota resident who lacks a permit to obtain a firearm through a one-time transfer from a dealer, after a background check by a local agency.

Nevada

In Nevada, both a state agency contact and local agencies are surveyed for FIST. The Nevada Department of Public Safety (DPS) is the POC for all transfer checks that occur when a purchaser attempts to obtain a firearm from an FFL. DPS provides data on the number of applications for Point of Sale transfers of firearms from FFLs throughout the state. Generally, FIST obtains data on Carry Concealed Weapons Permit (an exempt carry permit) applications from county sheriffs, except in the cases of Carson City (considered an independent city) and the Las Vegas Metropolitan Police Department, which reports for Clark County.

New York

New York requires a license to possess, carry, or acquire a handgun and certain other types of firearms. The state’s Pistol/Revolver License application is a standard form used by all local agencies. Generally, cities and counties in New York State have considerable discretion in processing Pistol/Revolver Licenses. This license is categorized for FIST as a purchase permit. In addition to the license, the state also requires a prospective handgun purchaser to obtain a “license amendment” for any additional firearm purchase other than the one initially granted with a new license. Each amendment for a firearm purchase (also called a “purchase coupon” by some local agencies) requires an additional background check. FIST attempts to capture the data from these amendments as well.

A license is authorized by a city or county licensing officer who has discretion to specify conditions for possessing or carrying the handgun described in the license. The law most relevant to FIST is that a valid license must be presented to a dealer in order to receive a handgun. Licensing functions include the granting of a license, maintaining records of applications, denials and approved licenses, and conducting background checks. Each jurisdiction divides its function in a different way. Generally, the county sheriff (or in some counties, the sheriff in addition to several municipal police departments) conducts the background check for applications for new licenses and purchase coupons. Applications for licenses/amendments are processed by county clerks in many counties and are, more often than not, the records keepers for licenses. The decision on whether or not to deny an applicant an amendment or license sometimes resides with a local judge (or judges, in larger counties). Thus, there are three separate entities potentially involved in the process of obtaining a license or amendment. FIST typically reaches out to county clerks for data on applications and denials of licenses. In other instances, the POC for FIST is the county sheriff.

The terminology used to describe the New York State purchase permit also varies among counties, with terms including “handgun license,” “pistol permit,” and “concealed carry license.” The reason the license may be referred to as a “concealed carry permit” is likely due to language contained within the license application form, whereby the applicant is prompted to select from three types of license designations: 1) carry concealed, 2) possess on premises, 3) possess/carry during employment.

Finally, state law governs most counties in the same manner with the exception of New York City, Westchester County and Suffolk County. The New York City approval process is more stringent than the rest of the state, and is carried out entirely by the New York City Police Department (NYPD). Data from the NYPD is obtained via a Freedom of Information Law (FOIL) request. Data on pistol licenses is obtained through the Westchester County Police department, the Suffolk Sheriff, and the Suffolk County Police Department.

Washington

Data on applications and denials of handgun transfers for Washington are collected from county sheriffs and municipal police departments. The local agencies conduct point of transfer checks after receiving applications from licensed dealers. As with other situations in which both municipal and county level agencies are surveyed, there are instances when smaller agencies contract with larger ones or several agencies use a central location to conduct checking activity. These arrangements vary by county and the populations served are controlled to reflect these instances.

BJS and REJIS have found, during the course of data collection from many local agencies in this state, that there is considerable confusion about tracking data for applications for handgun transfers (though information on denials is often retained). The confusion seemingly stems from differing interpretations by agencies of federal and state rules on retaining approved applications. Some agencies are unable to provide a yearly application count because they do not retain the approved applications. REJIS is attempting to remedy the situation by speaking directly to the agencies during data collection about their ability to retain statistical summary data, regardless of whether applications are saved. REJIS recently engaged a researcher at the Washington Statistical Analysis Center, who works directly with many local agencies, and encouraged him to find ways to help agencies report complete data.

Use of periodic data collection

Not applicable. BJS proposes to collect FIST data annually.

3. Methods to Maximize Response Rates

BJS and REJIS have applied their extensive experience working with the FIST data collection and understanding of the complexities of the firearm background check process to identify new approaches to maximize and improve response rates for the recent past FIST collections.

As described previously, REJIS has identified key state-specific issues that are unique to the FIST collection, and has taken steps to address them including tailoring survey notification letters to reference state-specific language that agencies are familiar with to reduce confusion and reduce respondent burden. REJIS will continue to adhere an aggressive contact schedule to follow up with nonrespondents. BJS will leverage its relationships with state agencies that receive BJS NCHIP and/or NARIP to conduct outreach and follow up activities as needed.

Nonresponse bias analyses

The overall 2015 FIST response rate is about 80%. All (100%) of state agency reporters and about 80% of local agencies provided data. These rates are consistent with the 2014 response rates: 80% overall; 100% for state agency reporters; and 80% for local agencies. The 2015 response rate was below 80% in seven states: GA (78%); MN (77%); NC (77%); NV (78%); NY (60%); WA (75%); and WV (75%).

In 2014, the response rate was below 80% in five states: GA (76%); ID (73%); MT (75%); NY (69%); and WV (78%).

For the 2014 FIST collection, BJS and REJIS conducted a nonresponse bias analysis for any state where the response fell below 80%.⁵ While the nonresponse bias analyses have yielded useful information, the nature of the FIST collection and the structure of the sampling plan, specifically the small number of local agencies that comprise the within-state strata, make it challenging to parse specific reasons why response rates fall below 80%. Although some states do not achieve an overall 80% response rate, the level of response is still relatively high and the non-response adjustments are small, which means it is difficult to detect specific differences between the characteristics of responders versus non-responders. Often times, a very small number of local agencies can have a sizable impact on the overall state response rate. For example, for the 2014 collection, only fifty-six agencies were surveyed in MT, so the decrease in response rate was attributed to two less agencies that provided a response. Similarly, in WV, there were fifty-five agencies surveyed, so one less agency had a sizable (2%) impact on the response rate. Further, BJS has not identified a comparable data source to serve as a comparison to FIST data, which also challenges the analyses.

Despite these limitations, BJS will continue conducting nonresponse bias analyses to maximize response rates. Beginning with the 2016 collection, BJS will look at nonresponse for all cases at the state level, rather than just at states where the response rate falls below 80%, to determine if and how the results can be used to inform enhancements to the FIST methodology.

BJS has found that addressing the types of state-specific issues described previously has most positively improved response rates. While BJS will continue to assess how the FIST methodology can be enhanced to maximize response rates and decrease burden, BJS has also prioritized the need to address more policy-related issues at the state and local levels. For example, BJS has identified the need to work through the NYPD approval process to obtain FIST data and to find ways to communicate to state and local agencies in WA that data collection activities are legally permissible. BJS is in a position to address some of these policy issues through its administration of the NCHIP and NARIP grant programs, possibly by providing funding for initiatives to support FIST-related tracking and reporting activities or by leveraging its relationships with state agencies that receive NCHIP and/or NARIP funds to identify new ways to initiate, improve, or expand FIST data collection activities.

4. Final Testing of Procedures

None planned.

5. Contact for Statistical Aspects and Data Collection

For information on the FIST survey, statistical methodology, and/or uses of FIST data, contact:

Allina Lee, Statistical Policy Advisor

Bureau of Justice Statistics

810 7^th Street, N.W.

Washington, D.C. 20531

Phone: 202-307-0765

Email: [email protected]

Connor Brooks, Statistician

Bureau of Justice Statistics

810 7^th Street, N.W.

Washington, D.C. 20531

Phone: 202-307-0765

Email: Connor. [email protected]

Attachments:

• Attachment I. BJS authority

• Attachment II. Proposed 2016 FIST survey

• Attachment III. Screenshots of proposed 2016 FIST web-form

• Attachment IV. Approved non-substantive change request to OMB

• Attachment V. Proposed 2016 FIST project schedule

• Attachment VI. Proposed 2016 FIST correspondence

• Attachment VII. FIST reporting structure, 2016

• Attachment VIII. Proposed 2016 FIST sample determination and allocation

• Attachment IX. Base weights for NSR states, 2015 FIST

• Attachment X. Standard errors, 2014 FIST

• Attachment XI. FIST terminology for firearm checks and permits, by jurisdiction

1

2

3 Only agencies in the small agency strata (categories 1 and 2) will be given weights. All agencies in the large agency stratum (category 4) will be surveyed due to 1) the agency’s population served size, and 2) the small number of agencies within these categories, which would make sampling impractical. The large agency stratum will be self-representing, so it will have a base weight of 1.

4 BJS and REJIS are currently analyzing the 2015 data and will publish standards errors in the Background Checks for Firearm Transfers, 2015 – Statistical Tables.

5 BJS and REJIS are currently assessing nonresponse for the 2015 FIST collection.

9