Nonsubstantive Change Memo to OMB

U.S. Department of Justice
Office of Justice Programs
Bureau of Justice Statistics

Washington, D.C. 20531
MEMORANDUM
TO:

Shelly Martinez
Desk Officer
Office of Statistical Science and Policy, Office of Management and Budget

THROUGH:

Lynn Murray
Clearance Officer, Justice Management Division
William J. Sabol
Acting Director, Bureau of Justice Statistics
Devon B. Adams
Chief, Criminal Justice Data Improvement Program, Bureau of Justice Statistics

FROM:

Allina Lee
Justice Statistics Policy Analyst, Bureau of Justice Statistics

SUBJECT:

Non-substantive Changes to OMB #1121-0314 to Firearm Inquiry Statistics Program

DATE:

May 23, 2013

The purpose of this memorandum is to advise the Office of Management and Budget (OMB) of changes to the
Firearm Inquiry Statistics (FIST) Program data collection schedule and provide copies of the revised sampling
plan and the web-form implementation plan to respond to the conditions of the OMB clearance. The Bureau of
Justice Statistics (BJS) is also requesting OMB approval of changes made to the approved survey form and to
the number of respondents in the collection. Briefly, the FIST program collects data on firearm background
check activities from a sample of local agencies authorized to conduct background checks and combines these
data with data obtained from state reporting agencies, the Federal Bureau of Investigation (FBI), and the Bureau
of Alcohol, Tobacco, Firearms, and Explosives (ATF) to produce a national estimate of firearm applications,
denials, reasons for denials, arrests, and appeals. Since 1995, BJS has conducted the FIST survey. The Regional
Justice Information Service (REJIS) was competitively awarded a cooperative agreement in FY 2011 to
collaborate with BJS and continue the FIST data collection efforts.
While the FBI reports on transaction data processed by the National Instant Criminal Background Check System
(NICS), the FIST collection is the only comprehensive source of national data on the number of applications
received and denied and post-denial activities pursuant to the Brady Handgun Violence Prevention Act of 1993
(the Brady Act) and similar state laws. Absent this collection, there would be no other means to assess the
enforcement of the Brady Act, including the effectiveness of post-denial activities, levels of background check
trends and activities nationwide, and appeal and arrest activities at the state and local levels. The FIST data

collection also contributes to the goals of the NICS by enhancing efforts to collect and analyze statistics on
denial for firearm purchase applications and to identify changing trends in disqualifying factors over time.
OMB clearance for the FIST collection, ICR reference #201010-1121-002, was approved on February 20, 2011.
Data Collection Schedule
The OMB clearance granted BJS approval to collect calendar year (CY) 2010 data using the same sample used
in previous years with the understanding that the subsequent data collection would be conducted with a revised
sampling frame. The FY 2011 FIST award to collect CY 2010 data was competitively awarded to REJIS in
August 2011. The time associated with competing the award and making a funding decision, coupled with the
delayed passage of the FY 2011 budget, resulted in significant delays in initiating the collection. Additionally,
BJS spent a significant amount of time reviewing the FIST methodology to both improve the reporting of 2010
data and to determine the sampling plan for future collections. The Background Checks for Firearm Transfers
2010 Statistical Tables publication was published to the BJS website in February 2013.
Due to the delayed start of the 2010 data collection and time spent on addressing methodological issues, BJS
was concerned that collecting CY 2011 data would create an unnecessary burden on respondent agencies by
asking them to report data so soon after their last submission. Moreover, BJS has prioritized collecting CY 2012
data because it has been a notable year in firearm background check activity, as evidenced by the increase in
transaction data reported by the FBI (up from 2010 and 2011), and a high demand for the most recent data on
background check activities observed since the school shooting in Newtown, Connecticut, in December 2012.
For these reasons, BJS decided to forgo the collection of CY 2011 data and will collect the next round of data
for CY 2012. Pending OMB approval of the changes outlined in this memo, the 2012 data collection is
scheduled to be initiated in June 2013 (see Attachment I).
Revised Sampling Frame
The sample of local checking agencies will be redrawn for the 2012 FIST collection. REJIS spent a significant
portion of time updating the FIST universe to ensure that all agencies that are authorized to conduct and are
known to conduct background checks are accounted for in the universe (see Attachment II). BJS and REJIS
worked extensively to develop the most appropriate sampling plan that will produce a reliable national estimate
while staying within the confines of the budget allocated for the FIST data collection. The new sampling plan
will also enable the creation of state-level estimates, which have become a source of increased public interest.
After considering multiple methodologies, BJS has decided to employ an enumeration of local agencies in eight
of the twelve states (DC, DE, IA, ID, MT, NC, NV, and NY) in which local checking agencies are responsible
for conducting background checks. Agencies in the remaining four states with local checking agencies (GA,
MN, NE, and WA) responsible for conducting background checks will be sampled due to the relatively greater
number of agencies in these states compared to the other eight. In these four states, the decision was to create a
stratified sample based on population size that roughly equates to Category 1) rural – places of less than 10,000
population; Category 2) small cities – places of between 10,000 and 99,999 population; Category 3) small
metropolitan areas – places of between 100,000 and 199,999 population; and Category 4) large metropolitan
areas – places of 200,000 or more population. These population categories were selected to be consistent with
definitions of various census place levels. Sampling will be done for agencies in Categories 1 and 2, while the
agencies in Categories 3 and 4 will be kept in the sample as certainty (see Attachment III).
BJS will plan for unknown nonresponse by creating two reserve samples within each of the four sampled states
with a 15% oversample for each. Agencies will be randomly assigned to one of the two reserve samples so that
the probability of selection will equal the initial probability of selection times the probability of being assigned
to a reserve sample. If a response rate within any state falls below 85%, the first reserve sample will be initiated.
If the response rate falls below 75% after the first reserve sample is instituted, the second reserve sample will be
employed. Appropriate sampling weights will be assigned such that state-level estimates may be made in
addition to a national estimate. BJS has initiated and will continue to employ a rigorous follow up schedule
based on Dillman, Smyth, and Christian’s work on mail and internet surveysi to maximize the response rate (see
Attachment IV).

A sample size of 791 is needed to implement this sampling design with the appropriate precision to calculate a
reliable national estimate and improve the overall robustness of the sample. The sample size also stays within
the budgetary parameters approved for the FIST data collection. BJS is requesting OMB approval to increase
the number of respondents from the currently approved 653 to 791. The impact of the increased number of
respondents on the overall burden hours is addressed in the following section.
Revised Survey
As part of BJS’ overall effort to improve the FIST methodology, BJS assessed the approved tally sheet that had
been used previously to collect FIST data and decided that changes were necessary to better clarify what data
are being requested and format the presentation to be more aligned with other BJS surveys. Copies of the
approved tally sheet and proposed revised survey are appended to this memorandum for OMB’s review
(Attachments V and VI). The following sections describe the changes that were made to the survey form and
underlying rationale for the changes.
Survey format
The revised survey is designed to collect the same data on firearm background check activity for the reference
year as the approved tally sheet. The revised survey includes screener questions to assist in maintaining the
universe file of agencies that conduct firearm background checks, a section to obtain information about agency
characteristics to better understand the sample population and assist efforts to conduct a nonresponse bias
analysis if necessary, and a section to identify how agencies collect and report their data. The majority of these
additions are in the form of a check box response, which will result in a minimal enhancement to the burden.
One of the greatest benefits of the revised survey is that additional clarification and directions are provided to
ensure that respondents are clear as to what data are being requested. BJS found that the lack of clear
instructions and definitions in the tally sheet was problematic because it introduced potential data quality issues
as it may have been unclear to respondents what information was being requested. Given the fact that the sample
will be redrawn to include new agencies that have not participated in past FIST data collections, it is imperative
that respondent agencies are provided with consistent guidance to improve the quality of data being collected
and inform BJS about how state and local agencies collect and report data.
Instructions and definitions were added to each section in addition to screener questions to determine if the
respondent agency tracked the requested data. A box was also added to applicable questions so the respondent
could check if the response was an estimate to help BJS to better understand the accuracy of the data.
The following enhancements were also made to the survey instrument:
 Page 1: Questions have been added to determine if the respondent agency is currently authorized to
and/or has conducted background checks during the reference year. These questions are designed to
help maintain the FIST universe of local checking agencies.
 Section I: A section titled “Agency Characteristics” has been added to request information about the
types of checks that the local agency conducted during the reference year. In the past, REJIS sent the
tally sheet to respondent agencies based on what types of checks they were known to conduct based on
state laws. These questions will help to verify the accuracy of and maintain the universe of local
checking agencies and help to monitor changes in how agencies conduct background checks.
 Section II: A question (#7) was added to ask if the agency tracked the reason(s) why an application was
denied. This question will help to determine which agencies are currently collecting this data. Two
questions (#9 and #10) were added to determine how the agency records and reports on reasons for
denial. This information is critical to inform BJS’s understanding of the tracking process and ensure that
the data can be accurately reported.
 Section III: No new questions have been added apart from the screener questions.
 Section IV: A section has been added to address the reporting of partial year counts. Prior experience
with the FIST survey has shown this to be a rare occurrence.
 Section VI: A section has been added to enable the respondent to provide a narrative response about its
data collection procedures and/or reporting of estimated totals

Feedback from testing
In accordance with approved OMB survey testing protocol, the survey was sent to five agencies with a request
to complete the revised survey and provide feedback. Of the five agencies, four were local agencies and one was
a state agency. The local agencies represented the different population strata (size A, B, and C) of those in the
sample. Three local agencies responded: one size A and two size B. Additionally, the revised survey was
presented to two subject matter experts who have significant experience in firearm background check activities
and procedures for comment. See Attachment VII for a list of pre-test questions asked and feedback received.
Burden
The respondent burden to complete the tally sheet approved in the original OMB package was estimated to be
fifteen minutes. Based on comments provided by the respondents and stakeholders that completed and/or
reviewed the revised survey and a corresponding analysis of estimated burden by number of permit types
checked (see Attachment VIII), BJS estimates that the respondent burden associated with completing the revised
survey will be approximately twenty-five minutes. As the new survey was developed and the burden associated
with responding to the questions was assessed, BJS concluded that the original estimated burden of fifteen
minutes to complete the tally sheet was too low and that the twenty-five minute estimate is a more accurate
reflection of the true burden associated with the FIST survey instrument to provide adequate time for the
respondent to read the instructions and accompanying definitions. Based on the results of a field test of the
revised survey instrument combined with REJIS’s fifteen years of experience administering the FIST data
collection, BJS estimates that the respondent burden will vary depending on the number of permit types the
respondent agency conducts background checks for.
The estimated respondent burden is as follows:
 20 minutes for agencies that conduct background checks for one permit type
 30 minutes for agencies that conduct background checks for two permit types
 30 minutes for state reporting agencies
As stated previously, the type of data that is being requested has not changed from the original tally sheet.
BJS does not anticipate that the twenty-five minute burden will negatively impact the response rate as the data
being requested are straightforward and are believed to be routinely tracked by most respondent agencies. While
respondents will be encouraged to utilize the new web-form reporting options, REJIS will also continue to
accept data via fax and paper survey to ensure that multiple response modes are available to minimize the
respondent burden. The FIST data collection received OMB approval in 2007 to administer and collect survey
data twice a year for an annual estimated burden of thirty minutes per respondent. BJS has not observed that the
change in burden has had any impact on the response rate over the years. Rather, the type and frequency of the
contact schedule has proven to be a key factor in the response rate. As noted, REJIS will employ a rigorous
contact schedule to maximize the response rate.
The FIST collection is currently approved for 653 respondents and 163 burden hours. With the increase in the
number of respondents to 791 and the estimated time to complete the survey at twenty-five minutes, the
recalculated number of burden hours totals 330 hours.
Web-Based Form and Implementation Plan
To satisfy the terms of the OMB clearance, BJS is submitting to OMB for review several screenshots of the
web-based FIST survey that was developed and will be employed in the 2012 FIST data collection (Attachment
IX). The web-based form was designed with input from survey methodologists, subject matter experts, and
background checking agency stakeholders. Following OMB testing protocol, four agencies were asked to
complete the web-form and provide feedback; despite multiple follow up requests, none of the agencies
responded to the request. Given that the web-form was designed to closely mirror the format of the paper
survey, BJS estimates that the respondent burden will be the same (twenty-five minutes).

The introduction of the web-based data collection instrument is intended to encourage participation by providing
a secure, convenient mode of responding to the FIST survey and is intended to streamline the survey process by
eliminating questions that the respondent may not need to see, thus also reducing response burden. It is also
designed to reduce item nonresponse by requiring that a response to critical items is submitted before continuing
to the next question. Each FIST participant will be provided a unique User ID and password that will allow
secure access to the web form. In order to reduce any differences in response as a result of the mode by which a
respondent chooses to participate in the survey (mode effects), particularly mode effects due to the different web
and mail response modes, the web form has been designed to mimic the visual presentation of the paper survey
as much as possible. The wording of the questions is consistent between the two modes. A detailed description
of the web-based survey implementation plan can be found in Attachment X.
Revised correspondences
BJS updated the approved correspondence letters and phone script to include information about the web-based
survey option. Copies of the updated communications can be found in Attachment XI.
Terms/condition of clearance
The approved OMB packet stipulated that the sample would be redrawn for the 2011 FIST data collection. As
addressed previously, BJS will initiate the next collection for CY 2012 data. The terms of the OMB clearance
stated that “OMB looks forward to a new analysis of sample design and size needed to meet precision
requirements after one round of experience with new procedures.” BJS will submit to OMB for review the
requisite analyses after the data collection for CY 2012 employing the new sampling frame is complete. The
current OMB approval for the FIST data collection expires in February 2014. BJS will be prepared to submit the
requisite package in advance of the expiration to avoid a delay to the data collection schedule for 2013 data.
Summary
BJS seeks OMB approval of the above noted changes to the FIST survey, revised sampling plan, and the plan to
implement the web-based survey form in adherence with the terms of the OMB clearance. In light of the fact
that gun control legislation and background checks are politically sensitive, high profile topics and there is
currently a high demand for data, specifically state-level data, BJS’s priority is to get into the field as quickly as
possible to collect and analyze CY 2012 data in order to report on recent data in next publication of the
Background Checks for Firearm Transfers series.
BJS respectfully requests OMB’s consideration of these factors during the review and will work to expeditiously
respond to any inquiries. Thank you in advance for your time and consideration.

Attachments:
Attachment I: Proposed CY 2012 FIST data collection schedule
Attachment II: FIST universe
Attachment III: Proposed sample allocation
Attachment IV: Data collection methodology
Attachment V: FIST tally sheet approved in OMB packet
Attachment VI: Revised FIST survey instrument
Attachment VII: Responses to survey testing
Attachment VIII: Analysis of burden
Attachment IX: Screenshots of web-based FIST survey form
Attachment X: Web-based survey implementation plan
Attachment XI: Revised correspondence

i

Dillman, Don A., Smyth, Jolene D., and Christian, Leah Melani. (2009). Internet, Mail and Mixed-Mode Surveys: The Tailored Design Method. John
Wiley and Sons: New York.