The universe for the LFIOS is the Nation’s estimated 400 State and local law enforcement agencies that house and maintain an AFIS used for latent fingerprint searching in the course of criminal investigations. To get a sense of the frame, an analogy that could be employed is that each State and local LEA with an AFIS represents a “household” and the sampling frame is all the households. Local LEAs include city, municipal, or county agencies. Since this survey seeks specific information regarding AFIS, the “households” consist of the component within each LEA that is responsible for managing the AFIS – such as an Identification Bureau, Identification Services Division, or Criminal Investigation Division – rather than the entire LEA. The estimate of 400 agencies was suggested by participants on the Latent Print AFIS Interoperability Task Force during the development of the questionnaire based on the number of local LEAs that have an AFIS within highly populous states compared to more sparsely populated states that may have few to no local LEAs with an AFIS.
In the “household” analogy, a “head of household” is identified at each State and local LEA to be the primary point of contact (POC) for the survey. This person would generally be regarded as a resident AFIS expert at their respective agencies who can provide particular information on latent fingerprint processing (e.g., enrollment, searching, etc.) on the AFIS and related information that may critically impact interoperability and data interchange. It is also anticipated that more than one person may be required to complete all the survey questions (e.g., AFIS or IT managers, latent examiners, or latent unit mangers may be better suited to know the information of some questions over others). The POC will be responsible for coordinating any appropriate staff to compile the requested data.
The estimated number of agencies permits collection from the entire universe rather than using systematic sampling, and the only stratification that is assumed is to divide the respondent population into the categories of State or local. Furthermore, sampling from this universe does not make sense given the size of the universe and the goals of the project to understand interoperability. For any numerical data that can be analyzed to determine aggregate information, the sample size would converge to the universe to arrive at acceptable degrees of precision given the estimated number of respondent agencies.
It should be noted that this is not a survey of crime laboratories. Furthermore, it should also be noted that this survey is not a questionnaire of latent examiners nor is it a questionnaire directed at any and all agencies that might employ a latent examiner. Care should be taken to distinguish the difference between agency-level AFIS usage and the latent examination process. We do not ask any questions about how latents are marked up and analyzed or the decision-making factors that go into latent analysis. That would be a different survey composed of person-level questions and posed to a related but much larger respondent population.
Constructing the Frame
Utilizing a complete and accurate frame is essential to the success of the survey and obtaining an accurate count for the size of the frame is an initial goal of the survey process. To build the frame, it is necessary to determine the State and local LEAs that house and maintain an AFIS for latent fingerprint searching in the course of criminal investigations. Developing this list of agencies and the corresponding POCs requires direct outreach via telephone and email done in two phases. Phase I focuses on State LEAs and Phase II focuses on local LEAs. Phase I involves building out the State frame by contacting each State LEA to confirm that they have an AFIS, determine which operational component within each LEA is responsible for managing the AFIS, and identify the POC for the survey. This outreach will generally progress in an iterative fashion until the correct person is identified. From the State-level POCs, local information will be solicited. Phase II involves following up both on the local information provided by the States and collected through other research to build out the local frame in a similar manner to the State frame.
Since it anticipated that all States operate an AFIS for criminal justice purposes with few or no exceptions to this expectation, Phase I of frame collection has not been viewed as a resource-intensive exercise. A complete State-level frame has already been compiled and an associated a set of POCs at 48 LEAs (47 States and the District of Columbia) has been determined through email and phone correspondence as outlined above but described in more detail here. A partial initial list of State identification bureaus and their respective POCs was developed through the Latent Fingerprint AFIS Interoperability Task Force with the remaining information available through publicly available resources like State government websites. A standardized email that gave a brief overview of the survey and the AFIS-related personnel for which it was intended was sent to each initial POC. It was at the discretion of the person receiving the email to determine if they were in fact the correct person or if they needed to recommend another component within the State LEA or another POC within their component.
Responses to the standardized email fell into two categories: (1) either the LEA component and person contacted was correct to act as the POC or (2) the respondent recommended a different LEA component and/or POC, in which case that person was then sent the standardized email. In most cases, the initial contact person at the LEA was at a minimum the correct individual to direct the email inquiry to the correct POC within the agency, indicating that the direct outreach approach employed here was highly successful. In the few instances where there was no response to the email from the initial contact person, further research telephone calls were conducted to find an appropriate initial contact to send the standardized email.
Phase II remains to be completed, although a limited number of local LEAs and POCs have been identified through the State frame and it is anticipated that the remaining local frame will be collected in a timely fashion. At this time, the State-level survey could be issued based on the State frame collected. It is anticipated that Phase II can be implemented to quickly build out the local frame with POC information in large part through communication with the State respondents, other individuals within from the criminal justice community, and other sources of criminal justice information (See attached “Appendix: Constructing the Frame”). Also, at this time, the survey only targets State and local LEAs and not Federal or tribal LEAs, although the two latter types of agencies could be included in future versions of the survey.
All respondents will be contacted by email with a link to the online LFIOS form, however the print version will be available on request. All State and local law enforcement agencies that house and maintain an AFIS used for latent fingerprint searching in the course of criminal investigations will be contacted. Once respondents reply to the canvas, they will be issued a login and password for instant access to complete the questionnaire in a secure online system. Once respondents reply to the canvas, they will be issued a login and password for instant access to complete the questionnaire in a secure online system. Where agencies need more than one person to complete the survey, answers can be saved and the survey returned to a later time by the same or a different person.
Nonrespondents to the canvas will be contact directly by follow-up email and/or telephone to encourage them to reply to the online or print questionnaire. Since this is a new data collection effort that will be unfamiliar to State and local practitioners, NIJ requests that OMB permit at least one year to complete the data collection as there may be a significant need to reach out repeatedly to nonrespondent agencies.
To maximize response rates, NIJ will employ the best practices for survey nonresponse follow-up. All State and local law enforcement agencies that house and maintain an AFIS used for latent fingerprint searching in the course of criminal investigations will be contacted. An initial invitation letter from the NIJ Director on official letterhead will be sent to prospective respondents via email from NIJ on behalf of the NIJ Director to engage the respondent population. Follow-up email and telephone calls to nonrespondents will be used to encourage high response rates. Once respondents reply to the canvas, they will be issued a login and password for instant access to complete the questionnaire in a secure online system.
For the online data collection system, the LFIOS is relatively simple in format with questions in black text behind a light gray background. Most questions are Yes/No or multiple choice that can be answered by clicking on radio buttons. Where text information is requested, information can be typed into a box and the questions have been designed to request the least amount of necessary information. The data collection forms submitted for this collection involve data that are available from the current law enforcement agencies, and iterative input from practitioners and subject matter experts regarding the survey content, wording, and length so that the questionnaire is best targeted to the individuals in the law enforcement agency best suited to provide the needed information such as AFIS managers and latent print examiners.
Where agencies need more than one person to complete the survey, answers can be saved and the survey returned to a later time by the same or a different person. This approach is anticipated to translate into higher response rates by allowing the respondent time to gather the necessary information and not feel overwhelmed by having to answer the entire survey at once and minimizes the burden on any one person. Instructions for the survey will stress that more than one individual may be needed to accurately answer all the questions on the survey to make this point clear. Respondents will also be given the choice to answer some questions in a manner that they may not know the answer right now but will find out and respond.
Through pretesting as discussed in Section B.4., issues regarding coordinating multiple personnel within a respondent agency to provide all the requested data did not rise to the level of concern originally anticipated. However, this could still play a role in some agencies, so some of the revisions incorporated into the survey were designed to mitigate some of the issues related to intra-agency coordination:
All the questions are available for preview on the secure survey website before the respondents complete the survey to help respondents quickly collect all the relevant data and information at once rather than on a question-by-question basis, which also reduces burden. All questions can be viewed in a read-only mode on one page after the secure login and prior to initiating the survey.
The survey is divided into different topical subsections similar to the Survey of State Criminal History information Systems conducted by the Bureau of Justice Statistics. This modular structure may facilitate handing off different sections of the questionnaire to staff at a responding LEA.
On the instructional page at the beginning of the survey, a sentence was also added as a disclaimer: “More than one staff member at each LEA may be necessary to complete the survey, but LEA points of contact will still be responsible for survey submission in its entirety.”
The LFIOS survey was developed from approximately February to November 2011 following Federal regulations and guidance published by OMB for clearance according to the Paperwork Reduction Act. It was developed iteratively using stakeholder input in coordination with the Latent Print AFIS Interoperability Task Force. An advanced draft version of the data collection instrument was publicly presented during the 60-day public comment period at the International Association for Investigation (IAI) Conference in August 2011 in Milwaukee, WI. Stakeholders from the proposed respondent population present at the IAI Conference were able to go through the questionnaire and provide feedback. This was a valuable exercise towards developing a consensus understanding of the data collection instrument at the time. Through stakeholders who provided feedback during the development of the original LFIOS questionnaire, NIJ found that the questionnaires covered important relevant topics and attempted to minimize burden on the respondents to the maximum practical extent possible. Common and insightful recommendations were incorporated into the final version submitted to DOJ and OMB for clearance in the beginning of 2012.
After initial review by OMB, pretesting was suggested to verify that the survey meets the goals and objectives for which it was designed since the survey was new and unfielded. Pre-testing with a number of State and local law enforcement agencies would help uncover any issues regarding comprehension, usability, recordkeeping, and coordination among respondent organization personnel. Up to nine responding agencies could participate in pretesting without prior clearance from OMB according to PRA, but the actual number of pretests may be more or less depending on the initial results. It was recommended that pretesting should proceed until saturation is reached—that is, the point at which no new substantive issues are uncovered. The results of pretesting should demonstrate that saturation has been reached to the extent possible for an unfielded establishment survey of this nature.
Pretesting following the following protocol:
Outreach to identify participants
Distribution of survey materials, this Supporting Statement, and a description of the pretesting for review by participants
In-person meeting or teleconference with participants to discuss survey and capture feedback
Distribution of revised survey to participants for review and comment
Direct outreach to agencies based on known contacts who have demonstrated willingness to participate in research, evaluation, or strategic planning efforts was considered the most viable approach to select candidates for pretesting. Having agencies of various sizes participate was considered an important factor to balance any differences with respect to practices or resources that may exist. Four agencies—two State and two local—from the target respondent population were considered a reasonable and manageable number to pretest the survey first. The results would be assessed to determine whether new substantive issues would likely to be uncovered by further pre-testing with more agencies beyond four. Also, having two State and two local agencies in the eastern and western parts of the United States participate was also considered important to balance any geographic bias that might exist. Should more pretesting beyond four be needed, the pretesting group could be expanded to nine, balancing the size and geographic diversity of the State and local respondent agencies. Any further pretesting beyond nine agencies would be done in consultation with OMB first.
Four agencies agreed to participate:
Local Agencies:
A bloc of local agencies in the National Capital Region that already have a workable network that is interoperable, which includes:
Fairfax County Police Department in Virginia and the Northern Virginia Regional Identification System (NOVARIS)
Prince William County Police Department in Virginia
Arlington County Police Department in VA
Montgomery County Police Department in Maryland
King County Sheriff’s Office in Washington
State Agencies:
Florida Department of Law Enforcement (FDLE)
Wyoming Division of Criminal Investigation
Relevant individuals at each agency were contacted by email and which explained the LFIOS survey effort, the reason for pretesting, and whether the agency would be willing to participate. In each instance, the agency agreed. The survey materials, this Supporting Statement, and a description of the pretesting were sent to participants at least one week prior to meeting either in person or via teleconference. Several critical aspects of the survey were highlighted for pre-testers to consider as they reviewed the questionnaires, in particular:
Comprehension
Usability
Recordkeeping
Coordination among respondent agency personnel
Content omissions and/or redundancies
Comprehension includes whether the questions make sense, are worded properly, and are presented in the best possible order. Usability includes whether the answers provided are adequate, reasonable, or appropriate choices for the questions asked. Recordkeeping includes whether the respondents are able to provide the requested data based on the records they maintain. Since the survey requests agency or “enterprise” level information, coordination among respondent agency personnel includes how well the agency personnel will be able to coordinate efforts to complete the survey if more than one person is necessary to complete the questions. Content omissions and/or redundancies includes whether there are any questions that are redundant or whether there is any critical information the respondents feel would be necessary and useful but is not requested.
Below is a summary of the meetings all held from early May 2012 to early June 2012:
National Capital Region local agencies
May 10, 2012
10:30 AM – 1:30 PM EDT
Duration of meeting: 3 hours
Type of meeting: In-person meeting held at NOVARIS
Florida Department of Law Enforcement
June 1, 2012
10:00 AM – 12:00 PM EDT
Duration of meeting: 2 hours
Type of meeting: Teleconference
King County Sheriff’s Office
June 1, 2012
1:30 PM – 3:30 PM EDT
Duration of meeting: 2 hours
Type of meeting: Teleconference
Wyoming Division of Criminal Investigation
June 1, 2012
1:00 PM – 3:00 PM EDT
Duration of meeting: 2 hours
Type of meeting: Teleconference
The first pretest meeting was an in-person meeting with National Capital Region local agencies. One of the main takeaways gleaned from this meeting was the survey would have great value to the community and that most of the concepts relevant to AFIS interoperability were contained in the survey. The participants from several local agencies were used to working together and provided an opportunity to discuss the questions in great detail from different perspectives. Discussion of the concepts underlying the survey also helped inform ways to improve wording and structure as well as include questions on topics important to local agencies such as Federal funding used for AFIS acquisition and maintenance.
Given the extremely thorough feedback from the first meeting, some degree of editorial revision would be necessary before the questionnaire would be suitable for wide release.
The approach taken with the remaining three teleconferences was to discuss the survey from beginning to end, but to anchor the discussion more on the concepts underlying the questions rather than on the particular wording or order of the questions. This was found to be an extremely effective use of time since there were already strong indications that editorial revisions were going to be needed anyway. This permitted deeper discussion on the various topics relevant to AFIS interoperability with respect to latent fingerprints.
All participants were asked to submit any further comments after the meeting portion on a purely voluntary basis should anything additional come to mind. After the meetings were completed, the comments and feedback from all participants were reconciled and necessary and appropriate revisions were made from early June to mid-July 2012. The revised survey questionnaires are discussed in Section A.2. The revised questionnaires were passed back to the participating agencies on July 20, 2012 for review by August 6, 2012 to verify that the changes addressed any issues or concerns that arose during the pretesting. Responses from pretesting agencies that provided feedback on the revised questionnaires indicated that no further major revisions were necessary. Only minor typographical errors or question-level points for clarification were noted for editing by those who provided responses.
The Office of Science and Technology of NIJ takes responsibility for the overall design and management of the activities described in this submission, including data collection procedures, development of the questionnaires, and analysis of the data.
NIJ contacts include:
Lead agency: National Institute of Justice
Office of Justice Programs
U.S. Department of Justice
Mark Greene, Ph.D.
Physical Scientist
Information and Sensors Technologies Division
Office of Science and Technology
National Institute of Justice
810 Seventh Street NW
Washington, DC 20531
(202) 307-3384
ManTech contacts include:
Lars Ericson, Ph.D.
Director of Advanced Technologies
1000 Technology Drive
Suite 2310
Fairmont, WV 26554
(304) 368-4216
Phill Wiggin
Deputy Project Manager
1000 Technology Drive
Suite 2310
Fairmont, WV 26554
(304) 368-4213
Mark Persinger
Principal Research Analyst
301 Summers Street
Suite 4
Hinton, WV, 25951
(304) 254-2334
NIJ Authorizing Legislation, Omnibus Crime Control and Safe Street Act of 1968, as amended, 42 U.S.C. 3722
OST Authorizing Legislation, Homeland Security Act of 2002, 6 U.S.C. 162
Data Collection Instrument (LFIOS-C)
Data Collection Instrument (LFIOS-S)
Data Collection Instrument (LFIOS-L)
Invitation Letter from NIJ Director to Respondents
Appendix: Constructing the Frame
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Mark Greene |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |