General Clearance Part A

1. JUSTIFICATION

1. Importance of Information

This is a request for a 3-year generic clearance for the FBI Uniform Crime Reporting (UCR) Program to develop, test, and improve its data collection and survey instruments and methodologies. The procedures include, but are not limited to, potential tests of data collection and survey operations, focus groups, cognitive laboratory activities, pilot testing, field testing, exploratory interviews, experiments with questionnaire design, and usability testing of electronic data collection instruments.

The FBI UCR Program is requesting the generic clearance in order to test new methodologies for data collection and survey activities. Specifically, pretests and burden assessments for UCR Programs including the Summary Reporting System (SRS), the National Incident-Based Reporting System (NIBRS), the Law Enforcement Officers Killed and Assaulted (LEOKA) Program, the Hate Crime (HC) data collection, the Cargo Theft (CT) data collection, and the Human Trafficking (HT) data collection are covered in this generic clearance request. Further, this generic clearance request includes any pretesting conducted for new data collections initiated by the UCR Program’s Advisory Policy Board (APB) or collections recommended by an upcoming and joint National Academy of Sciences (NAS) study of Bureau of Justice Statistics and UCR crime data collections. Currently, there is some attention being given to the idea of an animal cruelty data collection in the UCR, which would also fall under the assessments covered by this general clearance request.

The UCR Program has begun to add more statistical rigor to its data collection programs. As OMB requested data collection instrumentation and administrative pretesting, it was quickly apparent that pretesting limitations under the Paperwork Reduction Act (PRA) would not allow the UCR Program to fully test validity and reliability. Specifically, the PRA requirement for using nine or fewer persons as a sample to provide input and feedback on survey design and data collection methodologies is too small to generalize findings to a larger population and accurately test the statistical quality of the collection instrument.

The information collected via these mechanisms, while still helpful, was nonetheless somewhat limited in its ability to detect and diagnose problems with the instruments and the procedures being tested. The generic testing clearance will allow the FBI to take advantage of a variety of methods that are useful for identifying questionnaire/assessment and procedural problems, suggesting solutions, and measuring the relative effectiveness of alternative solutions. Through the use of these techniques, when employed routinely in the testing phase of UCR data collections, questionnaires and assessments can be simplified for respondents, respondent burden can be reduced, procedures for the collection of administrative data can be streamlined, and the quality of the questionnaires and assessments used in continuing and one-time surveys and assessments can be improved. Thus an increase in the quality of the data collected can be achieved as well.

The FBI UCR Program is requesting a three-year generic clearance for pretesting, during which the FBI will provide periodic reports on pretesting activities. The pretesting activities conducted under this generic clearance will be for development work only, not for estimation or publication purposes. These development activities will include investigations of item types, research on the availability and quality of administrative data from state and local justice agencies, state and local burden assessments, small scale tests to test appropriate access and retrieval methods for various types of administrative data, research about mode of administration (telephone, paper and pencil, computer-based, mail-out and mail-in, etc.), methodology of questionnaires and assessments, and testing of items. Activities covered under this generic clearance will not include field testing of a full-scale program implementation protocol.

This clearance package serves as a request for generic clearance. In this document we have provided a description of the scope of possible activities that might be covered under this clearance. The requested clearance is important to the FBI UCR Program’s use of pretesting activities, because of the length of time required to plan the activities. This generic clearance will go through the usual two Federal Register Review periods. Subsequent to these review periods, the FBI UCR Program requests that OMB review then comment on or clear proposed studies in a two-week period with no 30-day Federal Register Notice period required under the Generic clearance. This clearance is similar to the testing clearances held by the Census Bureau, the Bureau of Labor Statistics, the Bureau of Justice Statistics, the National Center for Education Statistics, and the National Center for Science and Engineering Statistics.

The specific methods proposed for coverage by this clearance are described below. Also, procedures are outlined according to the FBI UCR Program’s plans to put in place for keeping OMB informed about the identity of the surveys and the nature of the research activities being conducted.

The methods proposed for use in questionnaire and assessment development are as follows:

Pilot testing. For the purposes of this clearance, pilot tests are defined as data collection efforts conducted among either purposive or statistically representative samples, for which evaluation of the questionnaire and/or procedures is the main objective. The FBI UCR Program will only publish research and development (R&D) and methodological reports on the results of these pilot tests, but will not publish statistical reports or data sets based on the findings. Pilot tests are an essential component of this clearance package because they serve as the vehicle for investigating basic item properties for new or redesigned data collection efforts, such as reliability, validity, and difficulty, as well as feasibility of methods for standardized administration of forms. Under this clearance a variety of surveys will be pretested, and the exact nature of the surveys and the samples is undetermined at present. However, due to the smaller nature of the tests, we expect that some will not involve representative samples. In these cases, samples will basically be convenience samples, which could be limited to specific geographic locations, may involve expired rotation groups of a current survey, are known to have specific aggregate demographic characteristics, etc. The needs of the particular sample will vary based on the content of the survey being tested, but the selection of sample cases will not be completely arbitrary in any instance. Where applicable, pilot testing will include sample administrative data, allowing the FBI UCR Program to test procedures regarding data procurement and comparability of data.

Behavior coding. This method involves applying a standardized coding scheme to the completion of an interview or questionnaire, either by a coder using a tape-recording of the interview or by an in-person observer at the time of the interview. The coding scheme is designed to identify situations that occur during the interview that reflect problems with the questionnaire. For example, if respondents frequently interrupt the interviewer before the question is completed, the question may be too long. If respondents frequently give inadequate answers, this suggests there are some other problems with the question. Quantitative data derived from this type of standardized coding scheme can provide valuable information to identify problem areas in a questionnaire, and can be used as a substitute for or as a complement to the traditional interviewer debriefing.

Interviewer debriefing. This method employs the knowledge of the employees who have the closest contact with the respondents. In conjunction with other methods, we plan to use this method in our field tests to collect information about how interviewers react to the survey instruments, as well as understand problems and pitfalls encountered by interviewers during the interview.

Exploratory interviews. These may be conducted with individuals to understand a topical area and may be used in the very early stages of developing a new survey. It may cover discussions related to administrative records (e.g. what types of records, where, and in what format), subject matter, definitions, etc. Exploratory interviews may also be used to investigate whether sufficient issues are present related to an existing data collection to consider a redesign.

Respondent debriefing questionnaire. In this method, standardized debriefing questionnaires are administered to respondents who have participated in a field test. The debriefing form is administered at the end of the questionnaire being tested, and contains questions that probe to determine how respondents interpret the questions and whether they have problems in completing the survey/questionnaire. This structured approach to debriefing enables quantitative analysis of data from a representative sample of respondents, to learn whether respondents can answer the questions, and whether they interpret them in the manner intended by the questionnaire designers.

Follow-up interviews or re-interviews. This involves re-interviewing or re-assessing a sample of respondents after the completion of a survey or assessment. Responses given in the re-interview are compared with the respondents’ initial responses for consistency between responses. In this way, re-interviews provide data for studies of test–re-test reliability and other measures of data quality. In turn, this information aids in the development of improved, more reliable measures.

Split sample experiments. This involves testing alternative versions of questionnaires, and other collection methods, at least some of which have been designed to address problems identified in draft questionnaires or questionnaires from previous surveys. The use of multiple questionnaires, randomly assigned to permit statistical comparisons, is the critical component here; data collection can include mail, telephone, Internet, or personal visit interviews or group sessions at which self-administered questionnaires are completed. Comparison of revised questionnaires against a control version, preferably, or against each other, facilitates statistical evaluation of the performance of alternative versions of the questionnaire. Split sample tests that incorporate questionnaire design experiments are likely to have a larger maximum sample size than field tests using other methodologies. Larger sample sizes will enable the detection of statistically significant differences and facilitate methodological experiments that can extend questionnaire design knowledge more generally for use in a variety of FBI UCR Program data collection instruments.

Cognitive and usability interviews. This method involves intensive, one-on-one interviews in which the respondent is typically asked to "think aloud" as he or she answers survey questions. A number of different techniques may be involved, including asking respondents to paraphrase questions, probing questions asked to determine how respondents came up with their answers, and so on. The objective is to identify problems of ambiguity or misunderstanding, or other difficulties respondents have answering questions. This is frequently one of the early stages of revising a questionnaire.

Focus groups. This method involves group sessions guided by a moderator, who follows a topical outline containing questions or topics focused on a particular issue, rather than adhering to a standardized questionnaire. Focus groups are useful for surfacing and exploring issues (e.g., confidentiality concerns) which people may feel some hesitation about discussing.

Procedures for Clearance

Before testing activity is undertaken, the FBI will provide OMB with a memo describing the study to be conducted and a copy of instrumentation and debriefing materials that will be used. Depending on the stage of instrumentation development, this may be a printed questionnaire, a set of prototype items showing each item type to be used and the range of topics to be covered by the questionnaire, or an interview script. When split sample experiments are conducted, either in small group sessions or as part of a field test, the different versions of the questionnaires to be used will be provided. For a test of alternative procedures, the description and rationale for the procedures will be submitted. A brief description of the planned field activity will also be provided. The FBI UCR Program requests that OMB raise comments on substantive issues within 10 working days of receipt.

Data collections for these projects are permitted under the authorizing legislation for the tested questionnaires. In most cases, data collection activities conducted by the FBI UCR Program are authorized under enabling legislation:

• Title 28, United States Code (U.S.C.), § 534(a) and (c)–Requires the Attorney General to collect and publish UCR data. Title 28 Code of Federal Regulations (CFR) 0.85 (f), which supplements statutory requirements in 28 U.S.C. § 534–Provides that the Director of the FBI “shall ... operate a central clearinghouse for police statistics under the Uniform Crime Reporting Program…”

• Anti-Arson Act of 1982 - Mandates the addition of Arson as a Part I offense classification to the FBI’s UCR Program.

• Uniform Federal Crime Reporting Act of 1988 in the notes to 28 U.S.C. § 534–Requires federal law enforcement participation in the UCR Program and directs the Attorney General to collect crime statistics, which comprise the national UCR Program, from all federal agencies, including the Department of Defense, “that routinely investigate complaints of criminal activity.”

• Hate Crime Statistics Act of 1990, in the notes to 28 U.S.C. § 534, as amended–Requires the Attorney General to establish guidelines and collect data “about crimes that manifest evidence of prejudice based on race, religion, disability, sexual orientation, or ethnicity, including (where appropriate) the crimes of murder and non-negligent manslaughter; forcible rape; aggravated assault, simple assault, intimidation; arson; and destruction, damage or vandalism of property.”

• Conference Report on H.R. 3355 Violent Crime Control and Law Enforcement Act of 1993, Section 15006 Gang Coordination and Information (b) Data Collection.–Directs the FBI to collect information on incidents of gang violence for inclusion in its annual uniform crime report.

• USA Patriot Improvement and Reauthorization Act of 2005, Public Law 109-177 (March 9, 2006) H.R.3199: Section 307 (e) “Reporting of Cargo Theft–Requires the Attorney General to collect cargo theft reported by Federal, state and local officials as a separate crime category within the UCR Program and to publish those data separately as well by no later than December 31, 2006.”

• William Wilberforce Trafficking Victims Protection Reauthorization Act of 2008–Requires the FBI to collect information about offenses of human trafficking and additional information associated with prostitution offenses.

• Matthew Shepard and James Byrd, Jr. Hate Crimes Prevention Act (2009), Section 4708–Amends the Hate Crime Statistics Act (28 U.S.C. 534 note) by requiring the Attorney General to also collect data about hate crimes motivated by actual or perceived “gender and gender identity” after “race,” “including data about crimes committed by, and crimes directed against, juveniles.”

The authorizing statute will be specified in each information clearance.

2. Needs and Uses

The information collected in this program of developing and testing questionnaires and other data collection protocols will be used by staff from the FBI UCR Program to evaluate and improve the quality of the data in the surveys and assessments that are ultimately conducted. None of the data collected under this clearance will be published for reporting on the nature of crime in the United States.

Because the questionnaires being tested under this clearance are still in the process of development, the data that result from these collections are not considered official statistics. Data will not be made public excepting when results are prepared for presentations related to survey methodology at professional meetings or publications in professional journals.

Information quality is an integral part of the pre‑dissemination review of the information disseminated by the FBI UCR Program. Information quality is also integral to the information collections conducted by the FBI UCR Program and is incorporated into the clearance process required by the Paperwork Reduction Act.

The FBI UCR Program anticipates conducting testing work under this generic clearance for a number of data collections, including but not limited to the following:

1. The Return A and the Supplement to the Return A:  Starting in 2014, the FBI UCR Program will go live with a technical redevelopment of its data collection hardware and software systems. As such, law enforcement agencies will have new options for submitting monthly crime totals via the Return A and its more detailed Supplement to the Return A. Tests will be needed to verify these new methods do not contain programming errors which affect crime data. This is especially important since the FBI UCR Program is also phasing out paper form collections requiring all agencies to report data electronically by July 2014. Further, a recent change to the definition of rape in the UCR Program may require cognitive testing to ensure error is not introduced into the data from classification misunderstandings.

2. NIBRS: The FBI UCR Program NIBRS data collection changes are a burden to law enforcement agencies because often require software programming changes to agencies’ record management systems. NIBRS does not have a paper collection form. As such changes to NIBRS often require burden assessments not from front line data entry points (law enforcement officer incident reports) but rather from either law enforcement information technology (IT) employees or contracted vendors. Testing for NIBRS purposes focus on cognitive testing to ensure changes or additions to the NIBRS data collection are properly understood and do not introduce error through unclear documentation or instruction and to assess burden costs.

3. The Supplementary Homicide Report (SHR): The SHR is a more detailed incident report which is collected when a homicide occurs within a law enforcement agency’s jurisdiction. Similar to NIBRS, when changes are made to the SHR burden assessments and data integrity pretesting will need to be done to reduce the amount of error being reported due to system designs and reporting methods.

4. Monthly Return of Arson Offenses Known to Law Enforcement: There are no anticipated changes to the arson data collection for the FBI UCR Program aside from the requirement for agencies to report crime data electronically after July 2014. Testing to ensure standardized electronic reporting does not add error to the Arson data collection will be done and may require some survey or other outreach to agencies.

5. Age, Sex, Race, and Ethnicity (ASRE) of Persons Arrested: The FBI UCR Program collects information from law enforcement agencies on the age, sex, race, and ethnicity of persons arrested within their jurisdictions. Occasionally, new categories for race and ethnicity may be added to the arrest categories when mandated by OMB directives, or when the FBI Director approves changes recommended by the FBI UCR Program’s governance, called the Advisory Policy Board.

6. Hate Crime: The FBI UCR Program’s Hate Crime data collection has recently added new bias crime codes for a large number of anti-religious and anti-ancestry motivated crimes. The FBI UCR Program anticipates that extensive cognitive testing, data collection instrument testing, and other pretests to verify validity and reliability of data submissions will be needed over the three years covered by this general clearance. Prior to a pretest, the FBI UCR Program will notify the OMB of the scope the study and anticipated burden law enforcement agencies will experience. As with the other FBI UCR Programs, the Hate Crime data collection is also undergoing a system redevelopment and falls under the same electronic submission requirement for agencies to follow by July 2014.

7. Human Trafficking: While human trafficking offenses will be reported via the Return A or in the NIBRS data collection, human trafficking also has a separate data collection form which may require special pretesting if changes are made. Most of the time, these changes are covered under the Return A and NIBRS pretests discussed previously, however, when the human trafficking form pretests are needed, the FBI UCR Program will inform OMB for comment and feedback prior to the administration of the pretests.

8. Cargo Theft: The FBI UCR Program’s cargo theft data collection was not pretested prior to its most recent OMB approval. As the three-year limitation for the cargo theft will expire within the three-year approval of this general clearance, all of the needed pretesting for the next cargo theft data collection approvals will fall under this general clearance. In particular, cognitive testing of the data collection instruments will be needed prior to and included in the next OMB approval. Other needed cargo theft pretesting will be indentified and reported to the OMB prior to their administration.

9. Law Enforcement Officers Killed and Assaulted (LEOKA): The LEOKA Program has three unique data collections intended to understand the number and characteristics of violence perpetrated against the Nation’s law enforcement officers. Monthly summaries are reported in a manner similar to the Return A and will follow similar pretesting activities. LEOKA’s long form, an FBI agent administered 22-page detailed accounting of incident characteristics when a law enforcement officer is feloniously killed, will need focus group and cognitive lab testing prior to being reapproved.

3. Use of Information Technology

When the survey or assessment being pretested employs automated methods for its data collection, the research conducted under this submission will also use automated data collection techniques. This clearance offers the FBI UCR Program an opportunity to try innovative technologies that can reduce burden, improve data quality and reliability, and increase the use of information technology.

4. Efforts to Identify Duplication

This research does not duplicate any other questionnaire design work being done by the FBI or other Federal agencies. The purpose of this clearance is to stimulate additional research, which would not be done under other circumstances due to time constraints. This research will involve collaboration with staff from other agencies that are sponsoring surveys conducted by the FBI UCR Program, when applicable. The research may also involve joint efforts with staff from other Federal laboratory facilities. All efforts would be collaborative in nature, and no duplication in this area is anticipated.

To the maximum extent possible, we will make use of previous information, reviewing results of previous evaluations of survey data before we attempt to revise questionnaires. However, this information is not sufficient to refine our survey questionnaires and assessments without conducting additional research.

5. Minimizing Burden

This research will be designed as relatively small-scale data collection efforts. This will minimize the amount of burden required to improve questionnaires, data collection instruments, and procedures, to test new ideas, and refine or improve upon positive or unclear results from other tests. The results of the research conducted under this clearance are expected to improve the methods and instruments utilized in full scale studies and thereby improve information quality while minimizing burden to respondents.

6. Consequences of Less Frequent Collection

This clearance involves one-time questionnaire and data collection research and development activities for each survey or data collection connected with the clearance. If this project were not carried out, the quality of the data collected would suffer. In addition, activities covered under this clearance will allow for more specific and precise calculation of burden hours and costs associated with data collection and survey efforts conducted by the FBI UCR Program.

7. Special Circumstances

All the guidelines listed in the OMB guidelines are met. There are no special circumstances.

8. Consultations Outside the Agency

The 60-day and 30-day was published in the Federal Register and no comments were received.

Consultation with staff from other Federal agencies that sponsor surveys conducted by the FBI UCR Program will occur in conjunction with the testing program for the individual survey. Consultation with staff from other Federal laboratory facilities may also occur as part of joint research efforts. These consultations will include discussions concerning potential response problems, clarity of questions and instructions, and other aspects of respondent burden. Additional efforts to consult with potential respondents to obtain their views on the availability of data, clarity of instructions, etc., may be undertaken as part of the testing that is conducted under this clearance.

9. Paying Respondents

While no currently proposed projects involve the use of incentives, the FBI UCR Program may develop other projects where incentives could be used. Incentive plans will be reported to OMB in the project pretesting memo prior to their use in a study.

10. Assurance of Confidentiality

According to 42 U.S.C. Section 3735, the information gathered under this clearance shall be used only for statistical or research purposes, and shall be gathered in a manner that precludes their use for law enforcement or any purpose relating to a particular individual other than statistical or research purposes. All respondents who participate in research under this clearance will be informed that the information they provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form and that their participation is voluntary. For personal visit and telephone interviews, this information will be conveyed verbally by the interviewer. For personal visit interviews, respondents will also be notified in writing to give them something they can keep and read. For self-administered questionnaires, the information will be included in the mailing package, recruitment communications and materials, either on the questionnaire, or the instructions. For Internet-based data collections, this information will be displayed prominently, and in a format that allows the respondent to print it out. All participants in cognitive research will be required to sign written notification concerning the voluntary and confidential nature of their participation. We will also inform respondents in writing of the need to have an OMB number. No participant direct identifiers will be maintained. The Generic Information Clearance will specify the particular authority for the data collection.

11. Justification for Sensitive Questions

It is possible that some potentially sensitive questions may be included in questionnaires that are tested under this clearance. One of the purposes of the testing is to identify such questions, determine sources of sensitivity, and address concerns related to those questions, insofar as possible, before the actual survey is administered. Justification for any sensitive questions included in a project covered by this generic clearance will be included in the individual project submissions.

12. Estimate of Hour Burden

We estimate the hour burden for exploratory, field test, pilot, cognitive, and focus group work will average about 1,800 burden hours per year. Primarily, field tests, pilot tests, cognitive labs, and focus groups are anticipated to need 30 respondents for each of the programs within UCR. As there are nine current programs (See section 2, Needs and Uses) and the anticipated addition of an animal cruelty data collection within the time frame covered by this general clearance,
300 respondents for focus group and cognitive testing are anticipated annually over the next three years. Secondarily, we anticipate conducting UCR change burden surveys which ask how much it will cost law enforcement agencies to implement hardware and software upgrades to report UCR data in the most current formats. These assessments will be sent to the population of approximately 18,000 law enforcement agencies reporting UCR data and these burden estimates will only take about five minutes to complete and are conducted via emails and other electronic communications means, primarily through <www.leo.gov>. In many cases, the UCR Program office will only contact UCR State Program Offices and direct contributors to conduct change burden studies, however, many of the UCR State Program Offices will forward the survey to their contributing agencies. As the FBI UCR Program Office is in essence affecting the UCR State Program Offices to collect burden data from contributing agencies, we feel it necessary to include their burden in our estimates according to the guidelines of the PRA.

Table 1: Burden Estimate

	Qualitative Testing			Burden Assessments
Year	Particpants	Hrs/Participant	Hours	Participants	Hrs/Participant	Hours	Total Hrs
2014	300	1	300	18,000	0.083	1500	1800
2015	300	1	300	18,000	0.083	1500	1800
2016	300	1	300	18,000	0.083	1500	1800
3-Year	900	1	900	54,000	0.083	4500	5400

By combining 300 respondents for hour-long pretesting sessions (300 hours) and 18,000 five-minute burden assessment interviews (1,500 hours), the estimated the annual burden hours across all proposed project activities will be 1,800 hours. The total estimated respondent burden is approximately 5,400 hours for the period from January 2014 through December 2016.

A variety of forms will be used in conducting the research under this clearance, and the exact number of different forms, length of each form, and number of subjects/respondents per form are not thoroughly known at this time. However, we can project that our activities will likely include testing items and data collection modes, and conducting pilot tests, cognitive labs or interviews, exploratory interviews, re-interviews, behavior coding and focus groups.

13. Estimate of Cost Burden

There is typically no cost to respondents for participating in the research being conducted under this clearance, except for their time to complete the questionnaire or participate in an interview or focus group.

14. Cost to Federal Government

It is very difficult to anticipate the actual number of participants, length of interview, and/or mode of data collection for the work to be conducted under this clearance over the entire 3-year clearance period. Without that information, it is not possible to estimate in advance the cost of the work under this general clearance to the Federal Government. Costs associated with each individual project will be covered by the statistical unit conducting the research and will come from their data collection budgets. Information about costs in the individual submissions will be provided to OMB prior to pretesting.

15. Reason for Change in Burden

There is no change in burden, as this is an initial request for a generic information clearance.

16. Project Schedule

This research program is for questionnaire and procedure development purposes. Data tabulations will be used to evaluate the results of questionnaire testing. The information collected in this effort will not be the subject of estimates or other statistics in FBI UCR Program reports; however, it may be published in research and development reports or be included as a methodological appendix or footnote in a report containing data from a larger data collection effort. The results of this research may be prepared for presentation at professional meetings or publication in professional journals. Due to the nature of this clearance, there is no definite or tentative time schedule at this point. We expect work to be conducted more or less continuously throughout the duration of the clearance.

17. Request to Not Display Expiration Date

No exemption is requested.

18. Exceptions to the Certification

There are no exceptions to the certification.

1