INFORMATION COLLECTION
SUPPORTING STATEMENT
B. Collections of Information Employing Statistical Methods
In order to identify challenges in collecting a future nationally representative sample with this survey methodology and questions the proposed pilot study will employ statistical methods to analyze the information collected from respondents. It will use a stratified cluster design with law enforcement officers and supervisors completing the survey using a web-based application and agency head interviews being conducted by phone. Because this is a pilot study, inference will be limited to the sample, and results will not be mapped to the larger universe of all law enforcement agencies in the U.S. The following sections describe the procedures for respondent sampling and data tabulation.
B.1. Describe the potential respondent universe and any sampling or other respondent selection method to be used.
Respondent universe
The respondent universe theoretically consists of law enforcement staff from police, county sheriff and state highway patrol agencies in the United States, including Alaska, Hawaii, and the District of Columbia. The sample frame is the Census of State and Local Law Enforcement Agencies (CSLLEA) data set, which was last published in 2008 (http://www.bjs.gov/content/pub/pdf/csllea08.pdf) and is due to be updated in the upcoming months. This data source is the most complete and up-to-date source of information about law enforcement agencies (LEAs). According to the 2008 CSLLEA dataset, there are 17,985 state and local law enforcement agencies across the United States that employ at least one full-time officer (or the part-time equivalent). This sample is made up of 12,501 local police departments, 3,063 sheriffs’ offices, 50 primary state law enforcement agencies, 1,733 special jurisdiction agencies, and 638 other agencies (primarily constable offices in Texas). In addition, these agency types can be found in diverse geographical regions and are of varying size. The proposed pilot study will collect data from a stratified sample of these agency types and is described below.
In addition to agency types we are interested in sampling from different types of officers within each agency, simple random sampling will identify officers and supervisors to complete the survey (as well as one agency head). Each selected sample member will be provided a packet of information about how to access the web survey. We anticipate no coverage bias relating to the use of a web survey with this population because the respondents will have access to the web at work. The officers and supervisors may be allowed to use agency resources to complete the survey. Alternatively, the web instrument could be completed outside of the agency (e.g., via personal internet access at home, a public library, or another location) at the respondent’s discretion. There will be no weighting adjustments for internet access.
Statistical sampling methods
A sample of approximately 40 Law Enforcement Agencies will be selected in order to yield 1,200 completed surveys from line officers, 80 from supervisors, and 40 from command level staff. The survey will be a stratified cluster sample, representing Law Enforcement Agencies across the country with a variety of characteristics, such as agency size and type. The sample sizes discussed in this section are not selected to achieve specific statistical targets of precision because this is a pilot study. However, any future national sample will use one informed by the results of this pilot study.
The pilot study sample will be created in two-stages where the first stage of selection is the LEA, and the second stage of selection is the officers of a selected agency. The first stage sample will consist of selecting a state police agency (1), local police agencies stratified by three sizes (3), sheriff or constable based agencies also stratified by size (3), and three agencies stratified by size that are categorized as special jurisdictions (3). This sample will be duplicated across four geographic regions of the country to sample a total of 40 LEAs. Two replacement agencies in each stratum will be selected as replacements if the sampled agency is unwilling to participate.
Within each LEA a simple random sample of law enforcement officers and supervisors will be selected. The agency contact person will provide a list of officers and supervisors for sampling. The list will be randomized and the appropriate number of officers selected. At small LEAs, all officers will be selected. Larger agencies will have a simple random sample selected. The target number of completed interviews is 1,200 law enforcement officers, 80 supervisors, and 40 command level staff. The total number of sampled officers will allow for a response rate of approximately 60%. Thus, 4,450 officers will be selected to obtain 1,200 completed surveys from line officers. For the supervisors, we will expect a higher (80%) completion rate and sample accordingly. The respective response rates were estimated based on past experience with mail-to-web surveys and experience with other types of surveys of Law Enforcement personnel and agencies. The higher response rate for supervisors is based on their importance in and commitment to the organization.
B.2. Describe the procedures for the collection of information.
Data collection strategy
The Survey of Law Enforcement Officers/Agencies: Attitudes Towards and Resources for Traffic Safety Enforcement will be conducted with an average sample of 30 law enforcement line officers, two law enforcement supervisors, and one command-level staff interview at each of the 40 sampled law enforcement agencies. There will be approximately 1,200 completed web-based officer surveys and 80 completed web-based supervisor surveys. In addition, a telephone interview will be conducted with one member of each agency’s command-level staff for a total of 40 completed agency head interviews.
After the sample of Law Enforcement Agencies is selected and willing agencies are selected, staff will mail a package of information to the head of the agency. The package will include a lead letter introducing the study on NHTSA letterhead and signed by a member of NHTSA’s senior management. The letter will inform the agency that a project staff member will call the agency within a week to answer any questions that they have and ask for their participation in the study (see Appendix F). The package will also include a letter showing support for the survey which will include the logos of national agencies that support the data collection effort (see Appendix G). Approximately one week following the mailing, project staff will begin calling each agency to confirm receipt of information and to verify the correct point of contact.
Data collection will be conducted according to a schedule designed to facilitate successful recruitment of law enforcement agencies to develop a point-of-contact (POC) and agree to participate. The POC will produce a “de-identified” (i.e., no names of officers but rather a reference number such as a badge, roster line, employee ID number, etc.) list of current officers and supervisors. The provided list will be subjected to a random sampling process. Once officers are selected, the POC will receive a set of sealed envelopes labeled with the roster line (or other chosen ID) number for each sampled officer and supervisor. Each envelope contains an invitation letter to voluntarily participate in the survey. The letter provides an overview of the study and includes a link to the web-based survey along with a username and password specific to the selected respondent (see Appendix H). The POC will distribute the envelopes containing a description of the study and information on participation to the sampled officers. A follow-up thank you / reminder letter will be delivered via POC to all sampled officers and supervisors approximately 2 weeks following the initial invitation (see Appendix I). The letters will go to all selected staff so the POC does not know who participated or not.
The semi-structured agency head telephone interviews will be arranged by the POC. Since the POC will likely be a command-level staff member, he/she may be the person to complete the agency head interview. Otherwise, the POC will arrange for another command-level officer or, preferably, the head of the agency. Interviews will be conducted by a project staff member using a semi-structured instrument to collect quantitative and qualitative data during the call.
The POC will be asked to arrange for the completion of one additional document. The document will include specific administrative traffic-related data (itemized in Appendix E4) to prepare and send via mail or electronically.
B.3. Describe methods to maximize response rates and to deal with issues of non-response.
Cognitive testing
The law enforcement officer/supervisor/ and agency head instruments were cognitive tested in order to identify any problem issues that could deter respondents from completing the survey. Based on the cognitive testing results, a number of questions were revised or deleted.
Initial contact
The initial contact with the sampled law enforcement agency is crucial to the success of the project. While the brief introduction to the study concerning its sponsorship, purpose, and conditions will be sufficient for many agency respondents, others will have questions and concerns that the researcher must address. A respondent's questions should be answered clearly and simply. Project staff making recruitment calls will be trained on how to answer the most likely questions for this survey. An agency call protocol will be provided to each staff member making calls. Project staff will also be trained to answer all questions in an open, positive, and confident manner, so that agencies are convinced of the value and legitimacy of the study. Above all, project staff will attempt to create a rapport with the agency in the process of securing participation.
Refusal documentation and tracking
The Survey Manager and the Project Director will monitor the data regarding the agency recruiting progress to identify any problems with the agency protocol that might contribute to non-participation on the part of agencies. Adjustments that might be identified to mitigate LEA refusals to participate will be implemented and their impact monitored. Per our recruiting protocol, non-response on the part of LEOs will be monitored and reported once to the agency POC in terms of number of LEOs who have responded/not responded to date. As we won’t have actual identification information on the individual LEOs, we cannot make direct (e.g., via telephone or email) with individual non-responding officers. Respondents having any specific concerns will be able to ask questions or receive technical assistance from the web landing page as well as via telephone contacts provided in the survey invitation packet and web survey consent screen.
B.4. Describe any tests of procedures or methods to be undertaken.
Testing the programming
The project team will test the web-based survey instruments thoroughly with in-house staff. The analytical staff will attempt to test all possible response categories for each question in order to identify embedded logic errors, as well as obvious skip problems. Several analysts may test the program simultaneously to identify problems quickly, and to double check the comprehensiveness of the testing.
Survey pretest
The law enforcement officer/supervisor/ and agency head instruments were cognitive tested in order to identify any problem issues that could deter respondents from completing the survey. Based on the cognitive testing results, a number of questions were revised or deleted.
B.5. Provide the name and telephone number of individuals consulted on statistical aspects of the design.
The following individuals have reviewed technical and statistical aspects of procedures that will be used to conduct the Survey of Law Enforcement Officers / Agencies: Attitudes Towards and Resources for Traffic Safety Enforcement
Paul Biemer, DrPH
RTI Distinguished Fellow, Statistics
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
(919) 541-6056
Marcus Berzofsky, DrPH
Senior Research Statistician
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
(919) 316-3752
Cynthia Bland, MPH
Research Statistician
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
(919) 541-6154
James S. Higgins, Ph.D.
Research Psychologist
National Highway Traffic Safety Administration
1200 New Jersey Ave., SE
Washington, DC 20590
(202) 366-3976
| File Type | application/msword |
| File Title | TABLE OF CONTENTS |
| Author | ABlock |
| Last Modified By | USDOT_User |
| File Modified | 2016-01-14 |
| File Created | 2015-12-28 |