Supporting Statement Part Bfinal12-28-10

SUPPORTING STATEMENT

COPS Rural Law Enforcement Training Needs Assessment Survey

Part B: Collection of Information Employing Statistical Methods

1. Respondent Universe and Sampling methods:

The table below provides an overview of the groups from which the project team will draw the survey sample, including the respondent universe and sampling plans.

Table 1: Survey Sample

Survey Population Type	Estimated Respondent Universe	Sampling
Rural Law Enforcement Agencies	10,000 Rural Law Enforcement Executives	3,000 rural county and local general and special jurisdiction law enforcement agencies, randomly selected, and stratified by region of the country.
State Police	49 State Police Agency Law Enforcement Executives	No sampling—All 49 primary State police agencies.
State Special Jurisdiction Law Enforcement Agencies	300 State Special Jurisdiction Law Enforcement Executives	No sampling—All 300 State-level special jurisdiction law enforcement agencies that play important roles in rural areas within their States
Tribal Police	171 Tribal Police Agency Law Enforcement Executives	No sampling— All 171 Tribal police agencies.
State Peace Officer’s Standards and Training (POSTs)	49 POST Law Enforcement Executives	No sampling—All 49 State POSTs.
Police Training Academies	500 Police Training Academy Law Enforcement Executives	No sampling—All 500 police training academies serving rural areas (in whole or in part).
Police Training Academy In-service Trainees	300,000 Rural Law Enforcement Field-level Personnel	50 police training academies serving rural areas (in whole or in part), randomly selected, and stratified by region of the country. Within each academy, all in-service field-level personnel attending training during a designated 1-week period will be surveyed (an estimated 2,500 field personnel).

Precision Analysis: The project team acknowledges that the sample sizes of law enforcement executives (agencies) and field-level personnel appear large. This is driven largely by the RPI’s interest in determining the distinct training needs of law enforcement personnel: (1) in each of the five regions of the country, (2) by State, as possible, and (3) by agency type (e.g., municipal police, sheriffs, campus police, etc.).

The RPI’s need for this information is practical, not academic. RPI has the responsibility to develop and deliver law enforcement training that meets the needs of rural law enforcement personnel throughout the country. Those needs vary for a range of reasons, including: (1) differences in regional characteristics/conditions (such as the conditions in law enforcement in rural Massachusetts versus rural Montana), (2) differences in agency type jurisdiction/responsibilities (such as the varying responsibilities of campus police, a sheriff’s, or a municipal police department), and (3) differences in the availability of training by jurisdiction.

The agency executive sample is 600 per region. This will provide confidence intervals of < ±5% for regional-level estimates. Thus, for example, if the mean sample response in the Midwest regarding the current need for training on “strategies to reduce gang problems” is 2.0 (medium need), the project team will be confident that the population parameter is within 1.9-2.1. If the mean sample response for another training topic was 1.7, the RPI could be confident that “strategies to reduce gang problems” was the greater of the two training needs in that region. This level of precision is not unreasonable given the breadth and diversity of law enforcement agency training needs and the decisions that RPI must make about which courses to develop and where to offer them.

It is important to emphasize that RPI also desires estimates of training need for regional sub-samples. The confidence intervals for each State (sample n = 60 on average) will average 13%. The confidence interval for campus agencies in the Northeast region (sample n = 28, as an example) will be 19%. This level of precision is certainly not unreasonable and illustrates the need for what might seem, at first, to be relatively large regional/national samples.

Response Rate: Overall, the project team anticipates a 60% overall response rate, with higher response rates for agencies and institutions (law enforcement agency and training academy law enforcement executives) and a lower rate for field-level personnel. This survey has not been conducted previously. National establishment surveys (e.g., those conducted with organizations as opposed to individuals) in the law enforcement field typically achieve response rates in the 30%-90% range, depending on the topics being investigated and the follow-up level of effort. National surveys of individual field-level law enforcement personnel have been much less common, and it is harder to predict the response rate that will be achieved for that segment of the proposed survey population

2. Procedures for the Collection of Information

This is a one-time survey of several types of institutions/agencies and individuals engaged in rural law enforcement practice and/or providing training on rural law enforcement policy or practice. The goal of the survey is to collect information about the training needs of law enforcement personnel operating in rural jurisdictions.

The project team will send surveys to rural law enforcement executives, State POST and police training academy directors, serving rural communities; they also will survey field-level law enforcement personnel during their attendance at training. They will send the survey to 100% of the known respondent universe in five of the categories (State police agencies, State-level special jurisdiction law enforcement agencies, Tribal police agencies, and State POST and law enforcement training academies directors) because the universe (population) for each group is fairly small.

The team will also send the survey to a randomly selected sample of law enforcement agency executives. They will select the random sample of law enforcement agencies from the five geographic regions defined by the RPI/FLETC. The team will stratify these samples by region because RPI is interested in collecting information about the specific training needs of rural law enforcement personnel in each region, the results of which will guide the agency’s allocation of resources for future training deliveries.

Sampling: In addition, the project team will employ specialized sampling to obtain representative responses from field-level personnel (e.g., patrol officers, deputy sheriffs, troopers) because there is not a national database of field-level law enforcement personnel. They therefore plan to select e a sample of the POST and State and local law enforcement training academies (from those that indicate a willingness to participate in response to an invitation from the project), and then solicit via those academy directors the participation of all of the field-level law enforcement personnel who attend in-service training at those academies during a 2 week period. (The team will not engage in the survey basic/recruit or pre-service trainees who do not have on-the-job law enforcement experience on which to base their responses to the survey questions about the training needs of law enforcement personnel serving rural areas.)

Precise estimation is not an objective of the study because it is not necessary to determine the exact proportion of respondents who indicated that they need additional training on a specific topic. It will be important, however, to determine whether training on a particular topic: (1) is needed by a majority of respondents, (2) is in the top tier of needed training, (3) is needed by law enforcement personnel in every region (versus in only one or two regions), and (4) is needed by the various types of law enforcement personnel serving rural jurisdictions (e.g., by deputy sheriffs but not by police officers or conservation officers).

The RPI/FLETC needs reliable information about the training needs of law enforcement professionals serving rural areas and the training available to them. The agency may then allocate, with confidence, future resources for developing and delivering training. The project team will meet this need if the survey determines, for example, that training on five particular topics is of greater need than training on other topics. Of equal importance, RPI/FLETC needs to know which topics are of greatest need in each of the five regions designated as its service areas (e.g., the training needed by law enforcement professionals in rural areas in the Northeast region as compared to those serving rural areas in the Midwest region). RPI/FLETC is also interested in which training topics are reported as the greatest need by the different types of law enforcement personnel (e.g., Tribal law enforcement agency personnel as compared to sheriff’s department personnel). We therefore are surveying multiple populations/samples to ensure that the data will address those specific information needs. We also are surveying relatively large samples to obtain reasonably reliable information for each of five regions and for the law enforcement professionals working in different types of agencies.

This is the first administration of the national training needs assessment of law enforcement personnel serving rural areas. RPI/FLETC has not determined whether future assessments will be conducted.

The team has selected the methodology described above because of the diverse nature of the law enforcement personnel and agencies serving rural areas and therefore the likelihood of variation in the training needs identified by agency and personnel type and by region and State. They will survey multiple types of law enforcement constituencies, and have fairly large samples for those groups from which the random samples will be drawn.

Survey Deployment: The project team will use an array of recognized survey deployment strategies, including:

• Sending an e-mail to key stakeholder organizations to remind them to disseminate a project team-provided “e-mail blast” just before the survey is deployed to alert their law enforcement membership/constituency that they may be receiving the survey imminently and to encourage them to respond, if invited.

• Distributing the survey materials (e.g., the introductory e-mails and the surveys) on a Tuesday, which typically is considered the optimum survey day.¹

• Developing survey transmittal correspondence that is brief and engaging and describes how participation in the survey will benefit law enforcement professionals in rural areas and Indian Country.

• Providing respondents a brief one (1) week turnaround period in which to complete the survey (longer completion timeframes often result in respondents setting the survey aside and never returning to it).

Survey Instructions: In addition, the team has developed instructions about both taking and administering the surveys (See the attached Survey Correspondence Package):

The survey transmittal e-mails provide directions to recipients (e.g., the CEOs and POSTs/Training Academy Directors) for completing the survey (and accessing and completing the Web-based survey; the platform also will provide guidance).

The Survey Administration Guidance for POST/Academy Instructors Administering the Rural Law Enforcement Training Needs Assessment Field Officer Survey during In-service Training provides specific directions for administering the Field Officer Survey. (The field officers being administered the survey during a POST and/or State Training Academy training will complete it in hard copy.)

We do not intend to follow-up directly with the field officers, however, we will do so with the POST and State Training Academy Directors that are administering the survey to them. This effort will help to ensure that we collect as many of the completed Field Officer Surveys, as possible.

3. Methods to Maximize Response Rates and Deal with Issues of Non-Response

The project team will use several methods to achieve a substantial and high-quality rate of response to the survey. First, the team developed an Outreach Plan, and associated promotional materials, designed to both inform, and secure the support of, key stakeholder organizations regarding the national training needs assessment survey. They disseminated outreach correspondence to several Federal agencies working in related areas, 18 law enforcement stakeholder organizations, 49 State POSTs, and 15 Regional Community Policing Institutes.

Second, the project team produced a survey instrument and survey items that have substantial “face validity” in the eyes of rural law enforcement practitioners. The principal survey designer is a former rural law enforcement practitioner (police officer and police chief) and law enforcement trainer and State Training Commission member. He has designed and administered several national surveys of law enforcement agencies and personnel. Additionally, the project team conducted a survey instrument vetting process via which nine (9) law enforcement practitioners reviewed, completed, and provided feedback on the instruments (e.g., identifying any questions that were unclear, suggesting other training topics). The team then used the feedback to make minor revisions to the draft survey instruments before their submission to OMB.

It has been the team’s experience that surveys that are viewed by the recipients as relevant to their circumstances achieve much higher response rates, particularly in the law enforcement field.

Third the proposed survey will collect data that will inform Federal decisions about the allocation of training resources for the professional group being surveyed; the project team will include messages about the practical benefits of completing the survey in all correspondence to potential survey respondents.

Fourth, the project team will employ a multi-wave method for distributing to and following up on the survey with potential respondents. This will include an introductory letter or e-mail, which will be followed by delivery of the survey link by e-mail (i.e., providing access to the online survey), a follow-up reminder e-mail, delivery of a hard copy survey, as needed, and another e-mail reminder. They will also follow up with non-respondents by telephone, including offering to complete the survey with them during the call. This multi-wave technique has been successful in achieving respectable response rates for previous surveys of professionals in the law enforcement and criminal justice fields.

Finally, the team will provide copies of the survey to field-level personnel who attend in-service training at a random sample of law enforcement training academies during a single designated week. The field-level personnel group should approximate a random sample because in-service training requirements are fairly universal within each State and who actually attends training during any particular week is not systematically biased in any way. More important, personnel attending training are a somewhat “captive audience” and the team therefore anticipates that most of them will complete the survey.

The project team expects to obtain reliable data from the groups targeted by the survey, each of which has a vested interest in how the survey data will be used to guide the allocation of Federal training resources. With regard to the groups for which they will use random sampling to select potential respondents, the team will use larger than normal samples so that they can separately analyze the data for different geographic regions and also for different types of law enforcement agencies and personnel (e.g., municipal police, sheriffs, campus police). For the other survey groups, the team will survey the entire respondent universe/population, thereby maximizing the likelihood that they will obtain reliable and representative data. In both cases, the project team will compare the basic characteristics (e.g., size of agency, type of agency) of our actual respondents with the known population parameters so that they can assess response bias and, if necessary, correct for it when calculating any national estimates.

4. Tests of procedures or methods to be undertaken

The project team vetted the draft instruments with a few (less than 9) key stakeholders to test question clarity and reaffirm the estimated time to complete each instrument. On the basis of this process, they made minor changes to the survey items and questions. The team is satisfied that the three survey instruments are clear, understandable, and the appropriate length for the target audiences.

5. Individuals consulted on statistical aspects of the design and organization/persons collecting and analyzing the data.

CRA will manage the project in conjunction with their experienced law enforcement research consultant; below is information on the key contacts for the study:

Paula Seidman, CRA, Inc. (Contractor)

703-519-4510

Gary Cordner, Ph.D. (Consultant)

484-560-9123

Billie Coleman (COPS Office)

202-353-1706

1 For target audiences that are working professionals, it is best to avoid Friday, Saturday, or Sunday, and Mondays, during which many people are gearing up for the week ahead. (Survey Monkey: http://s3.amazonaws.com/SurveyMonkeyFiles/Response_Rates.pdf). In addition, many people use Thursday and Friday to begin to gear down for the weekend or to plan for the following week; studies have shown that the best time to send an e-mail is mid-week on a Tuesday or Wednesday between 2–3 p.m. (Zarca Interactive: http://www.zarca.com/Online-Survey-Resource/Survey-Best-Practices/Increase-Response-Rates.html).

7