SupportingStatementSectionB final

Community Policing Self-Assessment Tool (CP-SAT)

Part B. Statistical Methods

Part B. Collections of Information Employing Statistical Methods:

The CP-SAT effort will require the employment of only descriptive statistical methods.

B.1. Respondent Universe and Sampling Methods

For the vast majority of COPS Office hiring program grantees, the CP-SAT is intended to be distributed to all relevant employees and as such, the COPS Office will not be determining sampling methods. There are five groups of individuals who are intended to complete the survey within each participating agency: Officers, Supervisors, Command Staff, Civilian Staff, and Community Partners (specific individuals with whom the agency has a recognized partnership, not the general public).

In very large agencies (e.g., 1,200 or greater sworn staff), there is a point at which gaining more survey responses has less statistical benefit than the burden of the additional officers’ time (i.e., the decrease in standard error resulting from the increased responses is negligible). Thus, we offer sampling services for agencies over 1,200 sworn staff in which we identify a sample frame of 1,200 sworn staff using random sampling. To pull the random sample, the agency provides a roster of all sworn staff. Agencies wishing to protect the anonymity of their staff may use a unique identifier instead of names or email addresses. A set of random numbers are generated in Microsoft Excel and assigned to each staff person. The random numbers are sorted and the sample frame is drawn from the top 1,200 cases. If demographic information (e.g., rank, tenure) is provided, the random sample is checked against the proportion of staff in each level of each demographic variable. In the rare case that a random sample deviates significantly from the demographic proportions in the population, a new random sample will be drawn. The goal of providing sampling assistance to some CHP grantees is to minimize the burden on the agency in administering the CP-SAT so they can fulfill their grant requirement. Note that some of these agencies still prefer to distribute the survey to all relevant employees.

B2. Procedures for the Collection of Information

Each CHP grantee is contacted by the COPS Office within 1-4 months of returning their signed award and advised of this grant requirement and their future administration date. Agencies are grouped together and assigned in a specific timeframe (referred to as a “wave”) for administration.

The provider then contacts the grantees who are asked to assign a survey administrator (“key contact”) who will assist in the administration of the CP-SAT to participants. A sample notification letter is provided in as an appendix item. The survey administrator’s responsibilities include sending the initial survey invitation, which includes a hyperlink to the online survey, and at least two reminder emails, as necessary. The provider disseminates instructions on the survey process to the agency prior to their assigned administration date, and also provides other materials to support the administration process (e.g., communication templates to provide more background to staff and convey support from agency leadership). The provider also tracks survey response rates and sends an update to the agency regarding the level of participation and responses needed to complete the assessment.

At the completion of the process, the provider submits a final report to the chief executive of the law enforcement agency and the survey administrator, which summarizes the agency’s aggregate assessment results. Although the report does not interpret the data collected, it allows the agency to assess the extent to which community policing has been implemented across various components of the organization and among units and ranks. The agency will be provided descriptive statistics (e.g., number of responses, mean, and standard deviation) for each item on the CP-SAT to further aid in report interpretation and strategic decision making. There are no individual identifiers in the data, and the agency will not be able to link an individual’s data to the participant. The executive will determine how the report is distributed in his or her agency and community.

Administration of the CP-SAT is repeated toward the end of a grantee’s award period, at which time they will receive a report that displays their scores for both administrations (i.e., pre/post) to document changes in community policing activities over time.

The CP-SAT is administered in an online format using Vovici EFM Community Web-based survey software. Vovici’s survey hosting environment has been designed with security as a foremost consideration, with features such as 128 bit SSL encryption and redundant firewalls. Responses to the assessment are anonymous. There are no individual identifiers in the data and there is no way to link an individual’s data to their name or email address. Participants will be invited to participate by sending the survey link via email. If the respondent does not have an email account, participants can be directed to a URL address via alternate means (e.g., via agency memo with URL address, access to a common computer lab with the site loaded onto each computer). The confidentiality statement will also appear at the beginning of the alternative survey. All data exported from the Vovici Community secure Web site will be kept in a secured folder.

Emails to participants will include the following statement prominently displayed notifying them of the confidential nature of their surveys: “Your responses to this survey will be kept confidential. There are no individual identifiers in the data that the law enforcement agency will receive, and the agency will not be able to link an individual’s data to their email address. This is not a test and there are no right or wrong answers. Please answer each question honestly.”

To ensure that the data distributed to participating agencies does not compromise respondent confidentiality, agencies must have at least three participant responses for a given item or set of items in order for the summary data to be displayed on the agency report. If only certain items, subsections, or sections have less than three responses, summary data is provided for all of the report with the exception of those specific sections, subsections, or items with fewer than three responses. If multiple respondent types (e.g., command staff, line officers) each have fewer than three respondents, data from those respondents are not segmented and presented, but rather they are only included with the rest of the respondent-types as aggregate findings. These confidentiality protections do not apply to very small agencies with four or fewer sworn staff.

Note that the CP-SAT provider cannot protect respondent confidentiality while delivering agency results in very small agencies under the standard survey administration process. To address this, agencies with sworn force levels of four or fewer will be instructed to complete the survey as a group. They will receive specific instructions in how this should occur, along with a reminder that the confidentiality of individual responses will not be protected under this process. Moreover, an individual review of several data items for these agencies will occur to add additional verification that they followed the instructions to complete the survey as a group. This small-agency modified process will ensure that the agency will receive a report summarizing their results, which would not be possible if the surveys were completed independently.

At the conclusion of the data collection covered by this request, ICF and the contractor will make agency-level data available to the government with identifying information removed from the data set (also see description in Section A.10 above).  This data could be useful in identifying areas of need of community policing assistance and resources, and for conducting large-scale research on the state of community policing implementation across different community policing dimensions.

B3. Methods to Maximize Response Rates and Deal with Issues of Non-Response

COPS will not be directly involved in the collection of data. However, we will provide tips for increasing response rate within an agency. These tips include:

• Chief should stress the importance of the self-assessment process and participation in completing the forms.

• Gain support and “buy in” from each of the groups—officers, supervisors, and command staff—as well as police union leadership or other organized labor bodies. Support from these individuals and groups should be sought at the outset of the project.

• Training/orientation for agency personnel who will be asked to complete the assessment tool form will enhance the overall response rate and reduce the problem of incomplete forms being submitted.

• Respondents should be notified in advance that the assessment tool form will be distributed and that they will be given adequate time to complete the form.

• Respondents should be assured that their responses will be handled in a confidential manner.

• Respondents should be given reminders to complete the assessment tool forms.

• Offer to provide the respondents a summary report of the results of the self-assessment process.

• Convey how the results will be used to make positive changes, as well as to promote the work being done by the agency and its staff.

Additionally, the contractor will monitor response rates of each agency closely and provide agency-specific updates to the survey administrator of each agency. For agencies with low response rate, additional email and phone contact is made by the contractor to help answer questions, identify and rectify challenges encountered, and encourage participation.

B4. Tests of procedures or methods to be undertaken

As stated in earlier supporting statements, the project development and research team conducted six pilot tests of the content of the survey and usability of the format in law enforcement agencies across the country. Based on responses and the analysis of findings, improvements were made to both the content and the online format.

B5. Individuals consulted on statistical aspects of the design and organization/persons collecting and analyzing the data.

Beth Heinen, Ph.D.

Manager

Workforce Research and Performance

ICF International

(314) 918-0373