Pilot Test Report

Attachment 3 to Supporting Statement for Paperwork Reduction Act Submission

2014 National Institute of Justice Geospatial Capabilities Survey

Pilot Test Report

Carolyn Wong

RAND Safety and Justice Program

PR-707-NIJ

October, 2013

Prepared for National Institute of Justice

Not Cleared for Public Release

This document has not been formally reviewed, edited, or cleared for public release. It should not be cited without the permission of the RAND Corporation. RAND's publications do not necessarily reflect the opinions of its research clients and sponsors. is a registered trademark.

Table of Contents

Tables 3

1. 2014 National Institute of Justice Geospatial Capabilities Survey Pilot Test 4

Conduct 4

Feedback 5

Tables

Table 1: 2014 NIJ Geospatial Survey Pilot Test Participants and Administration Times 4

Table 2: Feedback Regarding Clarity of Survey Questions 6

Table 2: Feedback Regarding Clarity of Survey Questions (Continued) 7

Table 3: Feedback Regarding Clarity of Response Choices 8

Table 3: Feedback Regarding Clarity of Response Choices (continued) 9

Table 4: Feedback Regarding Ability to Answer Question 10

Table 4: Feedback Regarding Ability to Answer Question (continued) 11

1. 2014 National Institute of Justice Geospatial Capabilities Survey Pilot Test

Conduct

The 2014 National Institute of Justice (NIJ) Geospatial Capabilities Survey Pilot Test was conducted by the National Law Enforcement and Corrections and Technology Center (NLECTC) for Information and Geospatial Technologies Center of Excellence (COE), which is operated by The RAND Corporation. The RAND leader for the NIJ Geospatial Capabilities Survey task was responsible for all pilot test activities including contacting potential participants, providing access to the survey questionnaire and pilot test evaluation form, receiving responses, and conducting follow-up activities to encourage timely submissions. Pilot participants were nine law enforcement agencies recommended by the NIJ or the RAND COE. These nine participants were asked to receive the NIJ Geospatial Capabilities Survey invitation, access the survey questionnaire by a means detailed in the invitation, complete the questionnaire, complete a pilot test evaluation form, and return the completed questionnaires and pilot test evaluation forms. All nine pilot participants completed the questionnaire and pilot test evaluation form and returned both to RAND. All nine participants chose to participate in the pilot test using the electronic version of the questionnaire. Table 1 shows the pilot test participants identified by county and the amount of time each participant required to complete the survey questionnaire.

Table 1: 2014 NIJ Geospatial Survey Pilot Test Participants and Administration Times

On average, participants completed the NIJ Geospatial Capabilities Survey questionnaire in 49 minutes with a range of 20 minutes to 90 minutes. The median time was 45 minutes

Feedback

The pilot test evaluation form was used to provide RAND feedback on the pilot test experience. The evaluation asked administrative questions like the amount of time required to complete the survey questionnaire, but primarily focused on the clarity of the questions, the clarity and completeness of the response options, and the availableness/accessibility of the information required to answer the questions.

Feedback from the pilot indicated that the instructions to the survey were clear and easy to follow. All pilot participants used the electronic version of the survey. One participant had difficulty accessing the website and RAND quickly diagnosed and fixed the problem. RAND considered every comment and if there was any indication that the question or response choices were not clear, the question or response was reworded to provide clarity. Where participants found response choices confusing, the response choices were reworded or modified to address the issue.

Syntheses of the feedback from the NIJ Geospatial Capabilities Survey Pilot Test are shown below. Table 2 shows the comments received regarding the clarity of the questions. Table 3 shows the comments received regarding the clarity of the response choices. Table 4 shows the feedback for the participant’s ability to answer the question.

Table 2: Feedback Regarding Clarity of Survey Questions

Table 2: Feedback Regarding Clarity of Survey Questions (Continued)

Table 3: Feedback Regarding Clarity of Response Choices

Table 3: Feedback Regarding Clarity of Response Choices (continued)

Table 4: Feedback Regarding Ability to Answer Question

Table 4: Feedback Regarding Ability to Answer Question (continued)