CLETA2018_Supporting Statement - Part B 10192018

CLETA2018_Supporting Statement - Part B 10192018.docx

2018 Census of Law Enforcement Training Academies

OMB: 1121-0255

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 1121-0255 can be found here:

2023-02-09 - Reinstatement with change of a previously approved collection

Document [docx]

Download: docx | pdf

SUPPORTING STATEMENT – Part B

Collection of Information Employing Statistical Methods

2018 Census of Law Enforcement Training Academies (CLETA)

Universe and Respondent Selection

The 2018 Census of Law Enforcement Training Academies (CLETA) will utilize procedures successfully employed in the prior three administrations of CLETA to identify the universe of eligible academies. The universe is defined to include any academy conducting basic law enforcement training in 2018.

The 2018 CLETA frame file will be developed from the 2013 CLETA universe and respondent file. The file contains information on 839 potentially eligible training academies; however, during the 2013 CLETA, 175 academies were deemed ineligible. The frame cleaning process will focus on three tasks to update that frame: reviewing academies deemed ineligible in 2013, confirming that academies eligible to participate in 2013 remain so in 2018, and updating the frame with new academies. To complete these tasks, RTI will search and attempt to locate state-level lists, 2013 eligible academy websites, and other academy information sources available online. Information found online about academies will be checked against the 2013 CLETA frame file and records will be updated where appropriate.

During the 2013 CLETA 175 cases were deemed ineligible. Based on BJS guidance, cases with the following dispositions are to be re-evaluated for eligibility in the 2018 CLETA: agency did not exist (2); has not been a basic academy during the reference period (28); reason not specified (7); listed as complete but not in data file (18); record was a non-primary duplicate (16). A random selection of 40 out of the remaining 104 cases with the disposition of “no basic academy” will be reviewed to determine eligibility. Identification of more than four randomly selected cases among the 40 as having a basic training academy will trigger a review of all remaining “no basic academy” cases.

The best source for identifying new academies will be the State Peace Officer and Standards Training (POST) boards. The POST boards typically provide state-level training academy lists with contact details online for the public. These lists should also be a quality resource for confirming eligible and ineligible academies. Most academies also have websites that provide basic contact information. Prior to OMB clearance, there will be no direct contact with academies. Once OMB approval of this request is obtained academies that could not be verified online will be contacted by phone to confirm eligibility and identify a point-of-contact (POC) for CLETA.

The result, the 2018 CLETA frame file, will be a current and complete universe list of all regional, state, and local law enforcement training academies that provide basic training. Any duplicates will be identified and removed by examining the file for identical or nearly identical addresses, phone numbers, or names of academy officials.

As in previous administrations, the 2018 CLETA is planned to be a census rather than a sample survey. The reasons for this decision include:

The eligible population is expected to be approximately 775 academies. A sample survey of a universe of this size will not result in significant cost savings given the stratification dimensions needed to capture critical aspects of the universe. A census will allow BJS to produce national and state level parameters by type of academy, size, and other academy characteristics.

A census provides BJS the opportunity to describe how training academies vary across and within states. Being able to compare academies is particularly important considering the variability that exists among these organizations in terms of type of academy and administration, agencies served, policies, curriculum, etc. Table 3 shows the number of state and local law enforcement training academies providing basic training in 2013 by type of academy. Meaningful conclusions about academies in almost any of the strata would require a census within those strata, suggesting a sample survey strategy is not optimal for this universe.

Table 3. Distribution of Academies by Operating Agency, 2013

Primary operating agency	Number	Percent of Total
State Peace Officer Standards and Training Board	30	4.5
State police/highway patrol	41	6.2
Sheriff’s office	66	9.9
County police	22	3.3
Municipal police	132	19.9
Four-year college/university	43	6.5
Two-year college	221	33.3
Technical school	43	6.5
Special jurisdiction	17	2.6
Multi-agency/regional	49	7.4

The small increase in effort to conduct a census over a sample will allow BJS to report on training academies in all 50 states. Having data from all academies would increase the audience for publications that will be used to disseminate the information collected by this effort.

Procedures for Collecting Information

CLETA is designed as a multi-mode data collection using web as the primary mode with a hard copy survey instrument as an alternative for respondents who prefer that mode and a telephone option 15 weeks into the field period. The data collection and nonresponse follow-up period will last approximately six months and will include an initial invitation by mail and email, five reminders by mail and e-mail, an end-of-study letter, and a thank you letter. There will be data quality follow-up and non-response follow-up by e-mail and telephone. A brief description of the contacts is also provided.

Pre-notification letter. The letter (Attachment 5), on BJS letterhead, will be sent to all respondents and highlight the importance of the 2018 CLETA and encourage participation. It also provides contact information that can be used to obtain additional information about the 2018 CLETA or get assistance to complete the questionnaire.

Invitation package and email message. Two weeks after the pre-notification letter is sent, RTI will mail an invitation cover letter (Attachment 6) to the director/designee of all eligible academies. The letter stresses the purpose and importance of CLETA and the need for cooperation. It also notifies the recipient of the survey due date, provides instructions (including the web address and the academy password) for submitting the survey online, and provides an RTI contact if they have any questions or comments. The invitation package includes a flyer providing a brief history of the CLETA (Attachment 7), a summary of the information collected, the survey schedule, and contacts for additional information. An endorsement letter from the International Association of Directors of Law Enforcement Standards and Training (IADLEST) (Attachment 8) will also be included in the information package.

In addition, an email invitation (Attachment 9) will be sent to those directors/designees for whom an email address is available less than one week after the mailed invitation letter is sent. The email is aligned with the mailed invitation letter and contains a hyperlink to the web survey, which makes it easier for a respondent to just click the link and get to the survey, as well as instructions for submitting the survey online.

Mail and email reminders. Ten days after the email invitation is sent, we will mail the first reminder (Attachment 10) to nonrespondents using the United States Postal Service (USPS). Over the course of approximately three months, a total of two USPS mailed letters and one email reminders (Attachments 11-13) will be sent to those who have not yet responded. Both the mail and email reminders contain information for completing the web survey. However, as we anticipate some might prefer to respond using a hard copy questionnaire, the fourth mailed reminder will contain a paper questionnaire and a business reply envelope for easy return.

Telephone and email data quality follow-up. Within two weeks of receiving a completed survey, RTI will begin data review. During review, if data discrepancies or missing data values are discovered, RTI staff will follow up with respondents (Attachment 14) via telephone or email to clarify responses or obtain missing information.

Telephone and nonresponse follow-up. In week 15 of the data collection period, phone follow-up with nonrespondents (Attachment 15) will begin. Up to five call attempts will be made for each academy before the case is coded as “Maximum call attempts reached.” An attempt is defined as a call where an interviewer talks to a person at the academy or leaves a message on the academy’s answering machine. If a contact attempt is successful, the respondent will be reminded of the purpose and importance of the survey and informed of the goal of receiving a completed survey from each academy. The telephone interviewer will reference the most recent communication (e.g., reminder letters, reminder emails, etc.) in the introduction of the phone call to determine if they have received any of the communications sent to them. Those who did not receive any of the messages or the questionnaire packet will be assisted by the interviewer in getting the information they need to complete the survey. For those who received the communications or the questionnaire packet, the interviewer will determine why they have not yet completed the survey and try to gain cooperation and offer assistance. Respondents who agree to complete the full survey will be asked to submit the survey online but will be sent another hard copy version of the survey if requested. Those who are hesitant will be asked to consider providing responses to the most critical items. The interviewers will be prepared to collect responses to the critical items during the phone call or to schedule for a more convenient time. Critical item nonresponse follow-up will also include one follow by USPS mail for those with non-P.O. Box address or email for those with only a P.O. Box address (Attachments 16-17). The 2018 CLETA critical items are highlighted in Attachment 1.

Mail end-of-study notification. We will send an end-of-study notification by mail (Attachment 18) via USPS to notify nonrespondents that the study is coming to an end and that their response is needed within two weeks. Data collection will continue for approximately three more weeks to allow for receipt of any remaining questionnaires.

Mail letter of appreciation. We will send a thank you letter (Attachment 19) by mail via USPS to all respondents who complete the survey. The letter will be sent shortly after the completed survey is received indicating the responses have been received and are being processed. The letter also informs the respondent that they may be contacted again by RTI to confirm or clarify responses.

Table 4. 2018 CLETA Contact Schedule

Week	Contact description	Attachment
1	Mail pre-notification	5
3	Mail survey invitation, flyer, and endorsement letter	6,7,8
4	Email survey invitation	9
5	Mail 1st reminder	10
9	Mail 2nd reminder	11
11	Email 3rd reminder	12
12	Mail 4th reminder, with questionnaire and business reply envelope	13
	Start telephone/email data quality follow-up	14
15	Start telephone/email non-response follow-up	15
17	Mail 5th reminder via UPS – non-P.O. Box addresses	16
17	Email 5th reminder to those with only P.O. Box address	17
20	Mail end-of-study notification	18
	Thank you letter	19

Data Editing. As part of the data quality follow-up, RTI will attempt to reconcile missing or erroneous data through automated and manual edits of each questionnaire within two weeks of completion. In collaboration with BJS, RTI will develop a set of edits that will use other data provided by the respondent on the survey instrument to confirm acceptable responses or identify possible errors due to missing or inconsistent data elements. For example, if a screening question was left blank, but the follow-up questions were completed, a manual edit would be made to indicate the intended positive response to the screening question. Through this process, RTI can quickly identify which hard copy questionnaires require follow-up and indicate the items that need clarification or retrieval from the respondent.

Data Retrieval. When errors due to missing or inconsistent data elements are found during data review and editing, attempts to verify or collect the correct information with the respondent will occur. When it is determined that data retrieval is needed, an Agency Liaison (AL) will contact the respondent for clarification. Throughout the data retrieval process, RTI will document the questions needing retrieval (e.g. missing or inconsistent data elements) then request clarification on the provided information, obtain values for missing data elements, and discuss any other issues related to the respondent’s submission.

Data Entry. Respondents completing the survey via the web instrument will enter their responses directly into the online instrument. For those respondents returning the survey via hardcopy (mail or fax), the survey will be scanned in once received and the data will be extracted from the TeleForm. To confirm that editing rules are being followed, RTI will review frequencies for the entered data after the first 10 percent of cases are received. Any anomalies, inconsistencies, or unexpected values will be investigated and resolved. Throughout the remainder of the data collection period, RTI and BJS staff will conduct regular data frequency reviews to evaluate the quality and completeness of data captured in both the web and hard copy modes. Data files will be made available to BJS via an SFTP site when response rates reach 50%, 75%, and 90%.

Methods to Maximize Response Rates

The previous iterations of CLETA have achieved very high rates of survey response (between 90% and 100%). As described in the previous section, BJS and RTI will undertake various activities to ensure that high response rates are again achieved for the 2018 CLETA.

Prior CLETA administrations enjoyed widespread support among national law enforcement professional associations, and this continues to be case for 2018 project. IADLEST, which has supported previous CLETA administrations, is endorsing the data collection and strongly encouraging participation of its membership.

The survey instrument was reviewed to ensure the collection of the most pertinent information, removing any unnecessary questions to reduce burden. BJS staff conducted a review of the open-ended responses to “other, specify” fields in the 2013 data to determine if the option could be removed to reduce burden . The result was the removal of the “other, specify” options from six items. As discussed, the questionnaire was also reviewed by BJS and RTI staff for ease of use, flow, and additional survey methodology best practices to ensure ease in administration.

To promote 100% item completion by respondents, RTI will monitor item response rates as surveys are submitted. RTI uses a survey management system linked to the web-based application that flags missing items and invalid responses. RTI also flags missing items on hard copy submissions on a flow basis. The data collection manager oversees phone and e-mail outreach to respondents to clarify missing or invalid responses and to take corrective action. Any changes to survey responses obtained through this follow-up effort will be tracked and entered in the data collection database.

Final Testing of Procedures

The proposed questions in the 2018 CLETA data collection instrument were all retained from the 2013 version. Revisions made were reviewed by BJS and RTI staff, and cognitively tested.

The cognitive interviews included five participating academies and were conducted in June 2018 by two trained cognitive interviewers from RTI. The purpose of the cognitive interviews was to identify potential problems and improvements to the 2018 CLETA questionnaire. In July of 2018, RTI provided the Cognitive Interviewing Findings and Recommendations report (Attachment 20).

In addition, RTI has conducted thorough testing of the web-based survey administration system through systematic user testing, including testing skip patterns, attempting to “break” the instrument, and back-end data checks on entered responses.

The 2018 CLETA maintains similar respondent recruitment and support procedures as previous CLETA iterations, which have been field tested and successfully employed. As noted above, response rates for previous iterations of CLETA have typically approached 100%. BJS and RTI will work diligently to maintain high response rates for the 2018 CLETA.

RTI has utilized web-based survey instruments that are substantially similar to the format in design for the 2018 CLETA in the recent LEMAS and CSLLEA administrations. The web-based survey administration procedures successfully employed in the LEMAS and CSLLEA information collections are followed to ensure the successful administration of the 2018 CLETA information collection.