2014 NSP Supporting Statement Part B 7312014

Part B. Collection of Information Employing Statistical Methods

1. Universe and Respondent Selection

The 2014 National Survey of Prosecutors (NSP-14) will collect data on prosecutorial activities from all prosecutor offices who handle felony cases in state courts throughout the nation. The last census of this scale was conducted by BJS in 2007. Since that time, it should be expected that much has changed within the nation’s prosecutor offices, mostly attributed to the economic downturn. Offices are inclined to do more work with fewer resources. Additionally, crime trends continue to evolve with the advancement of technology (e.g., identity theft) thereby increasing the need for advanced training and new resources. In order to make well-informed decisions on the allocation of federal funding and staffing levels, it is imperative that agencies have access to updated, representative data regarding the current status of the nation’s prosecutor offices. The National Survey of Prosecutors is a unique data collection effort in that it serves as a point of comparison to better gauge changes in budgeting, staffing, caseloads, and resources. Government agencies, state prosecutor offices, and academic institutions rely on updated information to better provide recommendations for program planning and resource management.

This survey will build on previous surveys of the nation’s state prosecutor offices’ organizational characteristics. Emphasis will be placed on collecting information pertaining to staffing, salaries, and budget as well as measures of emerging prosecution techniques and issues such as computer and internet related crimes, human trafficking, and the use of forensic and DNA evidence. The most recent data available from this population is from the 2007 National Census of State Court Prosecutors (NCSP). The reference period for this project will be January 1, 2014 through December 31, 2014.

The universe for this survey will be all prosecutorial districts that handle felony cases in the U.S. BJS provided NORC with an electronic file containing 2,333 prosecutorial district offices with prosecutor’s identification and district office information. The list is an updated version of the 2007 NCSP frame based on the National District Attorney's Association (NDAA) 2007 Directory. Specifically, this list contains reasonably current mailing information for each chief prosecutor including prosecutorial district name and ID, contact person name and title, mailing address, phone number, available e-mail address, and available fax number. During data collection, NORC will verify and update the names, addresses, and appropriate contact information in the frame.

In general, each prosecutorial district office covers one or multiple unique counties. One exception is that Alabama has two Judicial Circuit offices that divide one county (Jefferson County): AL 10th Judicial Circuit-Bessemer and AL 10th Judicial Circuit-Birmingham. The other exceptions occur in Texas where both the Kerr and Kimble prosecutorial district offices have jurisdiction over Kerr County; the Pecos County District #1 and #2 offices have jurisdiction over Pecos and Reagan Counties; and the Tom Green County District #1 and #2 offices have jurisdiction over Tom Green County. We also note that Alaska, Connecticut, District of Columbia, Delaware, and Rhode Island each have just one prosecutorial office in the frame.

At the request of BJS, NORC originally developed and evaluated several sampling plans with various sample sizes to conduct a sample survey of state prosecutor offices. BJS requested that NORC present two potential solutions: (1) a recommended sample size that would reach the ideal level of power and precision, and (2) the total number of completes the current budget could accommodate. A revised sampling plan was delivered to BJS, with the recommendation to proceed with a sample size of 658. After evaluating the power of each sampling plan and the need for another census, it was decided to forgo a sample survey in favor of the census. After a thorough cost/benefit analysis, it was concluded that the relatively greater cost of doing a census was outweighed by the value of having a census establish sampling frames for future BJS efforts and the greater confidence in the national statistics that result from a census.

An additional advantage to conducting a census is that all Public Law 280 (PL 280) districts will be included in the respondent universe for both mandatory PL 280 and non-mandatory PL 280 states. Identifying the districts which serve tribal lands will allow BJS to target State and local prosecutor offices that handle cases arising on tribal lands in a future survey. Tribe’s located in the 15 mandatory and optional PL 280 States account for about 400 of the total 565 federally recognized tribes across the U.S. Although a State may be subject to PL 280 mandates, not all counties or prosecutor offices within will be in close proximity to tribal lands or have matters filed in their jurisdictions¹.

A census of prosecutors will also provide BJS with the opportunity to respond to the need for juvenile transfer statistics and white collar crime statistics. For example, BJS will gather information from local prosecutor offices on the number and type of juvenile transfers in 2014. This information on juvenile transfers gathered in the NSP-14 will also help account for nonresponse bias and the creation of national estimates on a transfer study planned by BJS. Similarly, BJS will be able to identify the number and types of white collar crimes across the 2300 prosecutorial districts using the NSP-14. BJS gathers information on federal and state level white collar crimes, but lack data on the prevalence of local cases prosecuted

2. Procedures for Collection Information

Data collection for the NSP-14 will involve a series of mailings and non-response follow-up activities, emphasizing questionnaire completion via a secure web-based reporting system. At first, all respondents will be mailed a pre-notification letter (see attachment 2) announcing the start of the NSP-14 data collection, requesting that the office begin gathering the necessary data to respond. This letter will provide the link to the web survey, the respondent’s unique login information, and a notification that the web survey is available for use and early reporting. A brochure (see attachment 3) will also be included with the pre-notification letter. The brochure lists the types of questions that are asked in the web survey to help respondents prepare for its completion.

Approximately one week after the pre-notification letter mailing, NORC will mail a survey invitation letter (see attachment 4) to all respondents who did not complete the web survey in response to the pre-notification letter. This letter will convey the importance of the office’s participation in the NSP-14 and their timely submission. As was done for 2001 NSP, this mailing will contain a letter from the Director of BJS and a letter of support from the Association of Prosecuting Attorneys (APA) (see attachments 5 and 6). The letter will also include instructions for accessing the web questionnaire and the office’s unique PIN and password. Unlike in previous rounds, the NSP-14 invitation letter will provide the URL to complete the web survey but will not include a hardcopy instrument. This will be done to increase the proportion of respondents using the on-line reporting system. Respondents will however be able to receive a hardcopy questionnaire if they prefer by contacting the project 1-800 number or e-mail address.

Approximately one month after the survey invitation letter, NORC will send a reminder postcard (see attachment 7) to all non-responding agencies. This postcard will continue to encourage non-responders to complete the questionnaire via the web. It will contain the NSP-14 toll-free number and e-mail address so that respondents can contact NORC with questions or if they need assistance.

Approximately two months after the survey invitation mailing, NORC will mail a questionnaire packet via first-class U.S. mail. The questionnaire packet will contain an introductory cover letter from NORC (see attachment 8), a personalized hardcopy questionnaire (see attachment 1), and a pre-paid business-reply envelope (BRE). The cover letter will again convey the importance of the NSP-14 and the office’s participation. It will include instructions for submitting the completed questionnaire via the web, as well as mailing the completed questionnaire back to NORC in the enclosed BRE, faxing each page of the questionnaire to NORC, or e-mailing a scanned copy of the questionnaire to NORC.

The data collection agent will then begin telephoning non-respondents approximately one week after the questionnaire packet is mailed. This will serve to verify receipt of materials, answer questions, determine and attempt to resolve potential problems with timely submission, and prompt for questionnaire completion. Data collection specialists may also offer respondents the option to complete the survey over the phone if the information requested is readily available and the respondent refuses to complete the online or paper questionnaire. Surveys completed over the phone will be recorded by data collection specialists using the preprogrammed web survey.

While telephone prompting continues, the data collection agent will send fax and e-mail reminders to all remaining non-responders. The fax and e-mail reminders will provide a different form of contact to gain the respondent’s attention, stress the importance of the study, and prompt for return of the questionnaire (see attachments 9 and 10). The customized e-mail will contain the web survey URL and respondent’s PIN and password. Moreover, while phoning, faxing, and emailing non-respondents, the data collection agent will send a second replacement questionnaire to each non-responder via priority mail to further convey the importance of timely data collection. Sending communication via high speed mail service communicates a sense of importance and urgency in receiving a response. In addition to follow-up contacts to non-respondents made by NORC, the APA may reach out to hard-to-reach respondents to encourage completion of the survey in a timely manner.

During the final weeks of data collection, the data collection agent will mail a postcard or send an e-mail alerting respondents of the scheduled data collection end date. A last chance contact has been implemented on previous studies, and it serves to motivate non-responders who had failed to complete and return the survey. This contact will be tailored to urge completion of the web survey for any respondents that completed a portion of on-line instrument and then broke off.

Verifying and validating the submitted data

Upon receipt, each questionnaire will be reviewed and edited, and, if needed, the data provider will be contacted to clarify responses or provide missing information. Additional follow-up contacts were accounted for in the submitted budget estimate. Prior to contacting the respondent jurisdiction, the data collection agent will attempt to address data inconsistencies via BJS-approved editing specifications. The data collection agent will also ensure that responses fall within the proper coding schemes specified by BJS. The following is a summary of the data quality assurance steps that will be implemented during the data collection and processing period:

Data editing: The data collection agent will aim to reconcile missing or erroneous data through a manual edit of each questionnaire. In collaboration with BJS, the data collection agent will develop a list of manual edits that can be completed by referring to other data provided by the respondent on the survey instrument. For example, if a screening question was left blank by the respondent but the follow-up questions were completed, a manual edit could be made to indicate the intended positive response to the screening question. Through this process, the number of cases requiring clarification or retrieval of missing information can be minimized.

Data Retrieval: When it is determined that additional data are needed because of errors, inconsistencies, or missing data, the data collection agent will contact the data provider for clarification. Throughout the process, the data collection agent will document the questions needing retrieval (e.g. missing or inconsistent data elements), request clarification on the provided information, obtain values for missing data elements and examine any other issues related to the respondent jurisdiction’s submission.

Data entry: Respondents completing the survey via the web survey will enter their systems’ responses directly into the on-online instrument. For those respondents returning the survey via hardcopy (mail or fax), data will be entered as responses are received. Completed hardcopy surveys will be edited; respondents will be contacted for missing or inconsistent critical items and then data entered. Once the data has been entered into the database, it will be made available to BJS via an SFTP site. To confirm that editing rules are being followed, the data collection agent will review frequencies for the entered data after the first 10 percent of cases are received and processed. All issues will be investigated and resolved. Throughout the remainder of the data collection period, the data collection agent will conduct regular data frequency reviews and logic checks to evaluate the quality and completeness of data received.

Adjusting for Non-Response

In the most recent NSP during 2007, the overall response rate was 96% among the 2330 prosecutor offices (Table 1). About 56 (2.4%) of all eligible 2330 prosecutors’ offices completely refused participation or did not return the entire survey. Although we are confident the abovementioned steps will result in participation from the large majority of respondents, there will always be a small percentage of jurisdictions that refuse to complete the survey for various reasons. For these respondents, we will compile an instrument that seeks to capture data for only critical items. Although not ideal, reducing the number of items and respondent burden will allow for collection of the most important data items from those respondents who previously could not complete the entire survey because of time and/or reporting constraints. Critical items for the NSP-14 will include information on staffing, expenditures, budgets, caseloads, juvenile justice transfers, and jurisdiction for tribal lands. Capturing critical items will enable BJS to conduct trend analysis on historical items and determine the universe for additional targeted surveys in the future.

Table 1. 2007 NSP Response, Contact and Refusal Rate.

	Count	Eligible		Rate
Overall Response Rate	2227	2330	=	95.6%
Overall Contact Rate	2283	2330	=	98%
Overall Refusal Rate	56	2330	=	2.4%

The overall average item nonresponse rate for the NSP 2007 was 14% including complete refusals and partial completed surveys. The item nonresponse rate varied by question and had a range between 8% and 26% for the entire survey. Several items with item nonresponse rates over 20%, like assistant prosecutor salaries, were modified or removed from the proposed NSP-14. Among the critical survey items replicated from the NSP-07, the item response rate for staffing (89% or more), budgets (84%), and caseloads 75% or more), and jurisdiction for tribal lands (85%) had a range of 75% to 89%. Items such as felony caseloads, convictions and jury verdicts have generally been sensitive to nonresponse in the NSP, therefore, BJS will also impute missing information.

If necessary, BJS will employ imputation or weighting adjustments to reduce non-response bias in the estimates. While high item response is anticipated, respondents may leave items blank due to lack of access to the information or the fact that the data are not being recorded by the prosecutor’s records management system. We will utilize imputation for the purpose of reducing bias due to item non-response, ensuring consistent estimates, and simplifying data analysis. In this effort regression imputation is particularly appealing because it allows us to leverage information such as population served that might be correlated with other key characteristics including defense expenditures, caseload size, and other important information. Regression models will be built to estimate missing information by taking advantage of certain variables that should be available for the majority of prosecutor offices. Once all missing values have been imputed, the dataset can then be analyzed using standard techniques for complete data.

Although we prefer the imputation approach to take advantage of the existence of historical data, traditional non-response weighting adjustments will be used to compensate for unit non-response. The first step under the weighting approach involves computing the sampling weight or base weight, defined as the inverse of the selection probability. Since we are conducting a census of prosecutor offices, the base weight will be one for all jurisdictions. The second step adjusts the base weight for non-response through a weighting class or response propensity-based adjustment to compensate for the non-respondents. For example, we will group similar jurisdictions (in terms of size, region, service delivery structure, etc.) together to form weighting classes and adjust the weights of respondents so that they represent the non-respondents within the same weighting class. Alternatively, we could construct a multivariate model to predict the response propensity and use the predicted propensity score to adjust the base weights. The final weighting procedures will be determined empirically based on a nonresponse bias analysis to identify variables that influence nonresponse.

3. Methods to Maximize Response Rates

As is standard for BJS surveys, the response rate expected is 90 percent or more. The 2001 NSP yielded a response rate of 95 percent and the 2007 NCSP reached a 96 percent response among all respondents.

As described in the data collection section, the data collection agent will utilize a variety of techniques to encourage high response rates. In particular, they will provide multiple response options to indigent defense systems, including a web-based questionnaire and the option to complete and return the survey form via mail, fax or to complete it with a telephone interviewer. The follow-up plan utilizes best practices and is comprised of letters, e-mails, faxes, and telephone prompts. The web-based questionnaire will be programmed in an intuitive manner to facilitate ease of completion, including clearly formatted questions and displays, automatic machine checks and skip patterns, drop down menus with pre-programmed responses, and available help-screens to assist respondents if questions arise. Mailing attempts will include a letter of support from APA which will convey the importance of the data collection and the need for response. Additionally, the APA will reach out to selected offices to prompt for their completion and return of the survey. NORC will also monitor a project specific e-mail address and toll-free number, both of which allow respondents to contact NORC with any questions.

4. Initial Pilot Test of Procedures

Pilot Test Methodology

The Association of Prosecuting Attorneys (APA) provided NORC recommendations for potential field test respondents. The APA recommended nine offices. These offices were located in Nebraska, Virginia, Vermont, Wisconsin, Minnesota, Oregon, New York, and Virginia. As per the Paperwork Reduction Act of 1980, a pilot survey of 9 or fewer people does not require OMB approval. NORC routinely contacts nine field test respondents for recurring data collection efforts where the overall methodology and survey instrument have not changed substantively, including the National Survey of Prosecutors and the Census of State and Local Law Enforcement Agencies. The procedures and methods for the NSP-14 were in alignment with earlier data collection efforts and therefore nine respondents were deemed sufficient.

Procedures

NORC sent field test packets to the nine offices via FedEx standard overnight delivery. Included within this packet was a cover letter from NORC, a letter of support from the APA, and a draft copy of the survey instrument as well as a return FedEx envelope for returning the completed survey. The NORC cover letter described the objective of the field test (see Attachment 15), provided instructions on how to return the survey, and requested that all offices return their survey within two weeks of receipt.

Follow-up

NORC made follow-up calls to all pending offices to confirm receipt of the FedEx packet and to answer any initial questions. A second round of follow-up calls was made to pending offices to encourage the offices to return the completed survey. A final round of follow-up calls were made informing the office that pilot data collection was ending and that all input would be greatly appreciated.

Debriefing

Upon receipt of a completed questionnaire, NORC contacted the respondent to schedule a debriefing interview. This interview included general questions about respondent burden, the readability of the survey, the availability of information, and if given the choice, would they have been willing to complete the survey on the web.

Pretest Test Results

NORC received completed surveys from all nine offices. No office noted any major difficulties with the questionnaire. Several respondents provided feedback to make some of the items more clear and concise. The average estimated time to collect the necessary information and complete the survey was 60 minutes. One respondent did not report a time but rather stated it was time consuming to collect all of the necessary information. The actual time for each office will be dependent on the size of the office and the resources available to locate and provide the requested information.

Recommendations

Based on the respondents’ feedback from the pilot test, several items were modified slightly to make them more clear and concise. Also, for questions where counts are requested, a “do not track” checkbox was added to allow respondents to select if they cannot provide specific counts for an item.

Respondents indicated that they would have preferred to know what type of information was necessary to complete the survey before beginning it. This prompted the creation of a brochure listing the general question topics that respondents can anticipate on the 2014 NSP. Once the brochure and pre-notification letter are received, respondents will have known in advance of the survey the type of information that will be requested.

Also, as a result of the pilot test results and to help ensure minimal nonresponse, BJS reduce the number of questions on the NSP-14 by half. The data collection agent subsequently submitted the NSP-14 to the Association of Prosecuting Attorneys (APA) to review the document to ensure the respondent burden was about 30 minutes per respondent for a total of 1,165 respondent burden hours. In addition to the estimated 30 minutes per respondent required to complete the NSP-14 form, BJS anticipates about 10% or 117 hours in additional burden will be needed for data validation and follow-up contact with the respondents for nonresponse, incorrect or missing information

5. Contacts for Statistical Aspects and Data Collection

The prosecution and adjudications staff at the Bureau of Justice Statistics, along with staff from NORC, takes responsibility for the overall design and management of the NSP-12 data collection, including the development of the questionnaire, sampling plan, and the analysis and publication of the data.

1. BJS contacts include

Steven W. Perry, Statistician

Prosecution and Adjudication Statistics Unit

U.S. Department of Justice

Bureau of Justice Statistics

810 Seventh Street, N.W.

Washington, D.C. 20531

(202) 307-0777

2. Persons consulted on data collection and analysis:

David Herda

Senior Survey Director II

NORC at the University of Chicago

55 East Monroe Street, 30^th floor

Chicago, IL 60603

3. Persons consulted on statistical methodology:

Edward Mulrow

Senior Statistician III

NORC at the University of Chicago

4350 East-West Highway, 8th Floor

Bethesda MD 20814

Attachments:

Attachment 1-NSP-14 Questionnaire

Attachment 2-NSP-14 Pre-notification Cover Letter

Attachment 3-NSP-14 Brochure

Attachment 4-NSP-14 Web Invitation Letter

Attachment 5-NSP-14 BJS Notification Letter

Attachment 6-NSP-14 APA Letter of Support

Attachment 7-NSP-14 Reminder Postcard Text

Attachment 8-NSP-14 Initial Questionnaire Cover Letter

Attachment 9-NSP-14 Fax Script

Attachment 10-NSP-14 Email Script

Attachment 11-NSP-14 Replacement Questionnaire Cover Letter

Attachment 12-NSP-14 Telephone Contacting Script

Attachment 13-NSP-14 Telephone Voicemail Script

Attachment 14-NSP-14 Telephone Retrieval Script

Attachment 15-NSP-14 Field Test Cover Letter

1 BJS has identified those states, pursuant to Public Law 83-280, where jurisdiction for serious crimes occurring on tribal lands and or Indian country has been transferred to State and local jurisdiction. Criminal jurisdiction -- the governmental powers to make and/or enforce laws -- in Indian country is divided among the Federal, State, and tribal governments. Jurisdiction in a specific incident depends on the nature of the offense, whether the offender or victim was a tribal member, and the State in which the crime occurred. (See Census of Tribal Justice Agencies, 2002. Bureau of Justice Statistics, http://www.bjs.gov/content/pub/ascii/ctjaic02.txt.) In PL 280 districts, many of the serious tribal crimes or cases are prosecuted by the locally elected district attorney or State’s Attorney.

Public Law 83-280 (commonly referred to as Public Law 280 or P.L. 280) conferred jurisdiction the Federal Government to six State governments. Congress extended criminal jurisdiction over tribal lands to California, Minnesota (except the Red Lake Reservation), Nebraska, Oregon (except the Warm Springs Reservation), Wisconsin, and Alaska. These are mandatory P.L. 280 States.

Public Law 280 also permitted other States to acquire jurisdiction at their option. These States could take partial jurisdiction until the 1968 amendment to the law. The nine optional P.L. 280 States assumed jurisdiction either in whole or in part over Indian country within their boundaries: Nevada, Idaho, Iowa, Washington, South Dakota, Montana, North Dakota, Arizona, and Utah operate under this arrangement. In States where P.L. 280 does not apply, the Federal Government retains criminal jurisdiction for major crimes. (See "Indian Country" Crimes Act (Title 18, United States Code, Section 1152), the Indian Country Major Crimes Act (Title 18, United States Code, Section 1153), and the Assimilative Crimes Act (Title 18, United States Code, Section 13.) The 1994 Crime Act expanded Federal criminal jurisdiction in Indian country in such areas as guns, violent juveniles, drugs, and domestic violence.

7