OMB Application for NCCS Part B_revised

Office of Management and Budget Clearance Package Supporting Statement

National Survey on Correctional Contraband (NSCC)

PART B. STATISTICAL METHODS

The National Institute of Justice (NIJ) has contracted with the Urban Institute (Urban) to collect information on contraband and contraband interdiction modalities across local and state correctional facilities through a web-based survey, entitled “National Survey on Correctional Contraband (NSCC).” The NSCC will be the first ever systematic data collection effort on contraband-related issues in the United States. There exist no empirical data of national or regional scope that can shed light on the prevalence and management of correctional contraband. Although most local and state correctional systems maintain administrative data on contraband, the availability and quality of such data are largely unknown. This survey primarily aims to compile administrative data on contraband and contraband interdiction modalities to develop a systematic, comprehensive understanding of the prevalence and management of correctional contraband.

1. Sampling Universe, Sampling Methods, and Expected Response Rates

1. Sampling Universe

The NSCC will be administered at the correctional facility level. There are two main types of correctional facilities – prisons and jails. Prisons are longer-term facilities run by the state that typically hold felons and persons with sentences of more than one year. Jails are locally operated short-term facilities that hold individuals who are waiting for trial, as well as individuals sentenced to a term of, usually, less than one year. Prisons and jails mainly differ in terms of governing authority, as well as the severity of crime for which respective inmates are sentenced to incarceration. In addition, prisons and jails are likely to have different architectural designs and features. Prisons are commonly characterized by heavy security features, such as tall perimeters topped with razor wire, visible guard towers, and heavy gates. In addition, they are often located in less populated, rural areas, while many jails occupy high-rise buildings in urban settings.

Given these functional and physical differences between prisons and jails, existing data collection efforts to understand correctional populations have traditionally examined prisons and jails separately. It is also likely that such differences between prisons and jails influence the type and scope of contraband and contraband interdiction modalities found in these facilities. The proposed survey will thus sample prisons and jails separately from the respective censuses developed by the Bureau of Justice Statistics – the Census of State and Federal Adult Correctional Facilities (OMB No. 1121-0147) and the Census of Jails (OMB No. 1211-0305).

The Census of State and Federal Adult Correctional Facilities, first fielded in 1974, reports facility-level data for state and federal prisons. Data obtained describe the characteristics of adult correctional facilities, such as conditions of confinement, crowding, workload, and facility functions (e.g., medical and mental health care). Several facilities in the Census of State and Federal Adult Correctional Facilities will be excluded from the NSCC sampling frame, including those operated by local authorities that are architecturally and functionally different than typical state-operated prisons, as well as federal facilities managed by the Bureau of Prisons (BOP). Rather, the sampling frame will focus on prisons operated by a state department of corrections or by a private company on behalf of the state correctional authority.

Similar to the Census of State and Federal Adult Correctional Facilities, the Census of Jails collects facility-level information for all jails in the United States, except for integrated jail and prison systems in Alaska, Connecticut, Delaware, Hawaii, Rhode Island, and Vermont. The use of these censuses will make it efficient to administer the proposed survey as there are data collection protocols and contacts already established as part of these censuses. Moreover, the census data provide rich baseline information about prisons and jails to be included in the NSCC. The availability of such information precludes duplicative data collection for the NSCC, which can be designed to focus on contraband-related issues.

As questions concerning contraband and contraband interdiction modalities have not received systematic attention in any of the existing data collections, the NSCC will complement a series of correctional data collections maintained by BJS.

2. Sample Selection

The breadth and quality of administrative data on contraband and contraband-related incidents, maintained by correctional facilities in the United States, are not well-known. Therefore, the NSCC aims to canvass a range of contraband-related issues while providing survey respondents with flexibility in how they provide requested information. Most survey questions offer a wide range of response options and allow survey respondents to choose multiple responses as applicable, as well as to provide further comments on how they capture and maintain administrative data on contraband. Therefore, statistical precision is not the highest priority for the NSCC sampling design. Given the exploratory nature of this project, it is critical to ensure that the NSCC sampling design capture a wide variety of correctional facilities and collect the most comprehensive information about contraband as practicable.

Table 1. Overview of data collection

Facility type	State correctional facilities (Prisons)	Local correctional facilities (Jails)
Roster	Census of State and Federal Adult Correctional Facility (Bureau of Justice Statistics)	Census of Jails (Bureau of Justice Statistics)
Sampling method	Stratified random sampling	Stratified random sampling
Strata	(1) region, (2) population size, and (3) percent inmates allowed to leave facility	(1) region, (2) staff size, and (3) percent pretrial defendants
Recruitment mode	Through state headquarters and individual facilities (n~300)	Through individual facilities (n~300)

As summarized in Table 1, the NSCC uses a stratified sampling design for prisons and jails. The primary strata of the NSCC sampling design are formed based on the size of facilities, the extent of inmate mobility, and region. The region and size of facilities are conventionally used for stratification in similar survey work. These stratifying factors capture how large of a facility or a population to monitor for contraband-related activity and how contraband varies across regions. The notion of inmate mobility assesses the extent to which inmates move in and out of the facility with exposure to risk of accessing contraband. For prisons, it is measured by the proportion of inmates allowed to leave the facility (e.g., work release). For jails, this information is captured by the proportion of pretrial defendants in the population, as these individuals tend to move in and out of the facility for court hearings.

Table 2 shows how these stratification factors are operationalized for the prison and jail samples based on the availability and distributional characteristics of relevant information in the prison and jail censuses, respectively.

Table 2. Stratification Scheme

	Prisons	Jails
Region	Central, Northern, Pacific, Southern, Western	Central, Northern, Pacific, Southern, and Western
Facility Size	Inmate size: Low (149 and below) Medium (150~999) High (1,000 or more)	Staff size: Low (99 or below) Medium (100-999) High (1,000 or more)
Inmate Mobility	Any inmates allowed to leave the facility: (Y/N)	Proportion of pretrial defendants to sentenced defendants: Low (up to 90%) Medium (90-280%) High (280% and above)

Following this stratification scheme, it would be necessary to contact approximately 600 facilities (300 each for prisons and jails) for the NSCC. The sample size calculations are summarized in Table 3, with different estimates for confidence levels and response rates.

Table 3. Sample Size Calculations

	Power	N0	N* (adjusted for finite population)	N*85­ (adjusted for 85% response rate)	N*70 (adjusted for 70% response rate)
Prison Sample	0.80	199	178	210	255
	0.85	227	200	236	286
Jail Sample	0.80	199	185	218	265
	0.85	227	209	246	299

We believe it is possible to achieve an 85 percent response rate or higher, based on Urban’s prior experience collecting survey data from criminal justice agencies. This is also consistent with the feedback we have received from the field through our pilot test and focus groups. However, because the NSCC is the first data collection of its kind on contraband and there is little guiding information for us to follow, we will instead apply more moderate assumptions. Establishing a conservative 70% response rate with 85% power using a 5%-level two-sided significance test, our power analysis suggests that we need to survey 286 prisons and 299 jails to yield a sufficient number of complete responses for the NSCC.

Table 4. Stratified Sampling of Prisons (n=286)

	No Inmates Allowed to Leave Facility			Some Inmates Allowed to Leave Facility
	Inmate Size			Inmate Size
Region	Small	Medium	Large	Small	Medium	Large
Central	17	3	1	1	10	22
Northern	16	7	2	2	10	14
Pacific	9	3	0	8	5	11
Southern	20	12	0	7	24	27
Western	14	5	1	2	16	17

Tables 4 and 5 show the distribution of prisons and jails by the stratification factors. To maintain a steady sampling fraction throughout the prison and jail populations, proportional allocation is used for identifying the allocation within each stratum. In the absence of a priori knowledge about how the cost of sampling may differ from stratum to stratum, it is reasonable to apply proportional allocation, which produces a sample size that is representative of the size of the stratum within the population.

Table 5. Stratified Sampling of Jails (n=299)

	Proportion of Pretrial Population: Low			Proportion of Pretrial Population: Medium			Proportion of Pretrial Population: High
	Correctional Staff Size			Correctional Staff Size			Correctional Staff Size
Region	Small	Medium	Large	Small	Medium	Large	Small	Medium	Large
Central	7	16	7	7	11	5	8	11	8
Northern	0	2	5	0	2	8	0	0	5
Pacific	4	4	2	2	3	7	1	1	4
Southern	10	13	14	5	10	12	5	14	15
Western	10	5	2	10	10	7	8	11	8

2. Procedures for the Collection of Information
   
   

As the data collection agent, Urban will provide day-to-day operations and management of the survey and will collect various paradata (e.g., respondent response mode, the time of survey completion, the time interval between respondent access to the survey and completion of the survey, etc.). This will allow NIJ to better evaluate the success of the online data collection effort and help reconcile potential inconsistencies in survey responses.

The project team will reach out to the sampled facilities and their headquarters, if necessary, to provide a full package of the NSCC materials both via mail and electronically. The package will include an introductory brochure of the NSCC, endorsement letters from professional organizations—e.g., the American Correctional Association (ACA), the Association of State Correctional Administrators (ASCA), and the American Jail Association (AJA)—to encourage survey participation, the survey questionnaire, and detailed instructions on how to complete the survey. The primary mode of data collection will be an online self-administered survey, but those who have technical difficulty will be given the option to complete the paper questionnaire and return it by mail.

The project team has a great deal of experience in collecting survey data from criminal justice agencies and achieving high response rates. Along with recommendations made by Dillman and colleagues (Dillman, Smyth, & Christian, 2009), this prior experience will guide Urban’s approach to data collection and non-response follow-up. Urban staff will use the five contacts outlined by Dillman et al. (i.e., a pre-notification letter, initial survey mailing, thank-you/reminder postcard, replacement survey, and telephone calls) to recruit the sample of facilities. As needed, the survey team will also supplement those contacts with extra follow-ups to improve response rates. After initial contact with a facility, Urban staff will request that the warden or chief executive of the facility assign an individual to help assure completion of the survey. Project staff will track when the survey respondents open and complete a data entry session on the survey website and periodically follow up with those who remain inactive for an extended period of time or do not open a session. Further, project staff will review survey responses and consolidate invalid, redundant, or anomalous responses as needed.

3. Methods to Maximize Response rates and Deal with Issues of Non-response
   
   

Project staff recognize the importance of maximizing response rates and collecting high-quality data. Urban has a track record of working with criminal justice agencies to collect information on their operations and caseloads. Urban’s prior work of similar scope and complexity has typically achieved a 90 percent response rate or higher (e.g., Law Enforcement Management and Administrative Statistics 2013). Although the primary mode of data collection is an online self-administered survey, the project team will offer other means of data collection to maximize response rates. For example, survey respondents will be given the option to complete the survey on paper and return it via email, mail, or fax, or will be allowed to complete the survey by telephone with a member of the project team.

The online survey will be programmed in an intuitive manner to facilitate ease of completion, and the project team will follow up with survey respondents via letters, e-mails, and telephone prompts, whichever line of communication is best for a given agency. In addition, the NSCC itself offers much flexibility in terms of how survey respondents provide an answer to survey items. For example, respondents can describe how they collect data on contraband recoveries and provide data in a manner that is consistent with their own records/data systems. By offering respondents a range of response options, we hope to alleviate their concerns and ensure they feel the survey is applicable and relevant to them.

For the NSCC, the project team will also draw on ACA’s strong networks with correctional agencies and leaders around the country to ensure that the invited agencies complete the survey in a timely manner with as few missing items as possible. With respect to how to handle non-response or missing items, the characteristics of the sampled facilities and the facilities that participated in the NSCC will first be compared on the basis of existing data on facility characteristics. In so doing, the non-participation process can be modeled and turned into the predicted probabilities of survey participation. Analytic weights will then be constructed from these probabilities to adjust for the extent to which certain facilities are systematically more likely than others to be dropped from the survey. As the validity of this approach partly relies on the adequacy of the regression model of survey participation, project staff will carefully explore all prognostic variables associated with survey non-response.

Second, multiple imputation (MI) procedures will be applied to the surveys that are only partially completed to replace each missing value with a set of plausible values. Multiple imputation has several advantages over other missing data approaches, the chief of which involves filling in the missing values multiple times, which makes it possible to consider the extent of uncertainty involved in imputing missing values and yield credible standard errors (Schafer and Graham, 2002).

4. Pre-testing of procedures and methods

To minimize the length and complexity of the survey, NIJ and Urban have completed several rounds of reviews, vetted the survey and survey questions in multiple focus groups with prison and jail administrators, solicited thorough feedback from field experts in organizations like ACA, ASCA, and AJA, and completed a pilot test of the instrument in nine facilities. Only those items of direct relevance and deemed critical by the project team were kept in the survey. The team assessed the level of effort necessary to complete the survey during the pilot test of the instrument in 2017 (attachment J contains the Pilot Report).

The key elements of the pilot testing and results from the test are summarized here. The test used a convenience sample based on recommendations from the ACA and correctional administrators who participated in previous focus groups to discuss early drafts of the survey instrument. Pilot test participants were asked to complete and return the appropriate survey, based on their position within the correctional facility, and to answer a series of debrief questions by email. In total, nine participants, including correctional administrators from three jails and six prisons, completed the NSCC.

From the pretest, NIJ and Urban confirmed that correctional administrators were able to access the data requested within a reasonable amount of time. Participants of the pretest also provided valuable feedback that was incorporated into the survey instrument. For example, duplicative questions were removed from the survey and questions that were confusing or ambiguous were clarified. Overall, the pretest has served to lessen the respondent data collection burden.

5. Individuals or contractors responsible for statistical aspects of the design
   
   

Individuals to contact for information on statistical methodology, conducting the survey, and analyzing the data:

Bryce Peterson KiDeuk Kim

Senior Research Associate Senior Fellow

The Urban Institute The Urban Institute

2100 M Street NW. 2100 M Street NW.

Washington, DC Washington, DC

Phone: 202-261-5802 Phone: 202-261-5346

Email: [email protected] Email: [email protected]

Jack Harne

Physical Scientist

National Institute of Justice

810 7th Street NW

Washington, DC 20531

Phone: 202-598-9412

Email: [email protected]

References

Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). New York: John Wiley & Sons.

Schafer, J.L. & Graham, J.W. (2002). Missing Data: Our View of the State of the Art. Psychological Methods, 7, 147-177

7