OMB Questions and Answers

Questions on the NIS-3 Supporting Statement:

1. When will the next administration of the Census of State and Federal Correctional Facilities be?  If the updates are only of closures, how did the universe size grow from the last administration of the NIS?

The trend the last few years has been for facilities to close rather than open. These populations were usually consolidated, increasing the overall count in the remaining facilities. Of the 171 prisons in the NIS-2 sample, 1 had closed; of the 320 jails in the sample, 7 had closed. These facilities were removed from the frame for the NIS-3.

As in the two previous iterations of the NIS, the federal facilities frame will be updated when the sample is drawn using the weekly population report which provides a listing of all facilities and population counts on a weekly basis (http://www.bop.gov/locations/weekly_report.jsp). BJS is currently in the planning stages to conduct the next administration of the Census of State and Federal Correctional facilities during 2011, which will provide a new sampling frame for the agency’s inmate surveys.

2. Please quantify and justify the components of sample size change (ie, both increase from new type of facilities and decrease from closed facilities) since last administration.

For prisons, 215 facilities will be selected. The base sample size of 170 facilities is the same as the previous administration of the survey. However, the sample size has been increased to accommodate two sub-populations of interest under PREA: inmates with mental health problems and youth held in adult facilities.

The Mentally Ill Offender Treatment and Crime Reduction Reauthorization and Improvement Act of 2008 (P.L.110-416) requires BJS to collect data on the mental health status of inmates. In addition, constituents associated with PREA, including the former National Prison Rape Elimination Commission and the Department of Justice Prison Rape Elimination Review Panel believe that mentally ill inmates may be at greater risk to sexual victimization. As a result, supplemental PREA funding allowed us to add 25 facilities to the sample to learn more about the mental health of inmates in general and about the possible correlation between mental health problems and sexual victimization.

There is also interest under PREA to learn more about juveniles in adult facilities. The prevailing belief is that youth held in adult facilities are more at risk to sexual victimization than other inmates, and at greater risk compared to youth held in juvenile facilities, but to date there is limited statistical evidence to support this. Additional PREA funding was allocated to include 16 and 17 year-olds in the NIS, adding 20 prisons and 20 jails that have a high concentration of juveniles to ensure a sufficient sample to general a national estimate of sexual victimization of youth held in adult facilities.

In all, 334 jail facilities will be selected. The base sample size of 305 facilities is the same as in the previous administration of the survey. We added 9 facilities that were unable to participate in the previous iteration due to space or logistical issues with certainty in the NIS-3. The remaining 20 facilities will be sampled according to their concentration of juveniles held.

A small decrease in the universe of facilities due to closures will not impact the sample size of facilities in this administration of the NIS.

3. Please clarify the sampling strata, including the interaction between facilities by gender, with or without mental health facilities, and with or without 16-17 year olds.

For prisons, the NIS sample design will have three strata, each with two levels. The strata are gender housed by facility, whether the facility has a mental health function, and whether the facility has a high level of 16-17 year olds. Thus, there are 8 stratum levels in the prison design. The 2005 Census of State and Federal Correctional Facilities will be used to assign facilities to a stratum. For facilities with or without 16-17 year olds, the total juvenile population reported in the Census is being used as a proxy (i.e., those facilities with a high total juvenile population are assumed to have a higher 16-17 year old population). Facilities where the Census indicates at least 10 juveniles being housed will be included in this stratum. The table below presents how the universe of prison facilities is distributed across these strata levels.

After accounting for the oversampling factor for facilities that house female inmates, the base sample of 170 facilities will be allocated proportionally across the 8 strata based on the number of inmates in each stratum. The additional 25 mental health facilities and 20 juvenile facilities will be allocated proportionally across those facilities that have a mental health function or high concentration of juveniles, respectively.

Distribution of prisons across strata level

Gender housed	Mental health function?	High level of 16-17 yr. olds?	No. of facilities on frame
Male	No	No	885
Male	Yes	No	95
Male	No	Yes	192
Male	Yes	Yes	42
Female	No	No	93
Female	Yes	No	17
Female	No	Yes	13
Female	Yes	Yes	16

For jails, the NIS sample design will stratify the universe of jails by whether they have a high concentration of juveniles. Similar to the prison design, jails that reported housing 10 or more juveniles on the Census of Local Jail Inmates will be assigned to the juvenile stratum. The base sample of 305 facilities will be allocated proportionally based on the number of inmates in each stratum. An additional 20 facilities will be added to the sample size for the high juvenile concentration stratum.

4. What is the representativeness of the data from the content that is only administered to those not reporting sexual victimization?  How will those results be used?

In order to help protect the confidentiality of all participating inmates, the NIS begins with core questions and then randomizes respondents to one of two survey instruments (see graphic on p.6). One survey instrument, allocated to 10% of respondents, receive a second core set of questions on mental health, disability, medical problems, and drug use, followed by modules with questions on facility climate, alcohol use and treatment. The other survey instrument, allocated to 90% of respondents, includes a core set questions on sexual victimization, facility climate, and mental health and disability questions. Since 95% of inmates receiving this instrument will not report sexual victimization they will finish relatively quickly; those with time remaining are split between two paths to pad the rest of the survey (one that prioritizes drug and alcohol use and treatment, and one that prioritizes medical problems).

These modules were tested in January in 3 facilities (with 377 inmates) and ordered according to research interests balanced by survey time constraints of an average of 35 minutes per respondent. The questions were divided into modules so we can use appropriate weighting techniques to maximize the use of the data (those reporting no sexual victimization will yield more information on the supplemental modules, but receive a different weight than those respondents assigned to the alternative instrument. Following a nonresponse bias analysis and applied weights, we will generate information on the non-sex questions that will be representative of all inmates in the United States. Specifically, we plan to use the data to estimate the prevalence of serious mental illness and medical issues and treatment among prison and jail inmates.

5. Is this the first use of PAPI instruments in the NIS?  If not, what were the experiences last time?

The PAPI NIS was first used during the NIS Pilot Study (conducted in 2006) and has been used in each of the first two years of the NIS.  Last year the NIS was administered to 726 inmates – less than 1% of the total number of interviews conducted.  The PAPI instrument contains only the most critical items from the ACASI NIS instrument: those that are needed to calculate the victimization rates, demographics, and items that provide useful context for the events reported.  A total of 34 items are included.  In order to make the questionnaire as simple as possible for inmates to complete, there is very limited use of skip routing.  

The PAPI instrument is only used with inmates who are unable to come to the private interviewing room to meet with the interviewer and complete the ACASI survey.  These inmates are typically those who are designated by the facility administrator as too dangerous to be transported within the facility or who would be a threat to the interviewer. However, the PAPI has also been used with inmates housed in a medical wing or who were temporarily being housed in disciplinary segregation for their own protection.  Once an inmate is determined to need the PAPI, the lead interviewer works with the facility to determine when an interviewer can meet with the inmate to obtain informed consent and administer the interview. Requirements regarding privacy of the interview setting are the same for PAPI interviews as for ACASI, namely the interviewer and inmate must be able to speak where they cannot be overheard and nobody can see the answers the inmate records on the questionnaire.  After administering the informed consent, the interviewer passes the PAPI and a pencil to the inmate to complete.  When the inmate is finished, the interviewer provides the inmate with an envelope in which to place the questionnaire and asks him/her to seal it.  The sealed envelope is passed back to the interviewer who then places a piece of tamper-resistant tape over the seal.  Completed PAPI questionnaires must remain in the possession of the interviewer for the remainder of the day in the facility and must be shipped via overnight delivery to RTI on the day they are completed. Interviewers may not leave completed questionnaires with facility staff to go out as part of the regular mail leaving the facility.

During interviewer debriefing sessions, interviewers have reported that inmates who completed the PAPI were very appreciative to be included in the study and appeared to take their task of completing the questionnaire seriously. We have also been fortunate that most facility administrators have worked with us to figure out ways to manage data collection in their facilities that has minimized the number of inmates who need to complete the PAPI because they are able to participate in the ACASI interview.  

6. What were the results of nonresponse bias analysis from earlier administrations?

To reduce bias due to nonresponse, a generalized exponential model (GEM) (Folsom & Singh, 2000) is applied to each participating facility. In each facility, bias could result if the random sample of inmates did not accurately represent the facility population or if the non-respondents were different from the respondents. Therefore, post-stratification and non-response weight adjustments were made to the data to compensate for these two possibilities. These adjustments included:

1. calibration of the weights of the responding inmates within each facility so that the estimates accurately reflected the facility's entire population in terms of known demographic characteristics. (These characteristics included distributions by inmate age, gender, race, date of admission, and sentence length.) This adjustment ensures that the estimates accurately reflect the entire population of the facility and not just the inmates who were randomly sampled.

2. calibration of the weights so that the weight from a non-responding inmate is assigned to a responding inmate with similar demographic characteristics. This adjustment ensures that the estimates accurately reflect the full sample, rather than only the inmates who responded.

By conducting these model adjustments at the facility level, the potential for bias is minimized for facility-level and national-level estimates.

However, despite the implementation of these weighting procedures, bias may remain. Therefore, nonresponse bias analyses were conducted for the first administration of NIS and are underway for the second administration to assess the extent to which bias remains. Analyses were conducted at the facility level as well as across all inmates. Bias was assessed across all demographic characteristics obtained on each facility’s roster of inmates. These characteristics included gender, age, race/ethnicity, sentence length, and time since admission. For each characteristic an effect size (Cohen, 1988) was calculated. In all cases the effect size was “small” as defined by Cohen (i.e., less than 0.10). We, therefore, concluded that the non-response adjustments we employed successfully minimized the bias in the NIS estimates.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum

R.E. Folsom, Jr., and A.C. Singh, (2002), "The Generalized Exponential Model for Sampling Weight Calibration for Extreme Values, Nonresponse, and Poststratification," Proceedings of the American Statistical Association, Section on Survey Research Methods, 598-603.

7. What is the nonresponse bias analysis plan for the next round of data collection?  We did not see one included in the package.

The nonresponse bias analysis plan for the next round of data collection will be similar to the plan for the first two administrations. As described in our response to Question #6, we plan to use GEM to reduce the bias at the facility-level. To verify whether these steps are effective, an effect size (Cohen, 1988) will be calculated for all available inmate characteristics on a facility’s roster. Since the NIS publishes both national- and facility-level estimates, a nonresponse bias analysis will be conducted at both the national-level and for each participating facility.

Questionnaire-related questions:

Please help us understand all of the substantive ways that the 90% and 10% sample questionnaires differ.  Besides the sexual violence items only on the 90% questionnaire, which sets of content are unique to one or the other?  Are any content areas covered differently by the two?

In order to help protect the confidentiality of all participating inmates, the NIS begins with Core 1 questions and then randomizes respondents to one of two survey instruments (see graphic on p.6). One instrument, allocated to 10% of respondents, receive a Core 2 set of questions on mental health, disability, medical problems, and drug use, followed by modules with questions on facility climate, alcohol use and treatment as time permits. The second survey instrument, allocated to 90% of respondents, includes a Core 3 set questions on sexual victimization, facility climate, and mental health and disability questions. Since 95% of inmates receiving this instrument will not report sexual victimization they will finish relatively quickly; those with time remaining are split between two paths to pad the rest of the survey (one that prioritizes drug and alcohol use and treatment, and one that prioritizes medical problems), allowing us to maximize the data we are able to use to generate national estimates.

The questions in the modules do not differ between paths, but the order in which the modules are randomized does. The graphic on the next page should clarify how the modules will be allocated to respondents -

The following numbering is based on the Questionnaire Specifications for the 90% sample, but some items apply to other instruments as well.

1. The introduction to question A2 says that “the first few questions are for statistical purposes only.”  We consider the entire questionnaire to be for statistical purposes only, as required by BJS statute.  Therefore, please delete or edit this statement.

We will delete this statement.

2. A13 does not meet OMB standards.  The question should indicate, “please select (or check) one or more.”

The respondent is asked “Which of these categories describes your race?” and then proceeds to answer each race category with a separate yes/no option displayed on the laptop screen. They select as many or as few races as they want.  If we need to be more explicit we could add, “Please answer yes or no for each race listed below” but to instruct them to “select one or more” indicates they can skip over some of the categories, which they cannot.

3. Did the veteran status questions change from what was field tested?  If so, please briefly describe.

The veteran status questions were previously revised with aid from a contact at the Office of Veteran’s Affairs, submitted for OMB approval for pretesting, and subsequently tested with the rest of the survey. There was no indication that these questions posed a challenge for respondents and thus, have remained the same for national implementation.

4. Please describe specific sources for the facility conditions questions as well as which were modified or developed specifically for the NIS.  Please also justify the different topical areas (eg, friends, visitors).  Specifically, is there something like a validated “facility climate” construct being applied here that is known to correlate with levels of violence or other outcome?

Measures of social climate will help us test whether victimization rates may be correlated with facility safety and perceptions of well-being. BJS has a history of including such items in the Survey of Inmates in State and Federal Correctional Facilities (SISCF), some of which are also in the NIS-3. Questions from the Bureau of Prisons’ Social Climate Survey (SCS) were also culled and adapted for ACASI administration. There is no specific construct to apply at this time; the final instrument incorporates various measures of quality of life in prison to allow analysis of how the outcomes correlate with individual and facility-levels of victimization.

Some questions, such as those on physical assault and types of injury, were in the previous version of the NIS (in Section X), but we added follow up questions to measure frequency and include physical assaults by facility staff. We also added some questions on general social ties designed to measure inmate support and security within the facility (S1-S5, S13-S16, S26, S30) and questions from the SISCF on contacts with friends and family as a measure of outside support (S10-S12). Questions S6 – S9 were adapted from the SCS to measure crowding, privacy, and attitudes toward staff.

Similar measures used in the National Survey of Youth in Custody, 2008-09 (NSYC) revealed a correlation between facilities in which youth reported greater dissatisfaction with staff and the environment and higher rates of sexual victimization.

5. Please describe specific sources for the questions on mental health, as well as which were modified or developed specifically for the NIS, and whether they have ever been applied to an incarcerated population before.  Please also map the questions to validated mental health constructs. 

The objective of the mental health questions in the NIS3 is to screen inmates to get an indication of the extent of mental health problems among the incarcerated populations. The mental health modules will only be used as screening tools; they cannot be used as diagnostic tools to either diagnose an inmate or to validate a diagnosis. A diagnosis would consist of a structured clinical interview that is conducted by a mental health professional. The mental health modules will also allow BJS to describe the characteristics of inmates with an indication of a mental health problem, compare their prevalence rates and characteristics with persons in the general population, and get an indication of the extent and types of treatment inmates may have received prior to and since their incarceration, as well as the continuity of treatment as they transitioned from the community to jail/prison.

The sources of the mental health questions included in the NIS3 are the National Survey on Drug Use and Health (NSDUH), the National Comorbidity Survey (NCS), National Health Interview Survey (NHIS), The World Mental Health (WHO) Composite International Diagnostic Interview (CIDI) and World Mental Health (WMH) surveys, Hurricane Katrina Community Advisory Group Study, BJS’ Survey of Inmates in State and Federal Correctional Facilities (SISFCF), 2004, and BJS’ Survey of Inmates in Local Jails (SILJ), 2002. While designing the mental health section of the NIS3, BJS worked closely with mental health experts at the National Institute of Mental Health (NIMH) and the Substance Abuse and Mental Health Services Administration (SAMHSA), as well as other mental health experts including Dr. Ronald Kessler, a professor of health care policy at Harvard Medical School. Dr. Kessler’s research has focused on mental health and the treatment of mental disorders and he has worked on several projects in which he was involved with the design and implementation of surveys that estimated the prevalence and correlates of psychiatric disorders and also treatment for those disorders.

Dr. Kessler assisted BJS in identifying brief mental health screening tools that have been validated on the general population and some that have also been validated on criminal justice populations, such as arrestees, probationers, and former prisoners (i.e., parolees), and that could be easily administered within the time constraints of the NIS3.

The K6 instrument is one of the mental health screeners included in the NIS3 (questions R1-R6 in NIS3). The K6, which was developed by Dr. Kessler, is a screening tool that has been validated and screens for severe psychological distress associated with serious mental illness. The K6 is included in a number of surveys, such as the NCS, NSDUH, NHIS, and the Arrestee Drug Abuse Monitoring (ADAM) program, and has been validated on criminal justice populations (i.e., arrestees, probationers, and former prisoners/parolees). Additional mental health screeners were also included in the NIS3 to provide an indication of the extent of specific mental disorders among the incarcerated populations. The specific mental disorder screeners that have been validated on the general population and are included in the NIS3 were pulled from the list of sources identified above, and include a screener for major depression (questions R7-R9), generalized anxiety disorder (questions R10-R12), panic disorder (questions R13-R14), intermittent explosive disorder (questions R15-R16), bipolar disorder (questions R17-R20), and post-traumatic stress disorder (questions R21-R23). During the questionnaire design for the NIS3, BJS, Dr. Kessler, and RTI, concluded that it was necessary to make minor changes to the wording of some questions or response options to either adapt the questions to the inmate population or to the ACASI mode of data collection.

The NIS3 also includes a question that asks inmates to report whether they have ever been told by a mental health professional that they have a specific mental disorder(s) (question R24). Because this specific question was included in BJS’ last iterations of the national omnibus surveys of prisoners and jail inmates, this question will allow BJS to conduct some trend analysis to determine if rates have increased over time, and also assess the rate of comorbidity of mental disorders among the incarcerated population. The NIS3 added follow-up questions designed to determine whether the diagnosis occurred prior to or since the inmate’s current admission (questions R36 and R39).

The remaining sets of questions included in the mental health section of the NIS3 measure treatment inmates received for mental health problems. Some questions measure types of treatment received for mental health problems ever (questions R26, R29 and R33), prior to admission (questions R27 and R30), and since admission (questions R31 and R34); these types of questions were also included in BJS’ last iterations of the national omnibus surveys of prisoners and jails inmates, allowing us to assess changes in treatment over time. Minor revisions to the wording of some of these questions were made to make the NIS3 questions more comparable with general population surveys, such as the NSDUH. In addition to these treatment questions, the NIS3 also includes a current measure of treatment by type (questions R32 and R35). There is also a series of questions that measure different types of treatment inmates received for specific disorders with which they reported being diagnosed, both prior to admission (questions R37 and R38) and since admission (questions R40 and R41). These series of questions will allow BJS to assess the types of treatment received and whether inmates have been treated for multiple mental disorders. These series of questions measure the continuity of care, by specific type of treatment and disorder, as the inmate transitioned from the community to the jail/prison.

Analysis of the data from the pretest, conducted in January 2010 in two prison facilities and one jail facility with 377 inmates, displayed no anomalies or difficulties in terms of time or question-skipping when answering either these questions or any other part of the revised instruments.