BJS responses to passback

BJS Responses to OMB Passback Questions on BJS Annual Parole/Probation Surveys

Question 1: …we would like BJS to give further thought to its proposal to discontinue attempting to collect death data by gender and ethnicity. Particularly useful would be more specifics about missing data rates and level of burden from previous collection rounds.

BJS’ decision to discontinue attempting to collect death data by gender and ethnicity in the Annual Parole Survey is based on three considerations: (1) BJS efforts to streamline its data collection efforts by looking for overlap with other data collections and eliminating duplicate efforts where feasible; (2) burden associated with providing death data by gender and ethnicity; and (3) BJS plans to expand upon the information it collects about deaths on parole by working to obtain incident-based records of deaths on parole.

First, in addition to the Annual Parole Survey (CJ-7), BJS also collects data on parolee deaths through its National Corrections Reporting Program (NCRP). The NCRP collects, among other items, individual-level records of parole discharges from 31 states. Included on the records are reasons for discharge from parole, and one category of discharge is death. Also included on the NCRP records are variables that measure age, gender, ethnicity, type of offense, length of sentence and time served in prison, and other variables associated with an offender’s prison term. The NCRP data therefore provide a richer source of information to study issues related to death on parole than do the aggregate counts of deaths by gender and ethnicity that have been requested through the Annual Parole Survey. Yet, the NCRP collection suffers from undercoverage, in that not all states submit records to the NCRP. To address undercoverage, BJS intends to commit additional effort to improving the coverage of the NCRP, in part by soliciting applications for a new data collection agent and including within the scope of work of the agent effort devoted to expanding the coverage of the NCRP. BJS current collection agent (Census Bureau) has not been able to increase coverage. We expect to issue the solicitation around the end of this calendar year.

Second, while not all states report parole discharges to BJS’ NCRP collection, nor have all states reported data on deaths by gender and ethnicity to the Annual Parole Survey. During 2005 and 2006, for example, 8 states were non-respondents on all items related to deaths by gender and ethnicity. (Six of these 8 also do not report to the NCRP.¹) For the remaining states that report to the Annual Parole Survey but not to the NCRP, there were 549 deaths in 2006, or 11% of the total number of deaths reported to the Annual Parole Survey. Although 8 states did not report death data by gender and ethnicity, the items related to deaths are among those requiring followup contact to obtain responses.

Third, BJS’ main interests in understanding deaths on parole lie in determining mortality rates on parole and the causes and timing of deaths on parole. To do this more effectively, BJS will need to expand its collection of information about deaths on parole. BJS plans to explore using one of its other collection programs to obtain additional data on deaths on parole. BJS’ Deaths in Custody Reporting Program (DCRP), implemented in response to the Deaths in Custody Reporting Act of 2000 (DICRA) (Public Law 106-297), required the collection of individual-level death records of persons who died in local jails, in state prisons, or during the process of arrest. This program has an established methodology for collecting and processing death records. Although parole death records are technically beyond the scope of the data collection requirements under the DICRA, the Act also does not preclude BJS from collecting these records. BJS will use several upcoming opportunities explore the feasibility of collecting detailed records of deaths on parole. For example, BJS has established a working group of stakeholders that meets regularly at the American Probation and Parole Association (APPA) meetings to discuss its data collections on community supervision. In addition, BJS regularly participates in a National Institute of Corrections’ sponsored meeting of executives of probation and parole. BJS will use these and other venues to explore the feasibility issues associated with collecting death records.

The patterns of deaths on parole are of great interest to BJS, as there are competing hypotheses about causes of death on parole. On the one hand, one speculation is that offenders on parole die primarily from medical problems that stem from conditions that they have prior to entering prison (such as cancer or heat disease, which are the leading causes of death in prison). On the other hand, life choices on parole, such as involvement in crimes leading to homicide, drug and alcohol use, and other behaviors could be leading causes of death on parole. Distinguishing the causes, as well as determining the timing of deaths on parole, would contribute significantly to issues related to prisoner reentry and crime control.

Given BJS intentions to enhance its collection of data under the NCRP and to explore expanding its collection of death records, BJS plans to discontinue collecting death data by gender and ethnicity on its Annual Parole Survey but to continue to collect data on the total number of deaths.

Question 2: …Has BJS considered adding one or two additional questions to these collections to ask whether parolees and probationers receive various types of assistance (e.g., housing, employment, medical, and mental health), and who provides the services (e.g., state, community-based non-profit service provider)? Could the Annual Parole/Probation Survey help to establish some baselines to know the prevalence of such services now and track their availability in the future?

BJS has considered asking questions such as the ones mentioned above and has done for parole supervising agencies through a separate collection effort, its 2006 Census of Adult Parole Supervising Agencies (CAPSA). On the other hand, BJS has concerns about the capacities of local probation agencies to provide responses to questions about services, and BJS intends to learn more about these capacities through its planned census of probation agencies. Finally, to improve collection of data on emerging policy issues—such as reentry—BJS has been working on developing a core-supplement approach to its probation and parole collections, in which a supplement or addendum to the core data collection would be used to gather data on emerging issues in the field.

First, included in CAPSA were three types of questions about services provided to parolees. BJS asked state parole supervising agencies two types of questions about drug treatment programs, sex offender programs, and mental health treatment programs: (1) Were any parolees enrolled in these programs; were any enrolled in programs run by formally-trained professionals; and for drug treatment, were any enrolled in self-help programs? (2) How many were enrolled in these programs? BJS also asked a third set of questions about provision of housing assistance and employment assistance to parolees. The questions about housing and employment assistance asked about formal programs within the agency, formal working relationships between the agency and state or county agencies, and occasional efforts by parole officers. These data from the 2006 CAPSA provide a baseline on the number of state agencies that provide various types of services. These data are reported in tables 9, 10, 11, and 12 of Characteristics of State Parole Supervising Agencies, 2006, a copy of which is attached.

While state parole supervising agencies reported on whether parolees were enrolled in the programs identified in the census, they were comparatively less able to report on the number of parolees that were enrolled in these programs. Thus, the data on the prevalence of parolees involved in these programs within states are less reliable than the data on the existence of these programs. For example, between 7 and 26 states were able to provide counts or estimates of the number of parolees that were involved in formally-run drug treatment programs, self-help drug treatment, sex offender programs, or mental health programs. Yet, between 47 and 49 states reported that they had these types of programs.

In sum, regarding programs offered to parolees, BJS has obtained some baseline data on the existence of these programs for 2006, but the data that state agencies were able to provide on the number of parolees participating in these programs were limited.

Regarding adding questions about programs to the Annual Probation Survey, BJS has concerns about the capacities of local (county-level) probation agencies to provide data on services. The majority of BJS probation respondents are county- or local-level agencies. (Only 54 of the 463 probation respondents are state or central reporters; the remainder consist of county- or local-level probation agencies.) Many county- and local-level respondents have limited capacities to provide even the core data on probation population movements, which is why BJS implemented the CJ-8A (or “short-form”) in our probation collection. In that form, we limit our data request to core items that measure probation population movements and we omit items that could be used to characterize probation populations in more detail, such as the details related to method of entry or exit from probation, details on types of offenses, details on the supervision status of the probation population. We therefore infer that the capacity of these agencies to provide more than an indication of a program would in all likelihood be limited. In addition, among respondents that receive the CJ-8 (long form), there are issues of nonresponse on some critical items. For example, during 2006, only 194 probation agencies are able to provide details on the types of entry onto probation, while 247 were able to provide details on the methods of exit or discharge.

Though it is reasonable to infer that local-level probation agencies may have difficulties providing data on the number of probationers that receive programs, that does not mean that they would be unable to provide data on whether their agency offers programs (such as drug treatment, etc.) to probationers. However, even that issue raises some challenges. For example, local probation offices perform a variety of supervisory functions, including those related to pretrial release, supervision of offenders in special court programs such as drug courts, as well as supervision of offenders that have been sanctioned by a court. This last group of offenders is the group on which we focus in our Annual Probation Survey. Hence, if we were to ask a question about whether the office provides probationers with a drug treatment program, we would need to know whether that program applied to sanctioned offenders versus other offenders. In order to make correct inferences about sanctioned probationers, the question about services would have to be accompanied with a series of additional questions about the organization of local probation offices.

To address issues such as the organization of local probation and the capacities of local probation offices to respond to BJS surveys, BJS plans to conduct a census of probation agencies. BJS has not undertaken such an effort since the early 1990s, but much has changed in community corrections since then. We are currently in a planning stage, and we are working with Census Bureau staff to develop a design plan for this effort. Three key objectives for this census are (1) to learn more about the capacities of agencies to provide reliable data on a variety of issues, including data on services; (2) to learn more about the organization of local probation; and (3) to determine the feasibility of using a panel or other sampling methods to obtain probation data from local jurisdictions. As part of our planning and design effort, we intend to meet with a working group of stakeholders to discuss this effort during the February meetings of APPA.

Finally, while reentry issues are currently important policy issues facing community corrections, there also are other important issues facing the field, and over time, the nature of policy issues will change. We have recognized the limitations of our parole and probation collections to capture data on emerging issues in the field, and we have recognized the need for flexibility in capturing data on new issues over time. At the same time, we are concerned about adding questions to the core elements (population movements) of these collections and the potential impact of adding these questions on response rates. For example, at a recent APPA meeting one of our respondents voiced a concern from the field that changes to the survey forms present challenges to the field in responding to our requests for data, and asked if we could alert the field (up to a year in advance) of forthcoming changes.

We believe that a vehicle for achieving these multiple goals may lie in our adopting a core-supplement approach to the surveys, in which the core focuses on the population movement data and the supplements address other issues. The supplements, or addenda, could be used as one-time efforts or could include questions that are repeated over time. This approach would allow us to announce to the field in advance upcoming requests for data, letting them know if they are one-time requests for data or are likely to be repeated. This approach would give us flexibility in administration of the supplements, as they could be fielded either at the same time as the core or on a different cycle. Based on what we find out in our upcoming efforts with the probation census planning work and our meetings with stakeholders at conferences, we may come back to OMB for clearance for a supplement during 2009.

Question 3: How many jurisdictions chronically report late due to not having the data available until the summer?

A point of clarification about the statement about late reporters: On page 20 of the “Justification” section, BJS wrote “Because some agencies do not finish their final reports (upon which they base their responses) until mid summer, the collection cannot be completed until that time.” The sentence is a little unclear on the following point: It is not necessarily the case that these respondents do not have the data available until summer; rather it is the case that they do not send data to BJS until after internal reports (internal to the agency) are reviewed and approved. Thus, this is not necessarily an information system problem—such as late posting of data—as it is an organizational issue—agency approval processes.

Regarding late reporters, during 2005, 2006, and 2007 about one quarter of the parole respondents (13) did not respond until after May. About one-third of probation respondents did not respond until after May.

Question 4: Why does BJS retain its current field data collection period when it knows that some jurisdictions can’t report until summer? What are the implications of moving to a later data collection period?

BJS fields the survey in late December to assist smaller probation agencies with less sophisticated information systems in obtaining year-end counts of their probation population, although at least one state (New Hampshire) also is in a similar situation. While we do not have an exact count of the number of such agencies, during followup contacts with the field and the followup contacts that our collection agent has made, we have heard from such agencies and the challenges that they face in providing year-end data when they have to go back into their systems to get the data. In order for them to provide us with December 31^st and January 1^st count of populations under supervision, they need to pull data from their information systems on or near the reference date. Waiting to field the survey until, say, late February, would prevent these respondents from obtaining and providing year-end counts.

Question 5: In order to release more timely statistics, has BJS analyzed the data quality implications of ending data collection without waiting for those chronically late reporters?

The major implication of ending the collection early without waiting for chronically-late reporters is that BJS would not be able to report complete state- or jurisdiction-level statistics. The effect on national-level statistics would comparatively small, as BJS has implemented imputation methods that could be used to generate national estimates. For the parole survey, BJS has rarely had to impute state-level estimates; for the probation survey, BJS regularly imputes data at the sub-state level and then aggregates these estimates up to the state level.

Among the chronically-late respondents in the parole survey are several large states—such as Virginia, Illinois—states such as Kentucky and New Hampshire, and the federal probation service (which has responsibility for supervising federal offenders released from prison to terms of supervised release). Ending the survey early would leave gaps in the data that BJS reports on the state and federal entities having supervisory responsibilities.

BJS efforts to reduce response time and end the collection early include: During followup contacts, request data only for selected critical items (such as item 4, yearend population count); impute or estimating selected quantities. To impute values for some items, BJS uses the followup contacts to ask respondents for estimates for items for which they did not provide values. BJS focuses these queries only on what it considers to be critical items—the yearend count of the number of persons on probation, the total number of entries and exits from probation, and distributions for such characteristics as race and gender. Agencies are virtually always able to provide an estimate of the number of persons they had on probation, but more frequently have only partial information on the number and types of persons entering and/or exiting probation.

Using the mathematical requirement that the number of persons on probation at the end of the year must equal the number of persons on probation at the beginning of the year plus those entering probation supervision during the year, minus those leaving, BJS will ask the respondent if the missing information, such as the number of exits, can be reasonably estimated from the other three items. If, after examining the available information, BJS and the respondent are unable come to agreement on estimates of the total number of entries and exits, BJS imputes values for these items based on similarly sized probation agencies in the same state. BJS also queries respondents as to the composition of their probation population when information on items such as gender and race has not been provided. Respondents are asked if major changes have occurred in the gender or racial and ethnic composition of their probation population over the last year, or whether the percentage distribution of these items provided in a previous year can be applied to the population total of the current year population. If the respondent asserts that major changes have occurred to the composition of the probation population, or that one or more percentage distributions provided in a previous year no longer appear appropriate, no further attempt is made to estimate these items. BJS does not report estimated entries, exits, imputed values of these elements, or estimates of other characteristics at the level of the respondent, but BJS uses them to generate its national-level estimates.

During the past year, BJS has given more attention to imputation methodology, exploring, for example, methods of multiple imputation and methods for formalizing the method into computer code so that it can assess the effects of alternate methodologies on imputing entries and exits to probation, and, potentially, for the first time, missing information for other characteristics on national (or state-level) estimates. The introduction of these new methods is still in its early stages, but BJS is now in a better position to begin to assess the effects of nonresponse on its estimates.

In addition, since receiving these passback questions from OMB, BJS has given consideration to additional methods to attempt to decrease the length of time that the survey is in the field. One idea that BJS is exploring is to send all respondents, either via fax or email, periodic status reports on the collection, to inform them of the response rate to the survey. These would be informational only, but it is our belief that such progress reports would help strengthen respondents’ investment in the surveys.

Finally, to increase the timeliness of release of data, BJS is working to reengineer its internal work processes in ways that allow for the production and dissemination of data tables directly from computer software applied to its databases and that incorporate estimates of imputation error to its national or state-level estimates. BJS traditionally has used statistical software for analysis, followed by keying data into tables that are then disseminated via the Web in various formats (e.g., .pdf, HTML, and .csv). However, BJS has begun to use statistical software to create and output tables in various formats directly from the software. These procedures allow for faster dissemination and facilitate updating data tables because the computer code can be run quickly against updated datasets to produce tables. This procedure would eliminate much of the time required to verify each datum in tables and allow for global as well as specific updating of tables. BJS is currently working to implement these procedures with several of its collections on prison and jail populations. Such procedures would allow BJS to facilitate a process whereby BJS releases preliminary data (such as national-level estimates along with some state-level data) early and at later date, as data collection becomes more complete, updated or final national-level estimates and complete state-level data.

Question 6: Please add more detail on how the names and addresses are updated yearly, as mentioned in B2.

The process for updating names and addresses is as follows:

• Staff from the Census Bureau, BJS data collection agent, make updates to the mailing labels as they receive completed forms from respondents. Bureau staff review forms for any changes that respondents have made to the address or contact section of the form. Before Bureau staff edit or key data, they make sure new address information has been updated in their contact database. Updates to the contact information are made continually as new information is received.

• If a package that was mailed out to a respondent is returned to the Bureau, Census Bureau staff will note this in their contact database. Any information associated with the reason for a package being returned (such as the person no longer works there, a facility or location has closed, etc.) is also recorded. (Each year, only a few packages are returned for these reasons.)

• Some respondents who submit data directly online also update their contact information directly online. Because many respondents who update contact information online do not call or otherwise notify Bureau staff that their contact information has changed, Bureau staff visually inspect the contact information on completed forms and compares that information to the information the Bureau maintains in its contact database. In case of discrepancies between the two sources, Bureau staff use the more recent contact information provided by respondents online and update their contact database.

• During followup phone contacts, Bureau staff will verify that the contact information contained in the contact database is up-to-date.

• BJS has also asked Bureau staff also regularly check the contact information for the parole survey with contact information for BJS National Prisoners Statistics collections, as many of the same respondents provide data for both of these collections to BJS. When discrepancies between the two are identified, if Bureau staff are unable to resolve the discrepancies, they get in touch with BJS staff to see if BJS has the most up-to-date contact information.

Question 7: …we encourage BJS to educate its respondents about the categories that all federal data collections must use, along with appropriate definitions.

To help to educate its respondents about the categories and definitions, BJS will include in the CJ-7 and CJ-8 forms an insert that provides the OMB categories and definitions for race and ethnic groups, along with instructions for respondents to refer to the insert for guidance. (Copies of the inserts are attached.) In addition, in the cover letter that accompanies the mailout of the survey forms, BJS refers respondents to this information. Finally, to enhance consistency across surveys, BJS has adopted the format for asking for data about race and ethnicity data that was used in the National Prisoner Statistics (NPS) surveys that were recently approved by OMB on May 29, 2008 (NPS-1A/1B, OMB# 1121-0102). We have therefore revised the CJ-7 and CJ-8 forms to show the same categories as appear in the NPS forms, and we have added the reference to the insert. (Copies of the survey forms are attached.)

In addition, BJS recognizes that Item H, "Additional categories in your information system" in question 10 of the CJ-7 (parole) and question 7 of the CJ-8 (probation) forms that ask for the number of offenders by race and ethnicity is not completely consistent with OMB categories but it is necessary to accommodate the information systems of state and local respondents to these two surveys. During this cycle, BJS will analyze the information obtained from Item H in these two questions and take steps to inform respondents about how to classify this information into OMB race/ethnic categories; will itself reclassify responses into OMB categories; will not report data unable to be reclassified from Item H and will reclassify those response otherwise unclassifiable into the “unknown” category; and will aim to eventually eliminate Item H from the next cycle of these surveys.

1 Some states that report to the Annual Parole Survey, such as Georgia, do not participate in the NCRP, and the reason for this is that historically, the NCRP data were obtained from state departments of corrections, including the data on parole discharges. As some states DOCs do not maintain the parole records, these states did not submit parole discharges. BJS is currently planning to enhance its efforts to enroll states into the NCRP, and part of this effort will be devoted to obtaining parole discharge data from departments of parole directly, rather than from DOCs only.

Page 8