CJP supporting statement B rev sept2009

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

The size of the respondent universe for the CJP for this OMB request is estimated to be about 1,000. There are 1,606 Geographic Probation Service Areas (GPSAs) in the U.S. GPSAs are defined as the lowest level of service provision and oversight (juvenile probation services) across the states. In the parlance of establishment survey research, the GPSAs represent discreet and identifiable functional units. However, the CJP is more likely than the CJPSO to involve state-level respondents who will provide data for numerous GPSAs. To date, we are aware of at least eight states with central state reporters. This reduces the respondent pool for this form to approximately 1,000 respondents for the 1,606 GPSAs.

The reliance upon GPSAs is made necessary by the variability of juvenile probation systems across the states. Juvenile probation systems may be considered as either state, local or mixed. A state system generally allows for some state role in probation administration, budgeting, policymaking, data collection, and service provision. Local systems tend to provide all of those functions autonomously. Mixed systems allocate functions to both state and local agencies. Regardless of the type of organization, all states divide into smaller geographic areas for the purpose of providing probation supervision and collecting relevant data. The geographic subdivisions of states are the GPSAs. Typically these GPSAs fall along county or municipal lines so that each represents a single county or municipality. These are referred to as “Single GPSAs.” States, particularly those with lower population density, sometimes group counties together for probation purposes. These areas, that is, those GPSA with more than one county or geographic area being served, are referred to “Multiple GPSAs.”

An alternative strategy that defined the unit as county/municipal level was considered and rejected. Although all respondents easily understand county and municipal boundaries, their use produces two significant problems. The first is the situation in which a Multiple GPSA (one encompassing more than one county or municipality) cannot break out data by county/municipality because either: (a) caseloads and or services within their region flow between counties/municipalities; or (b) data are not collected at the smaller levels of aggregation. A second obstacle occurs when a Multiple GPSA does not fall neatly along county/municipalities boundaries. For these reasons the unit is the GPSA. The data collections included in this request will describe what happens in GPSAs across the United States; some will be single GPSAs and some will be Multiple GPSAs.

OJJDP maintains the universe list of GPSAs through the National Juvenile Justice Program Directory Project. This project was developed specifically to provide OJJDP a means to routinely update all lists of juvenile justice agencies, facilities, courts, and programs. This project is operated by the Governments Division of the Bureau of the Census and George Mason University, through an inter-agency agreement with OJJDP.

Data collection activity for the CJP has only included the pilot test administered to the nationally representative sample of 176 offices. This sample was selected through stratification by office size and office type (multiple or single GPSA).

Future Collection Activity is as follows:

The CJP will be administered to the full field of the approximately 1,000 respondents representing the 1,606 GPSAs. The reference date is the fourth Wednesday in April. It is expected that the CJP will be administered in odd years.

The following schedule has been adopted by GMU and OJJDP for data collection procedures for the CJP instrument:

Time frame	Action
6 months prior to reference date	Begin phone contact to update respondent e-mail information, provide CJP support information, and alert to upcoming trainings.
4 months prior to reference date	Begin electronic preregistration to alert respondents to the collection, verify contact information, and provide resources and training to respondents.
3 months prior to the reference date	Regional training for respondents and electronic systems in place
2 months prior to the reference date	Verification of electronic security, load testing and pretesting of the electronic forms
1.5 months prior to reference date	Full-field electronic notification and mailed notification to respondents without electronic communications
2 weeks prior	Reminder of upcoming reference date (electronic and mail to those without electronic communication)
Reference day	Full-field electronic notification
2 weeks post reference day	Assessment of response rate and contact with respondents who have not activated their accounts
3 weeks post reference day	Personal contact (by phone or mail) with non-contact respondents
4 weeks post reference day	Report to the respondent pool on progress
6 week post reference day	Announcement of closing electronic forms
8 weeks post reference day	Electronic forms shut down, phone and in-person contacts with non-respondents to offer assistance
10 weeks post reference day	Preliminary reports on response rates and basic population structures
16 weeks post reference day	Commencement of ‘peer pal’ matched jurisdictions
20 weeks post reference day	Move to report generation for masked comparison sites

Should circumstances require changes the schedule will be altered accordingly.

2. Statistics Procedures for Collection of Information.

The CJP mailout requires no statistical sampling as it is administered to the full population of GPSAs. Non-response imputation will be developed based on whether non-response is determined to be systematic or random.

3. Methods to Maximize Response Rates

Response rates and respondent burden and buy-in are closely related in the CJP collection. OJJDP is committed to reducing the burden on respondents and making the task of supplying data as simple as possible (see notes on alternate reporting methods above). The steps identified from the focus groups and in the schedule provide detailed information on how this data collection will be designed to be user friendly and utilitarian for respondents.

Typically, OJJDP has been able to achieve a high response rate (85-95 percent) for its other census data collection projects (Census of Juveniles in Residential Placement and Juvenile Residential Facility Census). Such a level of response has proven sufficient for the designated analysis purposes. This was the case in the administrations of the CJPSO, however, the CJP will likely require several administrations to properly train respondents and increase response rates through more flexible reporting options. As noted earlier, this will be the first such collection for this field, and will have to expect a learning curve.

OJJDP will use the following techniques to maximize response:

• Submit and process data online or electronically

• Leave online access open for multiday interaction with online forms

• Create online forms that permit interactive clarification of problematic terms

• Send certificates of participation to responders

• Notify responders with ample advance notice – 90 days

• Provide the rationale for the reference day

• Use preregistration for online submission

• Tie criminal codes and state geographic areas to the automated forms

• Note that the race field might not reflect the juvenile’s self-identification

• Include focus group members in the training and recruiting of peers

• Consider offering a listserv for responders enabling consultation with one another during the CJP collection process

• Send CJP representatives to areas with especially large numbers of juveniles on probation that are in need of assistance with the coordination of record retrieval

• Advise supervisors of the participation value in responding to the CJP request.

• Conduct training at professional organization meetings and through webinars

• Ensure responsivity of the research team to respondent needs.

Action Items for OJJDP in general:

• Link sites across the country using useful similarities; offender composition, population size, type of community creating a “Peer Pal Program”

• Establish national definitions for a handful of parole/justice system terms

To further understand how the instrument is working in the field, occasional response analysis (RAS) tests will be conducted. This will involve selecting a subsample of respondents in the CJP and conducting followup telephone interviews, focus groups, or webinars. The sample will likely be selected based on particular points of interest or identified problem areas and will of sufficient size to detect significant differences in reporting of these items.

4. Test of Procedures

The CJP form has been pre-tested, pilot-tested, and subjected to focus group scrutiny. OJJDP has restructured the data collection process, changed data collection agents, and is committed to creating the least burdensome, highest quality and utility data collection on juvenile probation. Client-tracking data will also be used in this iteration to establish communication records and process measures.

5. Statistical Consultants.

Catherine A Gallagher, Ph.D.

Associate Professor

Administration of Justice

George Mason University

(703) 993-8480