Part B for 1121-0291

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

The size of the respondent universe for the two forms covered under this OMB request is 1,606. This number represents all Geographic Probation Service Areas (GPSAs) in the U.S. GPSAs are defined as the lowest level of service provision and oversight (juvenile probation services) across the states. In the parlance of establishment survey research, the GPSAs represent discreet functional units.

The reliance upon GPSAs is made necessary by the variability of juvenile probation systems across the states. Juvenile probation systems may be considered as either state, local or mixed. A state system generally allows for some state role in probation administration, budgeting, policymaking, data collection, and service provision. Local systems tend to provide all of those functions autonomously. Mixed systems allocate functions to both state and local agencies. Regardless of the type of organization, all states divide into smaller geographic areas for the purpose of providing probation supervision and collecting relevant data. The geographic subdivisions of states are the GPSAs. Typically these GPSAs fall along county or municipal lines so that each represents a single county or municipality. These are referred to as “Single GPSAs.” States, particularly those with lower population density, sometimes group counties together for probation purposes. These areas, that is, those GPSA with more than one county or geographic area being served, are referred to “Multiple GPSAs.”

An alternative strategy that defined the unit as county/municipal level was considered and rejected. Although all respondents easily understand county and municipal boundaries, their use produces two significant problems. The first is the situation in which a Multiple GPSA (one encompassing more than one county or municipality) cannot break out data by county/municipality because either: (a) caseloads and or services within their region flow between counties/municipalities; or (b) data are not collected at the smaller levels of aggregation. A second obstacle occurs when a Multiple GPSA does not fall neatly along county/municipalities boundaries. For these reasons the unit is the GPSA. The data collections included in this request will describe what happens in GPSAs across the United States; some will be single GPSAs and some will be Multiple GPSAs.

OJJDP maintains the universe list of GPSAs through the National Juvenile Justice Program Directory Project. This project was developed specifically to provide OJJDP a means to routinely update all lists of juvenile justice agencies, facilities, courts, and programs. This project is operated by the Governments Division of the Bureau of the Census, through an inter-agency agreement with OJJDP.

Data collection activity has proceeded as follows:

37796. The first data collection activity was the administration of the CJPSO (Census of Juvenile Probation Supervision Offices). This occurred in the spring of 2005. The form was sent to one person in each GPSA (a total of 1,314 respondents). No sampling was done for this data collection. The response rate was 88%.

37797. The second data collection was the CJP (Census of Juveniles on Probation) Short and Long Form. A CJP-L (Long form) was sent to a sample (stratified by population of formal probationers served, type of office –that is, multiple or single, and geography) of GPSAs (n=246). The CJP-L collects individual data on young people on probation supervision, including age, race, sex, and most serious offense. The remaining GPSAs in the universe received the CJP-S (Short form). The short form collects just total counts of formal and informal probationers, as well as geographic probation areas served. Sixty-two percent of GPSAs receiving the CJP-L responded with individual-level data.

37798. The third administration occurred in 2007, and involved a reduced version of the original CJPSO instrument.

Future Collection Activity is as follows:

The CJP-L will be administered to a larger sample (n=500) in spring, 2008, with a reference date of the fourth Wednesday in April. Efforts to further increase electronic reporting capabilities are currently being developed to reduce respondent burden. It is expected that the CJP-L will occur in even years.

The CJPSO will be administered in odd years, with its next fielding slated for the spring of 2009, also with a reference date of the fourth Wednesday in April.

OJJDP has provided the below schedule to the Census Bureau for data collection procedures for the CJPSO instrument:

Time frame	Action
4 weeks prior to reference date	mailout advance notice letter
2 weeks prior to reference date	mailout survey forms
1 week following reference date	mailout reminder letters (non-respondents only)
4 weeks following reference date	mailout a second-notice survey form (non-respondents only)
6 weeks following reference date	begin telephone follow-up

This schedule was developed based on experience with past censuses and in experience testing the new instruments.

The CJP-L data collection timeline is different because it requires more significant preparation on behalf of respondents selected into the sample. The following is the current timeline:

Time frame	Action
8 weeks prior to reference date	First mailout of advance notice letter with detailed instructions
4 weeks prior to reference date	Mailout of advance notice with detailed instructions and survey forms
1 week following reference date	Mailout reminder letters (non-respondents only)
4 weeks following reference date	Telephone follow-up begins (non-respondents only)

Should circumstances require changes the schedule will be changed accordingly. Of particular note for the CJP Sample (roster) data collection will be the rather open policy of accepting electronic data in a form convenient to the respondent. OJJDP is committed to reducing the burden on respondents and making the task of supplying data as simple as possible (see notes on alternate reporting methods above). As such, we will accept data in most any form a respondent wishes to provide it.

Typically, OJJDP has been able to achieve a high response rate (85-95 percent) for its other census data collection projects (CJRP). Such a level of response has proven sufficient for the designated analysis purposes. This was the case in the administrations of the CJPSO, however, the CJP-L will likely require several administrations to properly train respondents and increase response rates through more flexible reporting options.

2. Procedures for Collection of Information.

The CJPSO mailout requires no statistical sampling as it is administered to the full population of GPSAs.

The initial CJP-L form used the below methodology for sampling:

A. Background

The objective of this survey is to estimate national characteristics of probationers. Individual-level information is to be collected from a sample of Juvenile Probation Offices about each young person on supervised probation on the reference day. The information includes age, sex, race and most serious offense. Race and sex are nominal categorical variables for which the parameters of interest are counts and proportions that lie in each category. ‘Most serious offense’ can possibly be considered an ordinal variable so that medians and other percentiles may be of interest. ‘Age’ can be treated as numerical, so the mean age is also an appropriate parameter to estimate.

B. Analysis of Respondents to the CJPSO

To determine a sampling plan that would obtain maximum information at acceptable precision levels while minimizing respondent burden, an analysis of respondents to the Census of Juvenile Probation Offices was performed. The key response variable to the census was the number of young-persons on supervised probation on a reference day in 2003. The response rate exceeded 88%: 1,417 of the 1,605 offices surveyed responded. Eleven of the respondents had no juveniles on supervised probation on the reference day. Since these offices may indeed have juveniles on supervised probation on the reference day for the new survey, they must be included in the sampling frame.

Neither simple random sampling (SRS) nor PPS sampling (probability proportional to the reported number of juveniles on supervised probation) are appropriate for this survey. Under PPS, offices with no juveniles on probation in the 2003 survey would have zero chance of selection. In addition, PPS would produce a sample consisting predominantly of large offices with 500 or more on supervised probation, a group that constitutes less than 15% of (respondent) juvenile probation offices. By contrast, in simple random sampling very few of the large offices would be selected and too many small offices, especially from Texas, would be included: almost 11% of the respondents are located in Texas.

Stratification is clearly the best approach as it combines the elements of PPS and SRS that best suit this survey. The remaining problem is to define the strata appropriately. Of the 153 offices in Texas that responded, 122 reported less than 100 juveniles on supervised probation. This was selected as one stratum. After much analysis, four additional strata were chosen using the reported number of juveniles on probation as the stratifying variable. The 187 non-respondents form an additional stratum.

C. Sample Design

The table below provides the stratum sizes for CJPSO respondents to the 2003 survey along with sample allocations. Several sample allocations to the strata were considered. These include proportional, equal, and optimal allocation among others. The column labeled “Sample Allocation Implemented” is the one that was used in drawing the sample. The other allocations are given for comparison, along with the relative margin of error of estimating total juveniles on supervised probation at the 95% confidence level.

Stratum: # reported on Probation	Number of Offices in Population	Sample Allocation Implemented	Proportional Allocation	Equal Allocation	Optimal Allocation
0-99 (except Texas)	579	30	74	36	17
100-499	528	60	68	36	61
500-999	107	20	14	36	17
1000-6000	81	50	11	36	81
Texas, 0-99	122	20	16	36	3
Total Respondents	1417	180	183	180	179
Relative 95% Margin of Error for # on Probation		5.4%	11.8%	7.0%	4.5%

The SAS procedure PROC SURVEYSELECT was used to select a stratified sample of respondents to the 2003 CJPSO using the allocation indicated in bold. Although optimal allocation indicates slightly better overall precision, sample sizes are more extreme. Moreover, these margins of error are only approximations since count data on the new reference day may not be highly correlated with previous 2003 data.

A simple random sample of 40 non-respondents to the CJPSO was also selected by PROC SURVEYSELECT, for a total sample of 220 juvenile probation offices.

D. Method of Estimation

The numbers and proportions of juveniles on supervised probation in various categories defined by age, race, sex and most serious offense are the key parameters to be estimated. The proportions are ratio estimates since the total number on supervised probation on the new reference day is unknown.

The remainder of this section gives the formulas to compute estimated counts and proportions and their margins of error. For proportions, the combined ratio estimator is recommended because of the small sample sizes in some strata. (The separate ratio estimator can also be used but is less stable when sample size is under 30 in some strata.)

Notation:

= number reported on supervised probation on 2006 reference day by office i.

= number in category of interest on reference day reported by office i.

.

= sample mean number on probation in stratum h.

= sample variance of number on probation in stratum h.

= sample mean count in category of interest in stratum h.

= sample variance of number in category in stratum h.

estimated proportion of probationers in category of interest in stratum h.

= estimated total number on probation in stratum h.

= estimated total in category of interest in stratum h.

= estimated total number on probation on 2006 reference day.

= estimated total number in category of interest on reference day.

= estimated proportion of probationers in category of interest.

(This is the so-called combined ratio estimate.)

for office i.

= sample mean residual in stratum h.

= sample variance of residuals d in stratum h.

(1.96) = 95% margin of error for estimated number of juveniles on supervised probation on the 2006 reference day.

(1.96) = 95% margin of error for estimated number of juveniles in category of interest on the 2006 reference day.

(1.96) = 95% margin of error for estimated proportion of juvenile probationers in category of interest.

E. Investigative Analysis: Model Building

The data may provide insight into risk factors for the rate of juveniles on probation (measured against number of arrests, population size, etc.) and the prevalence of types of ‘serious offenses’ by age, race, sex, population density and other variables of interest. Poisson regression models would be an appropriate tool to model counts or rates.

Future procedures for the CJP-L sample, based on the initial fielding, may seek to increase the overall sample size. The CJP sample will derived in much the same manner as the mailout test for the CJPSO described above. The exception will be that rather than relying upon juvenile arrests, the sample will be stratified by population size of juveniles on formal probation, reported in the CJPSO.

Sampling procedure for future CJP-L administrations will be similar to the above, but may seek an increased sample size.

3. Methods to Maximize Response Rates

OJJDP is committed to very high response rates and high quality data. Several steps are being taken to maximize respondent motivation and thus response rate. Both forms rely upon the wealth of results in survey methods research relevant to self-administered establishment questionnaires. Considerations included question wording, ordering and consistency, cognitive and time burdens, task difficulty, sensitivity of the topic, and visual layout. OJJDP will use the following techniques to maximize response:

• Although respondents will receive the forms from the Census Bureau, the form includes a highly visible identification of OJJDP as the project sponsor. Understanding that the initiative is on behalf of OJJDP will increase a willingness to participate.

• Forms are streamlined and include clear response instructions

• The mailing addresses on the forms are separated into two distinct sections so that the respondent's name and address appears in the mailing window.

• The forms docuprint the targeted respondent’s name into the text of the response sets, and the specific office name into the question wording.

• All potential respondents of these data collections will receive a pre-notification letter telling them about the form they are about to receive.

• OJJDP will conduct a workshop at the annual meeting of the American Parole and Probation Association to educate and interact with potential respondents regarding these data collections. All potential participants in the research will receive a letter alerting them to the workshop.

• The cover letter that arrives with the form increases respondents' motivation by outlining the utility of accurate responses and the products that the respondents will receive.

• The cover letter includes a note on where and when results of the census will be disseminated.

• Respondents have an option to submit their CJP data electronically, in a manner acceptable to the respondent

• The Census Bureau will provide continued support through a toll free number to answer any questions that arise

• There will be continuous contact with respondents through the mail (see the schedule for mailout and reminder notices above)

• Call-back procedures will continue until data close-out.

These procedures have been developed and tested through the administration of the Children in Custody Census and through ongoing administrations of the CJRP (1997, 1999, 2001, 2003, 2005, 2007) and four administrations of the JRFC (2000, 2002, 2004, 2006). They have proven effective in achieving and maintaining a high response rate of which OJJDP and the Census Bureau are quite proud.

To further understand how the instrument is working in the field, occasional response analysis (RAS) tests will be conducted. This will involve selecting a subsample of respondents in the CJPSO or CJP and conducting followup telephone interviews. The sample will likely be selected based on particular points of interest or identified problem areas and will of sufficient size to detect significant differences in reporting of these items.

4. Test of Procedures

Both forms involved in this request have been pre-tested and undergo ongoing pretesting as needed. Cognitive interviewing techniques are used to diagnose problems with questionnaires. These methods primarily rely on the concurrent think aloud technique with the researchers asking predetermined and ad hoc questions. Response analysis surveys are also possible if unanticipated problems arise during fielding.

5. Statistical Consultants.

Catherine Gallagher, Ph.D. Assistant Professor Administration of Justice George Mason University (703) 993-8480

Richard Bolstein, Chair Dept. of Applied & Engineering Statistics George Mason University (703) 993-1689