OMB Package - ATDC - 9-8-09

Adult Treatment Drug Court Cross-Site Evaluation

Supporting Statement

1. Justification

1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Substance Abuse Treatment (CSAT) is requesting approval from the Office of Management and Budget (OMB) for 7 data collection instruments:

• Interview Guide for Drug Court Staff (Attachment A);

• Procedural Justice Questionnaire (Attachment B);

• Mental Health Screener for Women (Attachment C);

• Mental Health Screener for Men (Attachment D);

• Treatment Satisfaction Index (Attachment E);

• Focus Group Questionnaire (Attachment F).

Also included will be the burden for the GPRA instrument (OMB No. 0930-0208, Expiration 6/30/2012; see Attachment G). In 2008, CSAT/SAMHSA provided funding to 20 existing drug courts for enhancement and/or expansion of their drug court capabilities. The proposed data collection will enhance SAMHSA’s knowledge regarding drug court processes and will provide data on the value-added to these drug courts as a result of receiving Federal funds through SAMHSA’s discretionary funds portfolio. The results from this data collection may serve to inform future decisions regarding funding by SAMHSA for adult treatment drug courts.

Treatment Drug Court grants are authorized under the Public Health Service Act, as amended, in Section 509 – Priority Substance Abuse Treatment Needs of Regional and National Significance. SAMHSA’s legislative mandate is to increase access to high quality prevention and treatment services and to improve outcomes. Its mission is to improve the quality and availability of treatment and prevention services for substance abuse and mental illness. To support this mission, the Agency’s overarching goals are:

1) Accountability—Establish systems to ensure program performance measurement and accountability

2) Capacity—Build, maintain, and enhance mental health and substance abuse infrastructure and capacity

3) Effectiveness—Enable all communities and providers to deliver effective services

Each of these key goals complements SAMHSA’s legislative mandate. All of SAMHSA’s programs and activities are geared toward the achievement of these goals.

2. Purpose and Use of Information

The information gathered under this data collection will be used by SAMHSA staff to determine the value-added to 20 existing adult treatment drug courts resulting from Federal funds. Value-added will be measured as reduced recidivism among participants, increased access to treatment, and higher retention rates. This information will provide SAMHSA with evidence to support future decisions regarding funding for adult treatment drug courts.

Budgetary funds exist for this data collection. It was planned by SAMHSA to serve as an independent, cross-site evaluation of their enhancement and/or expansion treatment dollars. The 20 drug courts are required to conduct their own program evaluation. However, these evaluations will be specific to each drug court, and cross-site comparisons would be difficult. The current data collection will provide an independent review of the 20 drug courts that received SAMHSA funds. Additionally, because the data collection instruments will be standardized to collect the same information in all 20 drug courts and with drug court clients in each of those locations, it will also allow for cross-site comparisons between the sites.

There are two data collection arms for this project: information pertaining to the drug court itself (including operating procedures and personnel) and data on drug court clients. The drug court team members (i.e., SAMHSA grantees) are required to participate in the cross-site evaluation as a stipulation of receiving funding from SAMHSA. However, the drug court clients are not required to participate in the evaluation.

Data collection from drug court personnel

Data from drug court personnel will pertain to their demographics, court room processes and other specifics about their drug court. This data will be collected through visual observation of courtroom proceedings, collection and review of drug court grant applications and courtroom manuals, and interviews with the drug court team (e.g., judge, prosecutor, defense attorney, drug court manager, and treatment provider).

Visual observations of court room processes will be made during two site visits to the 20 drug courts. This phase of data collection requires little effort from the drug court team personnel. However, the researchers will need to coordinate with the drug court team to avoid potential schedule conflicts. The review of drug court grant applications and court room manuals also poses little burden on the drug court staff. The evaluation team has already received copies of the drug court grant applications from SAMHSA and most of the drug court manuals. Drug court grant applications and drug court operating manuals will be reviewed and data related to the court procedures and expected use of SAMHSA funds extracted.

Interviews with the drug court staff will be conducted during each of the two site visits and each interview session will take about an hour. The interviews will be conducted with each individual team member who is available during the scheduled site visit. These interviews will address questions about the drug court (Attachment A). The drug court team members will not be compensated for this interview because they are required by SAMHSA to collaborate with the cross-site evaluation.

Data collection from drug court clients

The drug court client interviews will take about 45 minutes each. GPRA (Attachment G), Procedural Justice (Attachment B), Mental Health Screener for Women (Attachment C) and Men (Attachment D), and Treatment Satisfaction Index (Attachment E) questionnaires will be administered to drug court clients who were enrolled into the drug court once SAMHSA funds began to be utilized and OMB approval has been received. GPRA data collection is already required by the drug court grantees at baseline, 6-months post-baseline, and discharge. The GPRA data collection addressed in this document pertains to an additional data collection point. In addition to the GPRA instrument, during this data collection, a 13-item questionnaire about the drug court clients’ perceptions of procedural justice (e.g., fairness by the court and court personnel), a mental health screener (gender specific), and a treatment satisfaction index will be administered. These instruments will be collected from drug court clients at one point in time: 6-months post-discharge from the program (e.g., after successful completion or dismissal from the program). The GPRA data will provide the first post-discharge information on clients receiving programs supported by SAMHSA funds.

In addition to surveys administered to drug court clients, official criminal justice data (rap sheets) will be collected from the courts and used to assess the legal histories of drug court clients. Of particular interest will be drug court clients’ involvement with the criminal justice system during and after participation in the drug court. Information about their criminal history prior to enrollment in the drug court will also be obtained and utilized in the analyses. The collection of this data will require effort on the part of the drug court staff. This data is public information, but obtaining it often requires judicial assistance. During the two site visits, the evaluation team will work with the drug court personnel to access this data. Again, because the drug courts are required by SAMHSA to collaborate with the cross-site evaluation, the drug court personnel will not be compensated for this effort.

Focus groups are planned at each site visit (20 sites and two visits for a total of 40 focus groups) to collect qualitative data from a small group (8-10) of drug court clients. During these sessions, the lead evaluators will ask clients questions about their perceptions of the drug court (see Attachment G). The clients participating in the focus groups will be selected from the population of drug court clients. A sample of convenience will be selected – clients who are appearing before the judge on the day of the site visit will be asked by the drug court manager to participate in the focus group.

3. Use of Information Technology

It is anticipated that 80% of the grantees will have criminal justice data available electronically. Availability of such data will reduce burden on the drug court and criminal justice system personnel.

Data collected pertaining to the drug court and drug court personnel will be collected in an informal setting (i.e., observations and conversations). Drug court personnel will be administered questionnaires using a paper/pencil format. This will create an informal, conversational atmosphere that will facilitate open discussions.

The drug court client surveys (GPRA, Procedural Justice, Mental Health, and Treatment Satisfaction Instruments) will be administered in paper/pencil format, with the interviewer reading the questions and recording the answers. Before the interview, a unique identifier (without personal identifying information such as name) will be written on the questionnaire. The informed consent paperwork will be separate from the questionnaire. After the interview, the questionnaire will be placed in a Fed Ex envelope and the consent form placed in a separate Fed Ex envelope; both will be sealed in front of the evaluation participant and placed in the mail to the contractor after the interviewer leaves the interview location. Paper/pencil data collection of GPRA is required by SAMHSA. The data will then be key-entered into the contractor’s evaluation database (not SAIS). The other instruments will also be collected using this format for continuity during the interview and so that information is captured in one place.

4. Efforts to Identify Duplication

The data to be collected for this specific program evaluation does not exist elsewhere. To date, this type of data has not been collected from drug courts or drug court clients who received funding from SAMHSA to expand and/or enhance the treatment component of existing drug court programs. Further, no systematic collection of post-program GPRA data has been obtained from SAMHSA grantees. According to a search of the SAMHSA website, and conversations with SAMHSA staff, this will be the first cross-site evaluation of adult treatment drug courts receiving SAMHSA funds. An independent evaluation of the previously SAMHSA funded adult treatment drug courts has not been conducted. While there are several studies in the literature that examine drug courts, many of these evaluate programs funded by the Department of Justice. The focus of SAMHSA (e.g., treatment) is not necessarily the focus of more corrections based evaluations. These studies focus less on treatment, and more on reductions in jail/prison beds.

5. Involvement of Small Entities

The evaluation described in this document will include data collection from 20 drug courts, and some of these courts will serve small jurisdictions (populations less than 50,000). However, the data collected from these entities will be observational (e.g., court room proceedings and meetings) and hard-copy data (e.g., court operating manuals) and will not require staff members from the courts to complete questionnaires. Therefore, there will be no significant impact to small entities.

6. Consequences If Information Collected Less Frequently

Drug Court Staff

The collection of drug court grant applications and operating manuals will occur once. Observational data from drug court staff is scheduled to be collected during two site visits. The purpose for two observational data collection points is to account for changes over time in staff, procedures, mechanisms, and how these variables affect drug court client outcomes.

Drug Court Clients

Data from the majority of drug court clients will be collected at one point in time, and all data collection is voluntary. A one-time interview is scheduled for approximately 6-months post-discharge from the drug court. GPRA, Procedural Justice, Mental Health Screener (Men and Women), and Treatment Satisfaction Index data will be collected during this interview. Focus groups will be conducted at each site visit with a small sample (8-10) of different drug court clients in each of the 20 drug court sites, during the three year project (a total of 40 focus groups). The focus group members will be selected from the larger pool of drug court clients. Therefore, some drug court clients will participate in both the interview and a focus group. However, these data collection efforts capture different data. The focus groups are conducted more frequently than the interviews to increase the sample size.

7. Consistency With the Guidelines in 5 CFR 1320.5(d)(2)

This information collection fully complies with 5 CFR 1320.5(d)(2).

8. Consultation Outside the Agency

The notice required in 5 CFR 1320.8(d) was published in the Federal Register on April 21, 2009 (Vol. 74, No. 75 FR 18239). No comments were received.

An expert panel meeting of drug court experts, and SAMHSA staff, was held in February 2009 to inform the evaluation design and data collection plan. The expert panel provided valuable insight and led to the addition of three brief instruments (Mental Health Screener for Men and Women, and Treatment Satisfaction Index) to the drug court client interviews.

The people who served on the expert panel were:

Steve Belenko

Temple University

Department of Criminal Justice

558 Gladfelter Hall

1115 West Berks Street

Philadelphia, PA 19122-6089
Phone: 215-204-2211
[email protected]

Caroline Cooper

Justice Programs Office of the School of Public Affairs

American University

Brandywine Building, Suite 100
4400 Massachusetts Avenue, NW
Washington, DC 20016-8159
Phone: 202-885-2875
[email protected]

Michael L. Dennis

Lighthouse Institute

Chestnut Health Systems

720 West Chestnut

Bloomington, IL 61701

Phone: 309- 827-6026

[email protected]

Michael Finigan

NPC Research

4380 SW Macadam Ave., Ste. 530

Portland, OR 97239-6408

Phone: 503-243-2436, x 102

[email protected]

Tim Jeffries

Bureau of Justice Assistance

Office of Justice Programs

U.S. Department of Justice

810 7^th Street NW

Washington, DC 20531

Phone: 212-616-7385

[email protected]

Peggy Fulton Hora
Superior Court of California
P.O. Box 5246
Walnut Creek, CA 94596

[email protected]

Akiva Liberman

National Institute on Drug Abuse

6001 Executive Blvd

Room 5185

Bethesda, MD 20892-9589

Phone: 301-402-0807

[email protected]

Douglas B. Marlowe

Treatment Research Institute and

National Association of Drug Court Professionals

600 Public Ledger Bldg.

150 South Independence Mall West

Philadelphia, PA  19106

Phone: 610-388-8606

[email protected]

Raymundo Perez

County of Orange
Health Care Agency
P.O. Box 355
Santa Ana, CA 92702

Phone: 714-834-4722

[email protected]

Louis Presenza

Criminal Justice Center

1301 Filbert Street

Philadelphia, PA 19107

Phone: 215-683-7200

Fax: 215-683-7203

[email protected]

Michael Rempel

Center for Court Innovation
520 8th Avenue, #1801
New York, NY 10018

Phone: 212-373-1681

[email protected]

John K. Roman

Urban Institute

Justice Policy Center

2100 M Street, NW

Washington DC 20037

Phone: 202-261-5774

[email protected]

Linda Truitt

National Institute of Justice

810 7^th Street, NW

Washington, DC 20531

Phone: 202-353-9081

[email protected]

9. Payment to Respondents

Drug court staff will not receive remuneration. Participate in the cross-site evaluation is a contingency of the SAMHSA grant they received. Drug court clients will receive a $20 gift card to a local store (e.g., Wal-Mart) for the 6-month post-drug court discharge interview. Travel will not be required since the interviews will occur in the respondent’s home or correctional facility. Focus group participants will be compensated with a $10 gift card to a local store. The focus groups will take place at the court on a day that the clients would need to appear before the judge, therefore eliminating any additional travel time and expenses.

The remuneration is commensurate with the type of sensitive information (e.g., substance use, criminal activity) that will be collected from a hard to reach population. Such compensation should increase response rates, in turn, increasing the reliability and generalizability of the data. Without remuneration, it is expected that response rates would be extremely low.

10. Assurance of Confidentiality

Data collected from the drug court personnel will be stripped of identifiers once it has been key-entered into the evaluation database. An evaluation ID will be assigned to each court, the cross-walk linking the drug court personnel and the evaluation ID will be stored in a locked file cabinet, and access will be available only to the cross-site evaluation team Project Directors.

Personal identifiers will be necessary for tracing drug court clients/respondents once they are discharged from the drug court. These identifiers will be accessible only to those staff members responsible for locating, contacting and interviewing respondents during at 6-month post-discharge data collection. During the focus groups, no personal information will be collected by the cross-site evaluation team. The drug court personnel will recruit participants for the focus groups.

Prior to data collection, the consent form that indicates that SAMHSA is funding this evaluation, the purpose of the data collection, that their involvement is voluntary (and that they are free to discontinue participation at any time without penalty), and the procedures in place to keep their information confidential will be read to the participant. They will be required to sign one copy of the form, and a second copy will be given to respondents for their own records.

Institutional Review Board (IRB) approval was obtained on March 21, 2009 (Attachment H).

11. Questions of a Sensitive Nature

The drug court clients will be asked several questions that are sensitive in nature. These include legal and illegal substance use, mental health, and criminal activity. These questions are necessary to meet SAMHSA’s evaluation goals.

Prior to administering the questionnaire, respondents will be asked to provide informed consent to conduct the interview. If a respondent agrees to participate in the evaluation, he or she will be asked to sign a copy of the informed consent and will be given an unsigned copy of the document for their records. The signed copy will be mailed to the cross-site evaluation team in a Fed Ex envelope, separate from the de-identified questionnaire, and housed in a locked file cabinet that is only accessible by the cross-site evaluation team Project Directors.

A consent form for each type of data collection has been created. Attachment I is the Drug Court Personnel Consent Form. Attachment J is the Drug Court Client 6-month Post-Discharge Consent Form. Attachment K is the Drug Court Client Focus Group Consent Form.

12. Estimates of Annualized Hour Burden

	Number of Respondents	Responses per Respondent	Total Responses	Hours per Response	Total Hour Burden	Hourly Wage Cost	Total Hour Cost ($)
Interview with Drug Court Personnel	160	2	320	.5	80	$30	$2,400
Procedural Justice Questionnaire	816	1	816	.09	73	$10	$734
Correction Mental Health Screener for Women	408	1	408	.08	33	$10	$330
Correction Mental Health Screener for Men	408	1	408	.08	33	$10	$330
Treatment Satisfaction Index	816	1	816	.08	65	$10	$650
Drug Court Clients – Focus Groups	400	1	400	1.0	400	$10	$4,000
GPRA	816	1	816	.33	269	$10	$2,693
Total	4,104		4,584		1,193		$11,137

The hours per response are based on previous experience with similar data collection activities. There are no direct costs to respondents other than their time to participate in the interview. We assume an hourly rate of $10/hour. This is higher than the current Federal minimum wage of $6.55 due to the population being interviewed (i.e., substance abusers) and the sensitive nature of questions being asked during the interview (e.g., current substance use, criminal behavior).

13. Estimates of Annualized Cost Burden to Respondents

There is no capital, start-up or operational costs associated with this data collection.

14. Estimates of Annualized Cost to the Government

The annualized cost to the Government, in addition to the cost estimates is $1,791,561, which includes 49% of the contract budget as well as 20% of the Government Project Officer’s time.

15. Changes in Burden

This is a new project.

16. Time Schedule, Publication and Analysis Plans

The data collection will begin once OMB approval is received; this is estimated to be December 31, 2009.

Schedule

Activity	Expected Date
IRB Meeting	January 13, 2009
Expert Panel meeting held	February 2009
Publish 60 day notice	April 21, 2009
Construct OMB Package	May – June 2009
Submit final package to SAMHSA’s OMB Office for review	July 1, 2009
Review of OMB package by SAMHSA	July 1 – August 15, 2009
Submit final package to OAS for review	August 16, 2009
30 day Federal Register Notice and package to Department	August 16, 2009
Package to Office of Management and Budget (OMB)	August 30, 2009
Field interviewers and data collectors trained	December 2009
Approval received	December 31, 2009
Drug court client interviews begin	January 11, 2010
Site visits to courts begin	January 11, 2010
Drug court client focus groups begin	January 11, 2010
Drug court client interviews end	May 31, 2012
Annual site visits to courts end	May 31, 2012
Final report to SAMHSA	September 30, 2012

Evaluation Questions

1. What were the various judicial and court practices implemented in the CSAT grant programs and to what extent are the 20 drug courts adhering to the 10 key components of drug courts developed by the National Association of Drug Court Professionals?

2. What is the impact of sanctions and incentives on drug court clients?

3. What are the different types of treatment services provided in drug courts? Are there any correlations between the different types of treatment services provided and the different types of offenses committed by the clients?

4. Who are the clients that frequent drug courts and what are their crimes? How many?

5. Who and where are they in terms of demographics, drug use and treatment history, race/ethnicity, gender, socioeconomic status?

6. What is the status/outcomes of drug court clients six months and one year after involvement in the program?

7. What are the traits and characteristics of judges who are in drug courts and what characteristics are associated with better outcomes?

8. To what extent do minority sub-groups have differential access to drug court programs or success or failure rates?

9. What is the effectiveness of the Adult Treatment Drug Court program? For instance, are clients improving in terms of better identification of problems, access to treatment services reduced drug use? What is the effectiveness of Treatment Drug Courts, including cost-effectiveness, compared with other treatments?

10. What effect does the introduction of Treatment Drug Court model have on existing treatment programs? What is the impact of this new development on the treatment field overall?

11. What are client perceptions of the program six months and one year after admission to the program?

Data Analysis Plan

Based on prior experience with drug courts specifically and multisite interventions in general, CSAT/SAMHSA expects substantial differences in the ways that grantees approach the implementation of their program. To make this diversity an opportunity rather than an analytic liability, the a context-mechanisms-outcomes (CMO) model will be used in tandem with concept mapping to identify the important differences and similarities in approach, and multilevel modeling to model those different approaches, including testing the degree to which services were implemented, process outcomes were achieved, and participant outcomes were achieved. CSAT/SAMHSA will integrate the multilevel modeling with an event history and cost-effectiveness analyses, described further below.

Multilevel models are well suited to sort out the cause-and-effect relationships of ATDC activities on participant outcomes. By properly adjusting standard errors for within-jurisdiction clustering of participants and serial correlation of longitudinal outcomes, an increase in the confidence with which observed changes can be attributed to the implementation of ATDC activities will be achieved. This in turn will lead to better grounded recommendations for improving ATDC effectiveness in the future. As in other analyses of cross-site evaluations that have been conducted (e.g., Orwin et al., 1994; 1999a and b; 2000; 2004; 2005), CSAT/SAMHSA will use these methods to take account of the “nested” character of the longitudinal epidemiological outcome data within participants, as well as nesting of participants within jurisdictions (Murray, 1998). The approach can also reveal sites that, for any reason, are discrepant from others with similar characteristics on the variables included in the model (that is, “outliers” in the distributions of outcomes) through graphic display of estimates and residuals. This can be the basis for beginning further analyses of the reasons for such discrepancies, which may involve values of other variables available in the data but not included in the multilevel model, or point the way to other, more global characteristics highlighted only in more qualitative data from the drug court teams (e.g., from the concept mapping exercise) or in the expertise of treatment system and community informants.

Modeling will be performed using Hierarchical Linear Modeling (HLM) Version 6 (Raudenbush et al., 2004). The program handles virtually every known variety of the 2- and 3-level mixed model. For outcomes that do not meet normality assumptions, HLM lets the user specify a nonlinear analysis appropriate for the distributional characteristics of the dependent variable (dichotomous, ordinal, counts, nominal, etc.). It also accommodates sampling weights in both linear and nonlinear models. This is relevant to the analysis because (1) most of the National Outcome Measures and other outcomes will not meet normality assumptions and therefore require nonlinear models, and (2) grantees will contribute unequal numbers of participants and longitudinal observations to the cross-site database. Therefore, inverse weighting by these inequalities at the appropriate level will increase the generalizability of the findings. Applied to longitudinal data, the mixed-effects approach can allow for individual-varying intercepts and slopes across time and can estimate the degree to which these time-related terms vary. Model covariates can be either time-varying or time-invariant. Thus they will accommodate both fixed and dynamic characteristics of programs and participants. The dynamic characteristics will include the transition rates from the event history analysis described next.

The multi-level analysis will be integrated with event history analysis to address key questions involving temporally sequenced events in the drug court program (e.g., what is the impact of sanctions and incentives on drug court participants?). The analysis will begin by examining mean time to recidivism and estimated median lifetimes (EML) to compare treatment and comparison participants on time to recidivism. Other temporal events (e.g., time to relapse, time to completion or termination) can be examined as well. Derived from survival functions, the EML indicates how much time passed before half the sample recidivates. Next, hazard functions will be estimated for each intervention, using the life-table method (see Kalbfleish and Prentice, 1980). The hazard function represents the probability of recidivism among participants over time. Hazard functions will be examined and descriptively compared across ATDCs and subgroups within ATDCs. It is expected that grantees will take steps to address recidivism problems as they emerge during implementation; in prior cross-sites these strategies have been content analyzed and codified. To gain some indication of which efforts may have been successful, changes in recidivism rates and time to event over the life of the grant will be examined, both statistically using correlations between time to event and program entry date, and graphically using survival curves.

SAMHSA/CSAT will use event history analysis to empirically classify, predict, and evaluate patterns of pathways to sanctions, relapse, or recidivism in the ATDC data. Event history analysis is a generalization of the single-event hazard analysis to multiple types of events that can each occur multiple times. Essentially, it consists of multiple hazard models describing processes in which individuals are in one of a set of mutually exclusive and exhaustive “states” at any point in time (Allison, 1984). Transitions between states are the “events” in the individual’s event history (e.g., the receipt of a reward or graduated sanction). Increasingly, applications are appearing in studies of behavioral interventions, including criminal justice. For example, Ventura et al. (1998) used event history analysis to assess whether case management provided to mentally ill offenders both in jail and after release from jail would reduce their recidivism. They found that receipt of jail-based case management, although not directly related to recidivism, significantly increased the probability of receiving community-based case management, which in turn was significantly associated with a lower probability of rearrest and a longer period before rearrest. The example is informative for “unpacking” the context of complex causal sequences, consistent with the exploration of intervention theory and mechanisms in the CMO model. For example, the event history analysis could determine the specific deterrent effect, if any, of sequences of judicial status hearings, sanctions, and consequences of sanctions. Single-event hazard analysis has been used in drug court evaluation (Banks and Gottfredson, 2004; Truitt et al., 2003) but multiple-event history analysis has not, to our knowledge.

First, participants will be classified in one of a pre-defined set of states at any given time during their observed ATDC history. The set of mutually exclusive and exhaustive pre-defined states will represent the “state space” for these participants (Allison, 1984). Second, the cumulative transitions between states across individuals will be expressed as a set of transition probabilities, followed by the computing and plotting of transition rates (probabilities across time). These will in turn be used to classify and characterize the universe of transition rates in the context of the multi-event drug court experience. Third, the transition rates will be analyzed as type-specific hazard functions in which events are distinguished by both origin and destination states. Like single hazard functions, they can be modeled as a function of time and explanatory variables, both constant (e.g., demographics, prior criminal history, and drug use) and time-varying (e.g., relapse, receipt of graduated sanctions). So, for example, research might show that different patterns of relapse, sanction, and reward – including the time lag between events and their consequences -- are predicted by different sets of explanatory variables. The results of this analysis will yield the predictors of transition patterns as the dependent variable. Finally, the transition patterns will be used as independent variables to model the participant outcomes reported during follow-up. These patterns can be characterized and input into the multilevel models like any other participant-level variables.

One objective of this evaluation is to assess the cost-effectiveness of ATDC’s provided by the 20 grantees. Cost-effectiveness analysis (CEA) is an economic evaluation technique for calculating the relative value of a program in terms of both costs and outcomes. In a CEA, the drug court program is compared with alternative programs or with the current practice (treatment as usual). The relative value of the drug court is captured by the cost-effectiveness ratio (CE ratio), shown below:

Finigan, Carey & Cox (2007) conducted a similar analysis in Multnomah County Drug Court. They found the total cost per drug court participant was $5,168, while the total cost per non-drug court participant was $6,560, creating a cost benefit of $1,392 per participant. Costs included arrest, booking, hearings, treatment, jail time, and probation. Drug court outcome costs, across the same categories, were then compared across drug court and non-drug court participants, including victimization costs. This resulted in an average savings of $12,218 per drug court participant. Also examined were the total outcome costs including victimizations across drug court graduates ($19,661) and all participants ($38,537), resulting in a savings of $18,876. This resulted in a cost-benefit ratio of $2.63 for every $1 spent on drug court. Using available estimates for costs of drug courts and outcomes including arrest, conviction, and treatment costs (Bhati, Roman & Chalfin, 2008; Finigan, Carey & Cox, 2007), plus determining appropriate measures of effectiveness through drug court interviews and review of administrative and program data, a cost-effectiveness analysis will be conducted. Within the CMO model, cost effectiveness can be considered first as an outcome. Once costs have been established, they can become part of the context.

The first step in a cost-effectiveness analysis is to identify the point-of-view (perspective) that the evaluation will take in order to measure costs and benefits associated with drug courts and alternative programs. SAMHSA/CSAT will adopt the societal viewpoint in order to identify all costs and effects of a drug court program and maximize the utility of the evaluation findings for policy-makers. Costs and benefits from a public program often spill over to others beyond the person receiving treatment. By adopting the societal perspective, SAMHSA/CSAT can include all costs and benefits that are attributable to the program, even if they do not involve the participant directly. For example, Carey and Finigan (2004) found that considerable savings were experienced by the Department of Corrections due to fewer subsequent arrests resulting in jail or prison time, averaging a cost benefit (cost savings) of $1,171 per drug court participant.

SAMHSA/CSAT will work closely with drug court grantees to determine the patterns regarding how participants move through the system. This will allow determination of the feasibility of applying existing cost evaluation data to the treatment drug court sites. In addition, it is essential to include both direct and indirect costs in a comprehensive cost calculation. Direct costs typically include those expenses in immediate support of the drug court program itself. Indirect costs typically are not immediately associated with the drug court program. SAMHSA/CSAT expects cost information to come from two sources: (i) The direct cost of labor involved in each drug court activity (judge’s time for the hearing) and (ii) The indirect costs associated with the supporting agency services plus other overhead costs (expressed as a percentage of direct costs). Summing these costs across all participants yields the gross cost of the program. The next step is to obtain the net costs, or cost savings, that will be used in cost-effectiveness analysis. Finally, effectiveness will be measured by conducting an in-depth evaluation of the objectives of all drug court programs. Common dimensions for measurement of success will be identified, and cost-effectiveness analyses based on these dimensions will be performed.

Conduct Integrated Analysis

In advance of conducting analyses, a comprehensive analysis plan will be developed. It will be closely tied to the evaluation questions and conceptual framework; and it will reflect the realities of the data, seeking to make the most of their strengths and compensate for their limitations. As described above, analyses that combine data at different levels, such as those focusing on the effects of court characteristics and services on participant outcomes, will entail multi-level modeling analyses that account for the clustering of one level within another.

Final Analysis and Reporting

Once the data are cleaned and in analysis-ready files, analyses will be conducted that are tied to the evaluation questions, identify lessons and best practices, report evaluation findings and recommendations based on them, and disseminate findings to diverse audiences.

Develop Lessons and Best Practices

Analyses that examine the relationship between grantee practices and results at the community and participant levels will yield valuable lessons and best practices. To make the most of this opportunity, the CMO model will be used to identify such lessons and practices in several different areas, ideally reflecting the major components of the conceptual framework. Such information will be useful to future cohorts of ATDC grantees and the field at large.

Sample Table Shells

Tables 1 and 2 illustrate two sample table shells. Table 1, part of the descriptive analysis, would show the frequency distribution of achieved implementation level for each of the identified Drug Courts. The achieved implementation levels will be derived from implementation ratings based on site visits, supplemented by archival sources such as progress reports and grantee applications. Table 2, part of the inferential analysis, would show the association between implementation level for one drug court key component and selected population outcomes, corrected for baseline differences and other potential confounders. The measure of association used is the gamma coefficient.¹ There are of course many other ways results could be presented, so these tables should be viewed only as examples of the kind of tabulation of results that will play a role in the analyses that will be used as a basis for assessing relationships between drug court component implementation and client outcomes.

Table 1. Sample Table Shell: Cross-Court Frequency Distribution of Key Component Implementation Level

Drug Court Key Component	Implementation level
	1	2	3	4
1. Alcohol and other drug treatment services are integrated with justice system case processing.	N (%)	N (%)	N (%)	N (%)
2. Using a non-adversarial approach, prosecution and defense counsel promote public safety while protecting participants’ due process.	N (%)	N (%)	N (%)	N (%)
3. Eligible participants are identified early and promptly placed in the drug court program.	N (%)	N (%)	N (%)	N (%)
4. Provide access to a continuum of alcohol, drug, and other related treatment and rehabilitation services.	N (%)	N (%)	N (%)	N (%)
5. Abstinence is monitored by frequent alcohol and other drug testing.	N (%)	N (%)	N (%)	N (%)
6. A coordinated strategy governs drug court responses to participants’ compliance.	N (%)	N (%)	N (%)	N (%)
7. Ongoing judicial interaction with each drug court participant is essential.	N (%)	N (%)	N (%)	N (%)
8. Monitoring and evaluation measure the achievement of program goals and gauge effectiveness.	N (%)	N (%)	N (%)	N (%)
9. Continuing interdisciplinary education promotes effective drug court planning, implementation, and operations.	N (%)	N (%)	N (%)	N (%)
10. Forging partnerships among drug courts, public agencies, and community-based organizations generates local support and enhances drug court effectiveness.	N (%)	N (%)	N (%)	N (%)

1 — Not or minimally implemented

2 — Partially implemented, significant shortcomings

3 — Partially implemented, minor shortcomings

4 — Fully implemented

Table 2. Sample Table Shell: Key Component Implementation Score by GPRA and Recidivism Outcomes

Outcome	Implementation level for Key Component X				Gamma (CI)
	1	2	3	4
30-day Drug Use	Mean (N)	Mean (N)	Mean (N)	Mean (N)	±0.xx (LCL, UCL)
Re-arrest Frequency	Mean (N)	Mean (N)	Mean (N)	Mean (N)	±0.xx (LCL, UCL)
Employment	Mean (N)	Mean (N)	Mean (N)	Mean (N)	±0.xx (LCL, UCL)
Mental Health Problems	Mean (N)	Mean (N)	Mean (N)	Mean (N)	±0.xx (LCL, UCL)

17. Display of Expiration Date

The expiration date for OMB approval will be displayed.

18. Exceptions to Certification Statement

This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions. The certifications are included in this submission.

B. Statistical Methods

1. Respondent Universe and Sampling Methods

Samples of Drug Courts and Participants

This cross-site evaluation examines the value-added of SAMHSA funds awarded in Fiscal Year 2008 to 20 existing drug courts for the expansion and/or enhancement of their current programs. All 20 drug courts will be included in the evaluation, and data will also be collected from 816 drug court clients enrolled in the SAMHSA funded expansion and/or enhancement program. All drug court participants who are administered a baseline GPRA instrument will be enrolled in the cross-site evaluation. There will therefore be no sampling within the pool of clients who enter programs after the enhanced/expanded programmatic changes have been implemented.

The data collection methods will include review of all the grantees’ applications and other documents, site visits to all drug courts, and cross-site analysis of all sites’ GPRA data from both intake, 6-month post-intake, and program discharge. SAMHSA/CSAT will supplement local data collection with a 6-month post-discharge follow-up interview and compile arrest record (“rap sheet”) data all for participants who are followed or located in drug court records during implementation of the evaluation.

Generalizability

The collection of data for all drug court participants enrolled during the evaluation will enable conclusions to be generalized to other SAMHSA funded drug courts, if the funding and drug court requirements are similar. Comparisons of the evaluation sample will be conducted with the outcomes in each court prior to the enhancement/expansion funding after checking for attrition bias. Conclusions about these comparisons will be generalizable regarding differences made by the ATDC funding, within the cautions raised by any historical change in the context of the drug court or the characteristics of participants that coincide with the introduction of the enhancement and/or expansions. This is known as the “history” threat to validity; attributing changes in outcomes to the effects of the drug court programs. This history threat is one key reason for collecting detailed information from each site on how drug court mechanisms and service system contexts have developed and changed during the comparison and implementation periods.

Statistical Adjustments

Statistical adjustments will be made in the cross-site analyses for the selection bias that can be introduced by attrition (dropouts, refusals, unavailability of archival records, or loss to follow-up). Characteristics of those followed at 6-months post-intake and 6-months post-discharge will be compared statistically to the characteristics of those lost to follow-up. If the number of statistically significant differences between the baseline and follow-up characteristics is small the participants who are not lost to follow-up can be considered an unbiased sample of those admitted to the drug court programs. If there are numerous (more than 10%) or very substantial baseline differences between the admitted and followed samples (an effect size greater than .75), statistical adjustments will be made for these differences in the analyses using propensity scores (described below) that predict membership in the followed or non-followed groups based on the set of measured baseline characteristics.

Statistical Adjustments in Multiple Comparisons

In addition to the overall longitudinal assessments of participants, the proposed design includes planned statistical adjustments for observed differences between the new drug court participants and each of several intact or constructed comparison groups. Comparison groups are needed to isolate program effects and determine the counterfactual: what would have happened to similar participants had they not experienced the enhanced or expanded treatment services? Gains or outcomes achieved by the ATDCs can be determined by comparing outcomes of interest between similar groups of drug court participants who did and did not receive enhanced/expanded treatment services. Random assignment is the best strategy to equally distribute the many unmeasurable variables that could present rival explanations of program effects. When random assignment is not possible, as is anticipated for this project, other available comparison groups can be constructed. Several strategies common to drug court evaluations will be considered for the final design. An appropriate comparison group will be determined when data collection begins and will be based on the available data and circumstances specific to each funded site. It is anticipated that multiple comparison groups will be utilized to address different evaluation questions. This, in conjunction with propensity score matching (discussed below), is believed to provide the strongest foundation for final conclusions with the available program and evaluation resources.

Local Contemporaneous Comparison Groups

One of the comparison groups that will be used is the local contemporaneous comparison group. This group contains individuals who were eligible for drug court but did not participate (either self-selected not to participate or could not participate due to a lack of available drug court funding). GPRA data will not be available for individuals who do not participate in the drug court. However, official criminal justice data will be available for both participants and non-participants. These data offer a number of advantages over the GPRA Criminal Justice items: they are independent of self-report biases, less vulnerable to follow-up loss, and available for a longer follow-up period (not limited to 6 months). They will support longitudinal analysis methods such as survival or event-history analysis that are more sensitive to degree of change and variations in change patterns over time. Potential matching characteristics for developing comparison groups may include severity of offense, age, race, and number of prior convictions. Their relationship to the drug court proceedings may be important for assessing selection bias. The size of these comparison groups and their baseline comparability to the participants will vary by site.

Local Historical Comparison Groups

SAMHSA/CSAT is accountable for justifying the value added from the CSAT initiative specifically, in addition to the effectiveness of the ATDCs in general. The most direct comparison group for addressing the value added for the CSAT initiative is the drug court participants in the 20 courts prior to the current grant award. However, there are limitations to this approach. Primarily, for ATDCs that have been in existence for only a year or slightly more, the participants’ drug court experience will be confounded with startup and implementation issues, which can potentially have a significant effect on the findings. Only one of the 20 grantees falls into this category. So as not to discard potentially useful data, the cross site evaluation team will collect pre-grant and, where available, pre-drug court criminal justice data from all 20 sites, and assess whether pre-grant outcomes were different than outcomes for newer programs.

Constructed Comparison Group of Non-ATDC Participants

While GPRA data would not be available for the local comparison groups, it will be available within the Services Accountability Improvement System (SAIS) data for other clients being served by CSAT programs other than ATDC. GPRA data are available for criminal offenders who are generally comparable to ATDC participants in important ways, including participation in drug treatment, but are not enrolled in a SAMHSA funded ATDC program.

SAMHSA/CSAT will therefore access non-ATDC GPRA data on intakes, follow-ups, and (where available) discharges. Matched comparison groups of clients residing in or near the 20 ATDC communities who are similar to identified groups of ATDC participants, but were not exposed to drug courts, will be created. Due to the GPRA data collection schedule at baseline, 6 months, and discharge, this will allow a comparison of substance abuse and related outcomes for up to 6 months of treatment (and longer when discharge data is available) with 3 groups: drug court participants who go on to graduate, those who terminate, and similar participants who did not enter drug court (identified using SAIS data).

Propensity Scoring

A matched comparison sample will be constructed of non-ATDC participants drawn from the SAIS/GPRA baseline data using propensity scoring, a recommended strategy in the drug court literature (GAO, 2005; Rempel, 2005). Introduced by Rosenbaum and Rubin (1984), propensity scoring is now widely used to analyze observational studies in the biomedical and behavioral health fields (Rosenbaum, 2002) and is most commonly used to reduce nonequivalence between two nonrandomly assigned groups. It is also a method for which Westat has provided important innovations (Orwin et al., 2004). Propensity scoring uses the full set of information in the measured baseline covariates to “predict” whether each case would be a member of the treatment or comparison groups. The matched group is then constructed by matching on the propensity scores rather than on selected sets of covariates themselves. This frees the matching process from its usual limitation of reliance on a small number of covariates and simplistic functional forms (such as regressions models with linear main effects only) to adjust for differences between the groups. Rather, a complex logistic regression model with interactions and higher–order terms can be fit at the propensity-scoring stage without concern about overparameterization, since the goal is simply to obtain the best estimated probability of group assignment from the observed covariates. When subsequently used to create matched pairs of ATDC and non-ATDC individuals, the propensity score carries all the information from the complex covariate model in a single variable, a number between 0 and 1 predicting group membership, and consumes only one degree of freedom. Thus the method can handle a large number of confounding variables, yet avoids the potentially adverse effects of multicollinearity on the stability of the estimates. The ability to test for balance (not possible with traditional methods) is particularly important for validating that the match was successful. To properly adjust standard errors for within-site clustering in the 20 sites, testing will be done within Westat’s own specialized software, WesVar. WesVar uses replicate weighting methods to calculate variances, thereby ensuring proper estimation of standard errors in clustered data (Westat, 2000).

Cost Data Sampling

In addition, the data needed to estimate the comparative costs of programs and components will be collected in a variety of ways. There are questions about the representativeness of the sample service costs on which the cost and cost effectiveness estimates will be constructed. SAMHSA/CSAT will work closely with drug court grantees to determine the patterns regarding how clients move through the system. This will allow determination of the feasibility of applying existing cost evaluation data to the treatment drug court sites. In addition, it is essential to include both direct and indirect costs in a comprehensive cost calculation. Direct costs typically include those expenses in immediate support of the drug court program itself. Indirect costs typically are not immediately associated with the drug court program. It is expected that cost information will come from two sources: the direct cost of labor involved in each drug court activity (judge’s time for the hearing) and the indirect costs associated with the supporting agency services plus other overhead costs (expressed as a percentage of direct costs). Summing these costs across all clients yields the gross cost of the program. The next step will be to obtain the net costs, or cost savings, that will be used in cost-effectiveness analysis. Finally, effectiveness will be measured by conducting an in-depth evaluation of the objectives of all drug court programs. Common dimensions for measurement of success will be identified and cost-effectiveness analyses will be performed based on these dimensions.

References

Government Accountability Office (GAO). (2005). Adult drug courts: Evidence indicates recidivism reductions and mixed results for other outcomes. United States Government Accountability Office, Report to Congressional Committees. GAO-05-219.

Orwin, R.G., Hornik, R.C., Judkins, D., Zador, P., Sridharan, S., and Baskin, R. (2004). Innovative design and analysis strategies in the evaluation of the National Youth Anti-Drug Media Campaign: Propensity scores and counterfactual projection weights in a national probability survey. Proceedings of the Federal Committee on Statistics and Methodology. Available at: http://www.fcsm.gov/03papers/Orwin.pdf.

Rempel, M. (2005). Recidivism 101: Evaluating the impact of your drug court. NY: Center for Court Innovation.

Rosenbaum, P.R. (2002). Observational studies. New York: Springer.

Rosenbaum, P.R., and Rubin, D.B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516-524.

Westat. (2000). WesVar® 4.2 User’s Guide. Rockville. MD: Westat.

2. Information Collection Procedures

The drug court team members (i.e., SAMHSA grantees) are required to participate in the cross-site evaluation as a stipulation of receiving funding from SAMHSA. Two site visits to each of the 20 drug courts over the three year project are planned and will provide the opportunity to collect information from the drug court team. These site visits will be arranged with the drug court project director and the cross-site evaluation team over the telephone. During these calls, it will be discerned (if it is not already known) when drug court sessions are held and what 4-day period is available for an on-site visit.

The self-report data from drug court clients will be collected at two points in time: focus groups and post-drug court interviews. Focus groups are planned at each site visit (20 sites and 2 visits for a total of 40 focus groups) to collect qualitative data from a small group (8-10) of drug court clients. These sessions will occur after clients have appeared before the judge and recruitment will occur on-site. The clients participating in the focus groups will be selected from the population of drug court clients. A sample of convenience will be selected – clients who are appearing before the judge on the day of the site visit will be asked by the drug court manager to participate in the focus group. Participants will be asked to sign a consent form (Attachment K) and will be compensated ($10 gift card) for their time and effort. A light snack and soft beverages will also be provided to the participants.

Individual survey data will be collected during a one time, in-person interview with drug court clients at 6-months post-discharge from the drug court. Discharge is defined as successful completion (i.e., graduation) or unsuccessful completion (e.g., dismissal by the judge, client drops out). Follow-up data will be collected after the clients are dismissed from the drug court and therefore SAMHSA/CSAT will need to track clients after they have been released from the drug court. Field interviewers who reside in close proximity to each of the 20 drug court locations will be employed for these in-person interviews. The project’s Field Operations Manager with work the field interviewers, and the contractor’s Telephone Research Center to contact evaluation participants and schedule interviews (Attachment L).

The interviews will take place in the drug court client’s home or another private location to ensure confidentiality, and interviews may occur in jail/prison for clients who have been incarcerated for failure to comply with the drug court program or arrested for a crime. The interview data to be collected from drug court clients will include substance use and criminal history data from the Government Performance and Results Act (GPRA) questionnaire (Attachment B), procedural justice (i.e., courtroom experience) (Attachment C), mental health screener (Attachments D and E), and treatment satisfaction index (Attachment F). Field interviewers will present each participant with a copy of the informed consent form (Attachment J) for his or her records, and will review the form with them. If the participant agrees to the interview, the field interviewer will have them sign another copy of the consent form which will mailed to the contractor in a separate envelope from the survey responses.

3. Methods to Maximize Response Rates

This evaluation includes data collection by the Drug Court grantees and the Cross-Site Evaluation Team. Grantees will collect GPRA data at baseline, 6 months post-baseline, and discharge/exit, while the Cross-Site Evaluation Team will collect these data at 6-months post- discharge/exit. The Evaluation Team will work closely with the ATDC sites to prepare for the data collection hand-off. This will include training grantees on tracking drug court clients for follow-up interviews, adding the Cross-Site Evaluation Team to the consent forms administered at intake, and maintaining a close working relationship with the drug court grantees to facilitate the handoff. All 20 sites (drug courts) are expected to participate in the evaluation, and an 80% follow-up rate for individual participants is expected to be achieved.

To maximize response rates, the Cross-Site Evaluation Team will contact drug court clients by letter every six weeks once they have been discharged from the drug court until the time of the 6-month post-discharge interview (Attachment M). In this letter, clients will be asked to return updated contact information to the Team in a postage paid envelope. Clients who respond and provide updated contact information will be mailed a $5 gift card to a local store (e.g., Wal-Mart). This will serve to initiate contact with the clients and increase the Cross Site Evaluation team’s ability to find clients to schedule the 6-month post-discharge interview at the appropriate time.

Drug court clients will be offered a $20 gift card to a local store for completing the 6-month post-discharge interview. The data being collected from this population, substance users who have been involved in the criminal justice system, is sensitive and remuneration for their time and responses is warranted. It is expected that response rates would be unacceptable without some form of remuneration. With the incentive and in line with the current GPRA follow-up rates of CSAT grantees of approximately 80%, SAMHSA/CSAT has have high confidence that an 80% follow-up rate at both time points is achievable for ATDC participants.

Drug court clients participating in the focus groups will be offered a $10 gift card to a local store. This data is also sensitive, but does not require identifying information to link to other data sources (e.g., criminal justice, drug court). Therefore a smaller incentive will be used.

4. Tests of Procedures

The instruments used in this evaluation have been previously tested and used by scholars and researchers in the field. The GPRA instrument is currently being used by SAMHSA and has prior OMB approval (OMB No. 0930-0208, Expiration Date 06/30/2012).

The 13-item Procedural Justice Questionnaire is taken from a recent study of drug courts in Baltimore conducted by Denise Gottfredson and colleagues. They obtained the Procedural Justice Questionnaire from “Hirst, A. (1999). Compliance with the Court System as a Function of Perceptions of Procedural Justice. Paper presented at the annual meeting of the American Society of Criminology, Toronto, Canada.” The results of Gottfredson et al.’s study were published and the citation is: Gottfredson, D.C., Kearley, B.W., Najaka, S.S., and Rocha, C.M. (2007). How Drug Treatment Courts Work: An Analysis of Mediators. Journal of Research in Crime and Delinquency, 44(1):3-35. This instrument has not been validated.

The Treatment Satisfaction Index (Version 1.0.3) was developed by Chestnut Health Systems. This instrument is considered to be a valid measure of early therapeutic alliance. The instrument was obtained from: Dennis, M.L., Titus, J.C., White, M.K., Unsicker, J.I., Hodgkins, D.V. (2002). Global Appraisal of Individual Needs (GAIN): Administration guide for the GAIN and related measures. Bloomington, IL: Chestnut Health Systems. [Online] Available at: www.chestnut.org/li/gain.

The Correctional Mental Health Screens for Men and Women were developed by Drs. Julian D. Ford and Robert L. Trestman at the University of Connecticut Health Center under a grant funded by the National Institute of Justice.  This questionnaire was designed for a criminal justice population.  The Men's and Women's screeners contain 6 identical questions (numbers 2, 3, 8, 9, 10, and 11 on the men's questionnaire and numbers 1, 2, 3, 5, 7, and 8 on the women's questionnaire).  The instrument has been validated (see National Institute of Justice. 2007. Mental Health Screens for Corrections. U.S. Department of Justice, Office of Justice Programs, National Institute of Justice).    

5. Statistical Consultants

The main statisticians for the Adult Treatment Drug Court Cross-Site Evaluation are Robert Orwin, Ph.D. and Joseph Sonnefeld, M.A. The expert panel members, including Michael Dennis, Ph.D., were also consulted on the design and data collection and are named above in Section A8. Westat and EMT staff members responsible for collecting the data are also listed below. The design was finalized in conjunction with the SAMHSA project officer, Angela Galloway.

Contractors:

Craig Love, Ph.D.

1650 Research Boulevard

Westat

Rockville, MD 20850

240-314-2443

Robert Orwin, Ph.D.

1650 Research Boulevard

Westat

Rockville, MD 20850

301-251-2277

Carrie Petrucci, Ph.D.

EMT Associates, Inc.
15720 Ventura Blvd., PH
Encino, CA 91436

818- 990-8301

Joseph Sonnefeld, M.A.

1650 Research Boulevard

Westat

Rockville, MD 20850

240-314-2522

Eileen Ahlin, M.A.

1650 Research Boulevard

Westat

Rockville, MD 20850

301-294-4471

Fred Springer, Ph.D.

EMT Associates, Inc.
230 Franklin Rd. #813
Franklin, TN 37064

615-585-2230

Mustafa Karakus, Ph.D.

1650 Research Boulevard

Westat

Rockville, MD 20850

301-292-2874

Project Officer:

Angela Galloway, M.A.

1 Choke Cherry Road, Room 1055

SAMHSA/CSAT

Rockville, MD 20857

240-276-1625

List of Attachments

Attachment A Interview Guide for Drug Court Staff

Attachment B Procedural Justice Questionnaire

Attachment C Mental Health Screener for Women

Attachment D Mental Health Screener for Men

Attachment E Treatment Satisfaction Index

Attachment F Focus Group Questionnaire

Attachment G Government Performance Results Act (GPRA)

Attachment H Institutional Review Board (IRB) Approval

Attachment I Drug Court Personnel Consent Form

Attachment J Drug Court Client 6-Month Post Discharge Consent Form

Attachment K Drug Court Client Focus Group Consent Form

Attachment L Telephone Script for 6-Month Post-Discharge Interview

Attachment M Drug Court Client Tracking Letter

1 Like the Pearson correlation coefficient, gamma varies from –1 to +1, with zero being no relationship, but unlike the Pearson correlation, does not assume that either the independent or dependent variable are measured as interval level variables. It therefore is appropriately used to estimate associations between ordered variables.

9