Targeted Capacity Expansion Grants for Jail Diversion Programs
1) Circumstances of Information Collection
The Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Mental Health Services (CMHS) is requesting a revision for approval from the Office of Management and Budget (OMB) for three years for the following CMHS Jail Diversion Targeted Capacity Expansion Initiative Forms; Baseline Interview (Attachment A), 6-Month Interview (Attachment B), 12-Month Interview (Attachment C), Arrest Data Collection Form (Attachment D), and Mental Health and Substance Abuse Service Use Data Collection Form (Attachment E). This clearance has been approved under OMB No. 0930-0277 which expires 5/31/2009. The requested revisions only affect the three interviews to be administered by the most recent round of TCE Grantees (those awarded in Spring 2006 and any subsequent awards) and include: 1) the addition of a Posttraumatic Stress Checklist (PCL) section to all three interviews, and 2) the re-administration of the D.C. Collaboration Study Violence and Trauma Screen (DC Trauma Screen) section at the 6- and 12-month follow-up interviews (for previous Grantees, this screen was only administered at Baseline). No other revisions have been made.
Since 2002, CMHS has funded five rounds of grants under the SAMHSA/CMHS Targeted Capacity Expansion (TCE) Grants for Jail Diversion program. These three year grants fund the diversion of individuals with mental illness from the criminal justice system to mental health treatment and appropriate support services. Grants awarded in 2002 and 2003 have reached the end of their three year cycle. Grants awarded in FY2004, FY2005, and FY2006 are currently collecting data under OMB No. 0930-0277. The grants awarded in FY2006 are expected to begin revised data collection in March 2007 pending OMB approval and any subsequent grants awarded will begin data collection thereafter. Therefore, OMB approval is requested for data collection by the jail diversion programs funded in FY2006 and FY2007 (with funding through 2010).
The TCE Grants for Jail Diversion Programs require a systematic federal scrutiny of outcomes through the Government Performance and Results Act (GPRA) of 1993 (OMB No. 0930-0208). GPRA mandates accountability and performance-based management by federal agencies. It focuses on results or outcomes in evaluating the effectiveness of federal activities and on measuring progress toward achieving national goals and objectives. All SAMHSA Grantees must comply with GPRA data collection and reporting requirements.
Historical Background
The TCE Jail Diversion Program is congressionally mandated under the Public Health Service Act, section 520G. It is coordinated with the Department of Justice's solicitation "Mental Health Court Grants Program," authorized in P.L. 106-515, Part V, Section 2201. The overall goal of this collaboration is to improve policy and practice for addressing the needs of persons with a mental illness or co-occurring mental health and substance abuse disorders who become involved with the criminal justice system. The term “jail diversion” refers to programs that divert individuals with mental illness and often co-occurring disorders in contact with the justice system from jail and provide linkages to community-based treatment and support services. The individual thus avoids or spends a significantly reduced time in jail and/or lockups on the current charge or on violations of probation resulting from previous charges. Over the past decades, jail diversion programs have been offered as a viable and humane solution to the criminalization and inappropriate criminal detention of individuals with mental disorders. Diverting certain individuals from jail to community-based mental health treatment has been heralded for its potential benefits to the criminal justice system, the community, and the diverted individual.
Grantees build service capacity by developing and implementing a strategic plan for creating a service delivery system for jail diverted persons, building the infrastructure to support the service delivery system, and providing treatment services directly or by arranging for them to be provided. Treatment services must be based on the best known practices and include case management, Assertive Community Treatment, medication management, integrated mental health and substance abuse treatment, psychiatric rehabilitation, and gender based trauma services. Grantees coordinate with social service agencies to ensure that life skills training, housing placement, vocational training, job placement, and health care are available to diverted persons.
The goals for the TCE Grants for Jail Diversion are to:
Divert persons with mental illness and/or co-occurring substance abuse disorders from jails to community based mental health;
Provide either directly or indirectly, treatment services that are based on best known practices; and
Promote the development of a comprehensive service delivery system.
The program also aims to improve access to and quality of treatment to persons from racial/ethnic minorities and rural settings and to foster cultural competence. Three of the main goals are to:
a) Create service linkages between individuals and groups that serve the targeted population (e.g., mental health and substance abuse service providers and criminal justice system personnel). This includes:
developing partnerships and coalitions among mental health, substance abuse and criminal justice systems to increase systems integration
developing specific linkages among key personnel in each system.
b) Undertake community outreach to communicate to the larger community the importance of mental health and the capacity of the jail diversion program to serve people with mental illness. Required activities include:
building consensus among stakeholders and potential stakeholders for the adoption, implementation, and evaluation of the jail diversion program
ensuring that services are available for the target population
ensuring that the community accepts the use of the services as beneficial.
c) Engage in program evaluation and dissemination to demonstrate program outcomes and the quality and completeness of services implementation. This includes:
collecting required Government Performance and Results Act (GPRA) data
obtaining, at minimum, an 80 percent response rate at each data collection point
disseminating program findings, including relevant materials directed to consumers, service providers, administrators, and community, state, and federal policy makers who need this type of knowledge.
In fiscal year (FY) 2002, 10 grants were awarded to support programs that divert persons with mental illness from the criminal justice system to community mental health and supportive services. Seven were added in FY 2003 and 3 more in FY 2004 for a total of 20 grants. Each Targeted Capacity Expansion Site is eligible to receive funding for up to 3 years. In addition, Grantees must provide a non-federal share of 25 percent (includes cash or in-kind fairly evaluated). Twelve grants were added in 2005 and 2006, and two more are anticipated in 2007. Across six years, an anticipated 34 grants will have been awarded.
The Technical Assistance and Policy Analysis (TAPA) Center's role with respect to the SAMHSA/CMHS TCE jail diversion Grantees
The TAPA Center was funded at the same time as the 2002 TCE jail diversion Grantees, as a result of the same SAMHSA/CMHS funding solicitation. The TAPA Center serves as the coordinating center for the 20 existing and 6 new CMHS TCE Jail Diversion Grantees and provides a number of services to them including:
Conducting site visits
Organizing and conducting an annual two-day meeting of all Grantees
Assisting sites in planning, goal setting, and process evaluations
Coordinating a multi-site evaluation of all Grantees, which includes, developing data collection instruments (including Government Performance Results Act data elements and other common data elements), conducting trainings, facilitating data collection and management, cleaning and analyzing data, coordinating site-specific evaluations, and producing reports
Providing technical assistance in all areas of program implementation and evaluation
Fostering and facilitating consumer involvement
Fostering and facilitating the development of gender-specific and trauma services
Grantees must evaluate the process of planning and implementing the program and participate in a cross-site evaluation of the impact of the program described later in this document. Data collection for Grantees of the Targeted Capacity Expansion Grants for Jail Diversion Programs is mandated under the program’s legislation: Public Health Service Act, Section 520G, 42 USC Sec. 290bb-38 “Grants for Jail Diversion Programs”.
Program participants are interviewed based mainly on Government Performance and Results Act (GPRA) measures at program enrollment (baseline), 6-months and 12-months.
In addition to GPRA measures, the interviews include the following measures:
DC Trauma Collaboration Study Violence and Trauma Screening to gauge traumatic events (Baseline only for FY2002 through FY2005 Grantees; all interviews for the FY2006 and subsequent Grantees)
Posttraumatic Stress Checklist to identify a specific set of trauma symptoms (All interviews only for the FY2006 and subsequent Grantees)
Colorado Symptom Index 1991 to gauge symptoms of mental illness (All interviews)
Perceived Coercion Scale (from MacArthur Mandated Community Treatment Survey) to enter jail diversion programs (Baseline and 12 months only)
Mental Health Statistics Improvement Program quality of life measures (6 and 12 months only)
Service use (6 and 12 months only)
In addition to data collected through interviews, Grantees collect the following information:
Events Tracking: This program captures the volume of screening and evaluation activities that jail diversion programs engage in to determine whom the program will serve.
Person Tracking: This program records basic information on all individuals who are diverted and served with grant funds. It also helps Grantees keep track of interview dates for those program participants who agree to take part in the evaluation.
Service Use: Grantees collect data on services provided from self-reported or from official sources, such as statewide/agency management information systems or other agency records about the types of services received following diversion.
Arrest and Jail Days Data: Grantees report arrest and jail days data collected from official sources, such as a statewide criminal justice database, or that have been tracked for themselves for one year prior and one year following diversion.
Data from the Baseline Interview (Attachment A), 6-Month Interview (Attachment B), and 12-Month Interview (Attachment C) instruments; the Arrest Data Collection Form (Attachment D); and the Mental Health and Substance Abuse Service Use Data Collection Forms (Attachment E) are reported to the Technical Assistance and Policy Analysis (TAPA) Center.
Purpose and Use of Information
The purpose and use of the tracking and outcome data collected under this program is to meet the congressional mandate to evaluate the program. TCE Grantees must submit tracking data bi-monthly and outcome data as collected (at least monthly). The tracking data will be used to measure eligibility screening activity and to monitor evaluation progress. Both of these uses allow for oversight of Grantee funding utilization. Outcome data, including interview and record review data, will be used to measure the success of the jail diversion programs through changes in mental health, substance use and criminal justice involvement measures.
SAMHSA/CMHS and the TAPA Center will use the TCE Initiative’s data collection information to report findings on:
The breadth and volume of activities (e.g. screening, assessment, evaluations) necessary to identify and enroll people for diversion.
The determination of those who are eligible/ineligible for diversion and their characteristics.
The biases in determining who gets diverted.
Services, including evidence based practices that are most effective in which settings among which populations.
Improvements expected over time as a result of
services received through jail diversion programs,
reduced arrests/less time spent in jail,
reduced substance use,
higher functioning/improved mental health and/or
improved physical health.
It is expected that the information collected through this evaluation will be of particular value to the Grantees, as well as to all levels of government and the private sector.
Measures Collected Through the TCE Initiative
There are three primary data sources utilized in this evaluation.
Interview Data – The baseline, 6- and 12- month interviews are administered by Grantee staff to consenting jail diversion program enrollees and are submitted to the TAPA Center for data entry The interviews are composed of the following sections with revised sections shaded:
The Government Performance and Results Act (GPRA) measures - Because these measures are approved under OMB No. 0930-0208, the program is not requesting approval of the burden for this instrument. However, the GPRA Client Outcome Measures constitute the main components of all three of the interviews, These measures are required by CMHS include questions related to the following areas:
Demographics (baseline interview only)
Education, employment, and income
Drug and Alcohol use
Family and living conditions
Crime and criminal justice status
Mental and physical health problems and treatment
The DC Trauma Collaboration Study Violence and Trauma Screening – The questions from this tool is collected at baseline for the FY2004 and FY2005 Grantees and will be collected at each of the interviews for the FY2006 and subsequent Grantees. This screen inquires about events that have been upsetting or stressful in the respondent’s life including the witnessing of violence and the experience of physical and/or sexual abuse. This tool was originally designed to be used by clinicians for determining if an individual’s trauma history warranted a clinical follow-up. In the evaluation its purpose is non-clinical and intended to provide a sense of the level and recentness of trauma in the jail diversion population.
Posttraumatic Stress Checklist – This scale measures the prevalence and severity of posttraumatic stress symptoms based on DSM-IV criteria. The addition of this scale to each of the three interviews will provide information about the extent of trauma symptoms, as well as the incidence rates of trauma and re-trauma. These questions are included in all three interviews for the FY2006 and subsequent Grantees only.
Perceived Coercion Scale (from MacArthur Mandated Community Treatment Survey) – This scale contains questions that ask about how one felt about entering the jail diversion program at baseline and about how one felt about receiving outpatient mental health services at 12 months. These questions are not included as part of the 6 month interview.
Mental Health Statistics Improvement Program (MHSIP) – This section includes a subset of items from the full MHSIP Consumer Survey. The full survey was designed to assess access to treatment, quality/appropriateness of treatment, general satisfaction, consumer perceptions of the effectiveness of services, and the cultural sensitivity of care. The items used in the instruments inquire about how an individual is handling daily life and the result of services received in the jail diversion program. They are included in the 6 and 12 month interviews only.
Colorado Symptom Index 1991 – This scale asks a series of questions about the occurrence (in the past month) of symptoms related to anxiety, depression, disturbed thought processes and perceived interference with normal activities in order to gauge the individual’s psychiatric symptoms status. These questions are included in all three interviews.
Services Used – This section was designed to obtain very basic service use information to be used as a starting point by the data collection staff in conducting the Service Use record review (described below). Grantee staff collects this information at 6 and 12 months only.
Interviewer Observation Questions – Asks the interviewers to report on the respondent’s understanding, cooperativeness and accurateness during the interview process and is collected after each of the three interviews. These questions provide some information regarding the validity of the data collected.
Benefits of Interview Revisions (FY2006 Grantees and subsequent Grantees only) – The benefits of including the PCL and DC Trauma Screen in all three interviews includes the following:
Identification of specific trauma symptoms – The PCL identifies a specific set of symptoms that can be directly linked to trauma; symptoms that are not otherwise measured by the TCE evaluation.
Examination of individual change across the participation period – Including both the PCL and the DC Trauma Screen in each of the three interviews will allow the TCE Initiative to draw additional conclusions about the relationships and roles of trauma in recovery and about the extent and changing patterns of trauma in the jail diversion population.
Informing treatment plans in jail diversion programs – Information obtained through administering the two trauma measures at each of the interviews will help improve jail diversion program design (and hence benefit future jail diversion program participants) by: 1) allowing for the identification of individual patterns of trauma and re-trauma across the participation period (using the DC Trauma Screen), and 2) linking trauma-related symptom types and severities directly to specific treatment plans.
Note, The additional trauma sections are not being included in the interviews of previous awarded Grantees because: 1) the FY2002, FY2003, and FY2004 TCE Grantees have completed, or will soon complete, data collection, and 2) the FY2005 Grantees are ready to begin (or have begun) data collection with the earlier, OMB approved version.
Tracking Data
Events Tracking Data – The Events Tracking Program is designed to help the funded Jail Diversion TCE sites and the TAPA Center count the number of people considered for jail diversion program eligibility and to measure the assessments conducted to screen those potential enrollees. In addition, information on individuals (e.g. charges, sex, race, age) involved in each event is entered into the program (this information is not linked to individuals and is thus not identifiable). Many programs have multiple layers of assessment before enrolling a client in a jail diversion program (though not all sites have each layer), including:
Initial screening, to determine mental illness and overall potential eligibility for the diversion program. This is usually done by a police officer, jail nurse, booking officer, pretrial services worker or other jail diversion staff person.
Subsequent assessment, usually done by jail diversion case manager, boundary spanner, or non-Ph.D./M.D. clinician, and often focuses on clinical issues.
Subsequent Evaluation, usually an intensive psychiatric evaluation done by a Ph.D. clinical psychologist or psychiatrist.
Court Decision (in post-booking program), where the court accepts the plan negotiated between client, legal defense and prosecutor (or other relevant parties).
All of the data collected through the Events Tracking Program is collected and entered by Grantee staff using information obtained either directly or indirectly from the potential diversion program enrollees.
Person Tracking Data - The Person Tracking program helps funded Jail Diversion TCE sites and the TAPA Center to keep track of individuals’ Baseline, 6-Month and 12-Month interview dates and to record basic information on all individuals who are diverted. The basic information gathered on each diverted individual includes:
Demographics
Diagnosis
Charge level and category
Point of diversion (e.g. pre-booking, post-booking, probation violation)
Condition of diversion (e.g. deferred prosecution, condition of bail, deferred sentencing, etc.)
Target arrest/incident date, release date, program enrollment date
All of the data collected through the Person Tracking Program is collected and entered by Grantee staff using information obtained either directly or indirectly from the diversion program enrollees.
Record Review Data
Service Use data – The Mental Health and Substance Abuse Service Use Data Collection form (Attachment E) collects information from official sources such as statewide/agency management information systems or other agency records about the types of services received following diversion if available. Types of treatment include: emergency room services, other crisis services (e.g. mobile crisis services), psychiatric inpatient/hospital services, outpatient services, case management services, medication management/monitoring, residential treatment/community living arrangements, detoxification services, vocational/rehabilitation services, community support services (e.g. homeless outreach services, representative payee services), and jail services. Service use data is collected 6 months post-baseline for all major service providers and 1 year post-baseline for all hospitalizations and emergency room usage.
Arrest and Jail Days data – The Arrest History Data Collection Form collects pre- and post-diversion arrest information from official sources (such as a statewide criminal justice database). Information collected includes dates of arrests, charges, and jail days occurring one year prior and one year following diversion.
All record review data is collected by Grantee staff through official sources (i.e., directly from service providers or criminal justice databases). The completed forms are submitted to the TAPA Center for data entry.
Data collected through this evaluation will have an impact on municipalities in adopting jail diversion strategies through the extension of ideas, concepts, and program models learned from this and other SAMSHA/CMHS funded programs. Information coming out of this evaluation will also be relevant to the 34 Grantees, all levels of government, and the private sector. SAMHSA/CMHS and the TAPA Center are planning to disseminate findings to the field through national conference presentations, papers, and journal articles. This dissemination plan will provide municipalities with information regarding development, implementation, and outcomes of jail diversion programs. For the initiative’s Grantees, the collected data will be particularly informative in regard to seeking support from other sources once their SAMHSA/CMHS funding expires.
3) Use of Information Technology
Grantees collect this data from official sources or self-report data from their programs and submit it to the TAPA Center. This data is reported through an electronic database system or through paper copies. Resulting compiled data is used to provide information of interest to policy makers, researchers, and communities engaged in developing jail diversion programs.
4) Efforts to Identify Duplication
This data collection is significant only to this program and is not collected anywhere else.
5) Involvement of Small Entities
There is no significant impact on small entities or small businesses.
6) Consequences if Information Collected Less Frequently
Participation in the cross site evaluation by program participants is voluntary. Each jail diversion program participant is approached to request their consent for participation in the evaluation. Participants who grant consent are interviewed at Baseline (within 7 days of enrollment into the jail diversion program), at 6 months from the Baseline interview (within a 60-day window), and at 12 months (within a 60-day window). The arrest data collection is captured beginning 1 year after diversion. The service use data are collected beginning 6 months after diversion. Completed interview forms are submitted to the TAPA Center monthly with completed record review forms submitted at least once annually.
7) Consistency with the Guidelines in 5 CFR 1320.5(d)(2)
This information collection fully complies with 5 CFR 1320.5(d)(2).
8) Consultation Outside the Agency
The FR notice required by 5 CFR 1320.8(d) soliciting comments on the information was published on August 28, 2006, Vol. 71, No. 166, p. 50934-50935. There were no public comments received.
SAMHSA/CMHS, the TAPA Center, and Grantees (including evaluators, program directors/staff, and consumer representatives) awarded TCE Grants for Jail Diversion Programs in 2002 participated in the design of the multi-site evaluation and selection of measures. The TAPA Center provides Grantee sites with trainings and supportive materials on each evaluation component as well as participant protections. Annual meetings between Grantees, federal project officers and the TAPA Center staff review evaluation components and collection efforts.
Name/Title |
Address |
Contact Information |
Henry Steadman, Ph.D. Director National GAINS Center |
Policy Research Associates 345 Delaware Avenue Delmar, NY 12885 |
Phone: (518) 439-7415 x229
|
David Morrissette, Ph.D. Government Project Officer |
Center for Mental Health Services, SAMHSA |
Phone: (240) 276-1912 |
Joseph Morrissey, Ph.D. Deputy Director Cecil G. Sheps Center for Health Services Research |
The University of North Carolina at Chapel
Hill |
Phone: 919-966-5829 |
Karen Cusack, Ph.D. Postdoctoral Fellow Cecil G. Sheps Center for Health Services Research |
The University of North Carolina at Chapel
Hill |
Phone; 919-966-6725 |
Organizations and individuals that reviewed the additional sections include the following:
The FY2006 Grantees were also provided with drafts of the full interviews, including the additional sections, so that they could make comments.
9) Payment to Respondents
Individual Grantees may provide incentives to program enrollees to participate in the data collection efforts. Without providing these small incentives, clients would be significantly less likely to participate in the initial interview and/or subsequent follow-up interviews. Examples of incentives used include bus passes, grocery store and restaurant vouchers, phone calling cards, and clothing. Often incentive items are received as donations from local businesses. In cases where cash incentives are used, the amount provided does not exceed $30. Further, each Grantee must undergo human subjects review by an Institutional Review Board (IRB) with prisoner certification, consistent with Protection of Human Subjects Regulations (45 CFR 46) to receive approval for use of any incentives proposed to be given to respondents in exchange for their participation. These incentives are only provided for individual respondents, and do not apply to data collected from other agencies, programs or service providers affiliated with the Grantee site.
10) Assurance of Confidentiality
SAMHSA will not receive any individual level data. The TAPA Center provides trainings via conference call to all Grantees regarding appropriate procedures for meeting and maintaining privacy protection. These trainings include information on requesting a Certificate of Confidentiality and Prisoner Certification if obtaining IRB approval, obtaining appropriate releases from participants that comply with state and federal law, including the Health Insurance Portability and Accountability Act (HIPAA), and insuring privacy protection from interviewers and other Grantee staff. In addition to the trainings, the TAPA Center provides the Grantees with materials that cover all of the information presented in the trainings with the intention that they be used by current Grantee staff for their own reference and to train any new staff regarding privacy protection procedures.
11) Questions of a Sensitive Nature
There are questions of a sensitive nature such as drug and alcohol use that has been approved as GPRA measures under the OMB approval No. 0930-0208. The purpose of data collection is to evaluate the effectiveness of jail diversion programs. Program participants are approached to request their consent for participation. Each Grantee must develop procedures to obtain informed consent. TAPA provides an interviewer training that addresses how to handle questions of a sensitive nature. TAPA also encourages Grantees to devise a set of procedures that outline how interviewers should handle potentially dangerous or unsafe situations in order to ensure their own physical safety. An example of a Consent Form currently in use may be found in attachment F.
12) Estimates of the Annualized Hour Burden
The total amount of time that is estimated for completion of the client interview, record management by Grantee staff, and data submissions by the Grantees is 2,120 hours in CY2007, 2,170 hours in CY2008, 518 hours in CY2009, and 93 hours in CY2010. The annualized hourly costs to respondents are estimated be $23,402 in CY2007, $23,138 in CY2008, $6,126 in CY2009, and $1,130 in CY2010. The decrease in the annualized burden estimates for the TCE Initiative is due to fact that each round of Grantees (FY2004, FY2005, FY2006, and any subsequent) will reach the end of their grants at different points across the three year grant cycles. The burden estimates, summarized in the following table, are based on the reported experience of previous SAMSHA/CMHS Grantees and contractors in compiling, completing, and reporting this same data for previous funding rounds of this grant program. More senior Grantee staff is expected to handle the data extraction and submission at an average salary of $25/hour (as estimated for SAMHSA’s Treatment Episode Data Set (TEDS), OMB No. 0930-0106). The minimum wage rate of $5.15 was used to calculate the hourly burden for client interviews. The estimated wage rate of $15 for Grantee staff who conducts record management is based on the experience of the GPRA Services (OMB No. 0930-0208).
CY 2007 Annual Reporting Burden |
|||||||
Data Collection Activity |
Number of Respondents |
Responses per Respondent |
Total Responses |
Average Hours per Response |
Total Hour Burden |
Hourly Rate |
Total Hour Cost |
Client Interviews for FY2004 and FY2005 Grantees: |
|||||||
Baseline (at enrollment) 1 |
330 |
1 |
330 |
0.75 |
248 |
$5.15 |
$1,277 |
6 months |
270 |
1 |
270 |
0.75 |
203 |
$5.15 |
$1,045 |
12 months |
210 |
1 |
210 |
0.75 |
158 |
$5.15 |
$814 |
Sub Total |
810 |
|
810 |
|
609 |
|
$3,136 |
Client Interviews for FY2006 and anticipated FY2007 Grantees (includes additional trauma sections): |
|||||||
Baseline (at enrollment) 1, 2 |
220 |
1 |
220 |
.83 |
183 |
$5.15 |
$942 |
6 months 2 |
90 |
1 |
90 |
.92 |
83 |
$5.15 |
$427 |
12 months 2 |
10 |
1 |
10 |
.92 |
9 |
$5.15 |
$46 |
Sub Total |
320 |
|
320 |
|
275 |
|
$1,416 |
Record Management by FY2004, FY2005, FY2006, and anticipated FY2007 Grantee Staff 2: |
|||||||
Events Tracking |
15 |
2,400 3 |
36,000 |
.03 |
1,080 |
$15 |
$16,200 |
Person Tracking |
15 |
50 |
750 |
.10 |
49 4 |
$15 |
$735 |
Service Use 5 |
9 |
25 |
225 |
.17 |
38 |
$15 |
$570 |
Arrest History 5 |
9 |
25 |
225 |
.17 |
38 |
$15 |
$570 |
Sub Total |
48 |
|
37,200 |
|
1,205 |
|
$18,075 |
FY2004, FY2005, FY2006, and anticipated FY2007 Grantees 2: |
|||||||
Interview and Tracking data submission |
15 |
12 |
180 |
.17 |
31 |
$25 |
$775 |
OVERALL TOTAL: |
1,193 |
|
38,510 |
|
2,120 |
|
$23,402 |
1 – Only those program enrollees agreeing to participate in the evaluation receive a Baseline interview. 2 – The anticipated FY2007 Grantees are not expected to begin any data collection until January 2008 due to the 8 month planning period. The FY2006 Grantees will not begin 6-month data collection until the middle of CY2007 and will not begin 12 month data collection until the end of CY2007. 3 – The number of responses per respondent for the Events Tracking depends on the design of the jail diversion program and can range from a single screening for eligibility to four separate screenings; here 2,400 responses represents the average number of responses per respondent based on the experience of the previous Grantees. 4 – This estimate is an added burden proportion which is an adjustment reflecting the extent to which programs typically already collect the data items. The formula for calculating the proportion of added burden is: total number of items in the standard instrument, minus the number of core items currently included, divided by the total number of items in the standard instrument. For the Person Tracking program the burden estimate was calculated as follows: 75 times 0.65 (the proportion of added burden) = 49. 5 – Record management forms (Service Use and Arrest History) are only completed for those evaluation participants who receive both a Baseline interview and at least one follow-up (6 and/or 12 month) interview. Hence, the FY2006 Grantees will not begin collecting these data until CY2008 (the second year of data collection) and the FY2007 Grantees will not begin collecting these data until CY2009.
|
|||||||
CY 2008 Annual Reporting Burden |
|||||||
Data Collection Activity |
Number of Respondents |
Responses per Respondent |
Total Responses |
Average Hours per Response |
Total Hour Burden |
Hourly Rate |
Total Hour Cost |
Client Interviews for FY2005 Grantees: |
|||||||
Baseline (at enrollment) 1 |
220 |
1 |
220 |
.75 |
165 |
$5.15 |
$850 |
6 months |
180 |
1 |
180 |
.75 |
135 |
$5.15 |
$695 |
12 months |
140 |
1 |
140 |
.75 |
105 |
$5.15 |
$541 |
Sub Total |
540 |
|
540 |
|
405 |
|
$2,086 |
Client Interviews for FY2006 and anticipated FY2007 Grantees (includes additional trauma sections): |
|||||||
Baseline (at enrollment) 1 |
300 |
1 |
300 |
0.83 |
249 |
$5.15 |
$1,282 |
6 months |
210 |
1 |
210 |
0.92 |
193 |
$5.15 |
$994 |
12 months |
150 |
1 |
150 |
0.92 |
138 |
$5.15 |
$711 |
Sub Total |
660 |
|
660 |
|
580 |
|
$2,987 |
Record Management by FY2005, FY 2006, and anticipated FY2007 Grantee Staff: |
|||||||
Events Tracking |
14 |
2,400 2 |
33,600 |
0.03 |
1,008 |
$15 |
$15,120 |
Person Tracking |
14 |
50 |
700 |
0.10 |
46 3 |
$15 |
$690 |
Service Use 4 |
12 |
25 |
300 |
0.17 |
51 |
$15 |
$765 |
Arrest History 4 |
12 |
25 |
300 |
0.17 |
51 |
$15 |
$765 |
Sub Total |
52 |
|
34,900 |
|
1,156 |
|
$17,340 |
FY2005, FY2006, and anticipated FY2007 Grantees: |
|||||||
Interview and Tracking data submission |
14 |
12 |
168 |
0.17 |
29 |
$25 |
$725 |
OVERALL TOTAL: |
1,266 |
|
36,268 |
|
2,170 |
|
$23,138 |
1 – Only those program enrollees agreeing to participate in the evaluation receive a Baseline interview. 2 – The number of responses per respondent for the Events Tracking depends on the design of the jail diversion program and can range from a single screening for eligibility to four separate screenings; here 2,400 responses represents the average number of responses per respondent based on the experience of the previous Grantees. 3 – This estimate is an added burden proportion which is an adjustment reflecting the extent to which programs typically already collect the data items. The formula for calculating the proportion of added burden is: total number of items in the standard instrument, minus the number of core items currently included, divided by the total number of items in the standard instrument. For the Person Tracking program the burden estimate was calculated as follows: 70 times 0.65 (the proportion of added burden) = 46. 4 – Record management forms (Service Use and Arrest) are only completed for those evaluation participants who receive both a Baseline interview and at least one follow-up (6 and/or 12 month) interview. Hence, the FY2007 Grantees will not begin collecting these data until CY2009 (the second year of data collection).
|
|||||||
CY 2009 Annual Reporting Burden 1 |
|||||||
Data Collection Activity |
Number of Respondents |
Responses per Respondent |
Total Responses |
Average Hours per Response |
Total Hour Burden |
Hourly Rate |
Total Hour Cost |
Client Interviews for FY2006 and anticipated FY2007 Grantees (includes additional trauma sections) 1: |
|||||||
Baseline (at enrollment) 2 |
70 |
1 |
70 |
0.83 |
58 |
$5.15 |
$299 |
6 months |
70 |
1 |
70 |
0.92 |
64 |
$5.15 |
$330 |
12 months |
58 |
1 |
58 |
0.92 |
53 |
$5.15 |
$273 |
Sub Total |
198 |
|
198 |
|
175 |
|
$901 |
Record Management by FY2006 and anticipated FY2007 Grantee Staff 1: |
|||||||
Events Tracking |
8 |
2,400 3 |
9,600 |
0.03 |
288 |
$15 |
$4,320 |
Person Tracking |
8 |
50 |
202 |
0.10 |
13 4 |
$15 |
$195 |
Service Use 5 |
8 |
25 |
98 |
0.17 |
17 |
$15 |
$255 |
Arrest History 5 |
8 |
25 |
98 |
0.17 |
17 |
$15 |
$255 |
Sub Total |
32 |
|
9,998 |
|
335 |
|
$5,025 |
FY2006 and anticipated FY2007 Grantees 1: |
|||||||
Interview and Tracking data submission |
8 |
12 |
48 |
0.17 |
8 |
$25 |
$200 |
OVERALL TOTAL: |
238 |
|
10,244 |
|
518 |
|
$6,126 |
1 – Grant funding for the FY2006 Grantees ends April 30, 2009. Therefore, all burden numbers are accordingly prorated for these sites. 2 – Only those program enrollees agreeing to participate in the evaluation receive a Baseline interview. 3– The number of responses per respondent for the Events Tracking depends on the design of the jail diversion program and can range from a single screening for eligibility to four separate screenings; here 2,400 responses for the FY2006 Grantees (and 800 responses for the FY2007 Grantees) represents the average number of responses per respondent for the period based on the experience of the previous Grantees. 4– This estimate is an added burden proportion which is an adjustment reflecting the extent to which programs typically already collect the data items. The formula for calculating the proportion of added burden is: total number of items in the standard instrument, minus the number of core items currently included, divided by the total number of items in the standard instrument. For the Person Tracking program the burden estimate was calculated as follows: 20 times 0.65 (the proportion of added burden) = 13. 5 – Record management forms (Service Use and Arrest) are only completed for those evaluation participants who receive both a Baseline interview and at least one follow-up (6 and/or 12 month) interview. |
|||||||
CY 2010 Annual Reporting Burden (calculated up to the anticipated grant end date of April 30, 2010) |
|||||||
Data Collection Activity |
Number of Respondents |
Responses per Respondent |
Total Responses |
Average Hours per Response |
Total Hour Burden |
Hourly Rate |
Total Hour Cost |
Client Interviews for FY2006 and anticipated FY2007 Grantees (includes additional trauma sections): |
|||||||
Baseline (at enrollment) 1 |
0 |
1 |
0 |
0.83 |
0 |
$5.15 |
$0 |
6 months |
20 |
1 |
20 |
0.92 |
18 |
$5.15 |
$93 |
12 months |
15 |
1 |
15 |
0.92 |
14 |
$5.15 |
$72 |
Sub Total |
35 |
|
35 |
|
32 |
|
$165 |
Record Management by FY2006 Grantee Staff: |
|||||||
Events Tracking |
2 |
800 2 |
1,600 |
0.03 |
48 |
$15 |
$720 |
Person Tracking |
2 |
17 |
34 |
0.10 |
2 3 |
$15 |
$30 |
Service Use 4 |
2 |
8 |
16 |
0.17 |
3 |
$15 |
$45 |
Arrest History 4 |
2 |
8 |
16 |
0.17 |
3 |
$15 |
$45 |
Sub Total |
8 |
|
1,666 |
|
56 |
|
$840 |
FY2006 Grantees: |
|||||||
Interview and Tracking data submission |
2 |
4 |
32 |
0.17 |
5 |
$25 |
$125 |
OVERALL TOTAL: |
45 |
|
1,733 |
|
93 |
|
$1,130 |
1 –Since enrollment is anticipated to have ended for these Grantees by the end of CY2009 there is no Baseline burden in CY2010. 2 – The number of responses per respondent for the Events Tracking depends on the design of the jail diversion program and can range from a single screening for eligibility to four separate screenings; here 800 responses represents the average number of responses per respondent for the period based on the experience of the previous Grantees. 3– This estimate is an added burden proportion which is an adjustment reflecting the extent to which programs typically already collect the data items. The formula for calculating the proportion of added burden is: total number of items in the standard instrument, minus the number of core items currently included, divided by the total number of items in the standard instrument. For the Person Tracking program the burden estimate was calculated as follows: 3 times 0.65 (the proportion of added burden) = 2. 4 – Record management forms (Service Use and Arrest) are only completed for those evaluation participants who receive both a Baseline interview and at least one follow-up (6 and/or 12 month) interview. |
|||||||
Client Interview Data Collection -
For the forms currently approved under OMB No. 0930-0277, there are 37 baseline interview questions and 41 follow-up interview questions (including record management items and excluding GPRA) which will take approximately 45 minutes per enrollee/participant to administer at each of the data collection points. For the revised forms (to be used by the FY2006 and subsequent Grantees only), there are 54 baseline interview questions and 66 follow-up interview questions (including record management items and excluding GPRA). The baseline will take approximately 50 minutes per enrollee/participant to administer and the follow-up interviews will take approximately 55 minutes each to administer.
Record Management by Grantee staff -
There are four management burdens placed on the Grantee staff:
Events Tracking data are managed and entered by the Grantee staff. These data provide counts of the number of people considered for jail diversion program eligibility and measures the number of assessments conducted to screen those potential enrollees. In addition, information on individuals (e.g. charges, sex, race, age) involved in each event is entered into the tracking program (this information is not linked to individuals and is thus not identifiable). All of this information is generally collected through the normal course of business so that the only burden on the staff is data entry into the tracking program. It is estimated that it should take approximately 2 minutes to enter these forms for each participant.
Person Tracking data are also managed and entered by the Grantee staff. The Person Tracking data helps Grantees to keep track of individuals’ interview dates and to record basic information on all individuals who are diverted. Collecting and entering this information should take approximately 6 minutes for each participant. Of the 17 items, however, 6 are commonly collected by the providers. The resulting Added Burden Proportion is the total number of items in the standard instrument, minus the number of core items currently included, divided by the total number of items in the standard instrument which calculates as follows: (17-6)/17, or 0.65.
Service Use forms collect information about the types and number of services received following diversion and is obtained directly from provider records. These forms are estimated to take approximately 10 minutes to complete for each participant.
Arrest History forms collect pre- and post-diversion arrest information from provider records and include dates of arrests, charges and jail days occurring one year prior and one year following diversion. These forms are estimated to take approximately 10 minutes to complete for each participant.
Grantee Data Submissions –
Grantees are responsible for submitting data to the TAPA Center regularly. For Events and Person Tracking data this involves extracting and emailing their data bi-monthly. For Interview and Record Review data this involves mailing the completed forms at least monthly. It is estimated that these submissions take approximately 10 minutes monthly.
The averages for the four years of evaluations are 686 respondents, 21,869 total responses, and 1,225 hours of burden.
13) Estimates of Annualized Cost Burden to Respondents
There are no startup or capital costs, nor are there maintenance costs to the respondents.
14) Estimates of Annualized Cost to the Government
The TAPA Center coordinates, monitors, collects, reports and analyzes the data provided by the Grantees and submits a monthly report to the Government project officer. The Center also provides Grantee sites with training and supportive materials on each evaluation component as well as participant protections. The TAPA Center’s total budget for its evaluation activities is $363,038. The Federal Government employee (GS-14, $77,793) expends 20% of time overseeing the TAPA Contract, equaling $15,200.
15) Changes in Burden
Currently there are 1,373 hours in the OMB inventory. The program is requesting 1,225 hours. This program change is due to the reduction of 148 hours which includes the burden related to the addition of the six FY2006 Grantees and anticipated two FY2007 Grantees, as well as the burden related to the inclusion of additional trauma sections to the interviews administered by the FY2006 and anticipated FY2007 Grantees. The reason that the burden decreases, instead of increases, due to these additions is because the average burden includes a decreasing number of Grantees participating in the evaluation across the four year period.
16) Time Schedule, Publication and Analysis Plans
16.a. Time Schedule
Tasks Dates
OMB Approval: Pending
Data Collection: Immediately upon OMB approval
Data Collection Ends: April 2010
Analysis of Data: May 2010
16.b. Publication Plans
Tracking and outcome data will be collected through the TAPA Center from each TCE Initiative Grantee. Data will be reported to Congress regarding program performance as specified in the SAMHSA Budget Justification report. In addition, data summaries will be presented at annual Grantee meetings in order to provide a performance overview of the entire group of attending Grantees. Furthermore, the TAPA Center plans to disseminate information related to the development, implementation and outcomes of this initiative’s jail diversion programs through journal articles, monographs/fact sheets and national conferences.
16.c. Analysis Plans
The primary purpose of the TCE Initiative’s evaluation data collection effort is to provide program monitoring and oversight. There are three categories of evaluation questions that this initiative expects to address through its data analysis: Descriptive, Individual Change and Overall Outcomes.
Descriptive– This type of analyses will consist of counts, frequency distributions and basic aggregate tests (such as t-tests and Chi-square tests) using Tracking, Interview and Service Use data. These analyses will address questions such as:
What volume of activities (e.g. screening, assessment, evaluations) goes into identifying people for diversion? (Events Tracking)
How many people are determined eligible/ineligible for jail diversion and what are their characteristics (demographic and criminal justice)? (Events Tracking)
How many people are enrolled in the diversion programs and what are their characteristics (demographic, criminal justice and mental health)? (Person Tracking)
How many program enrollees agree to participate in the evaluation and what are their characteristics (demographic, education, employment, drug/alcohol use, criminal justice and mental/physical health)? (Interviews)
What services do people who are diverted receive? (Service Use)
As required, SAMSHA/CMHS reports on the characteristics of the participants seen in its grant portfolios to the Department of Health and Human Services and the Office of Management and Budget. Also CMHS will provide SAMHSA with program monitoring reports to use for performance review, improvement and oversight.
Individual Change – Analysis of these data will consist of a pre/post measurement methodology so that the individual acts as his own control. This methodology will be implemented in the following manner:
For interview data, consenting participants will receive a baseline interview within seven days of enrollment in the jail diversion program followed by 6 and 12 month follow-up interviews. Change within each participant across the time period will then be determined by examining the difference in scores on the mental health, substance use and other outcome measures between the baseline and 6 and 12 month interviews.
For Arrest History, criminal justice data will be collected both one year prior and one year post enrollment in the diversion program. Change within each participant across time will then be determined by comparing the number and type of criminal justice events and the amount of jail time occurring before and after enrollment.
Once individual difference scores are calculated, central tendency measures and frequency distributions will be utilized to examine how individuals have changed over the 6 and 12 months on specific measures. Tables will be constructed to summarize participant outcome changes across 6 and 12 months.
Overall Outcomes – The analyses discussed above will provide insight into the process behind jail diversion programs and how these programs are affecting individuals on specific measures. The TCE Initiative is, however, also interested in examining the following broader evaluation question that considers how the many different levels work together:
Do people who are diverted improve over time as a result of services received through the jail diversion program as measured by;
reduced arrests/less time spent in jail
reduced substance use
obtained housing
higher psychological functioning
improved mental health and/or
improved physical health.
In order to answer this broad-based question, the TAPA Center expects to utilize regression techniques in an attempt to predict which factors (demographics, service use, arrest history etc.) are related to and/or contribute to positive mental health, substance use and criminal justice outcomes in both the short-term (6 months) and the long-term (12 months). While these analyses will be valuable, there will be limited generalizability due to the nature of the sample.
Analyses based on the additional trauma sections – The additional trauma sections in interviews for the FY2006 and subsequent Grantees will provide further insight in three main areas:
Identification of specific trauma symptoms – The Posttraumatic Stress Checklist (PCL) identifies a specific set of symptoms that can be directly linked to trauma; symptoms that are not otherwise measured by the TCE evaluation. This new information from the PCL will allow us to draw conclusions in the following areas:
The incident rates of trauma and re-trauma (in conjunction with the DC Trauma Screen);
The extent (severity)of specific trauma symptoms;
Which factors are (and are not) related to trauma and re- trauma (e.g., substance use, homelessness) (in conjunction with the DC Trauma Screen);
How targeting individual-specific symptom profiles can inform the use of appropriate evidence-based interventions.
Examination of individual change across the participation period – Including both the PCL and the DC Trauma Screen in each of the three interviews will allow the TCE Initiative to draw additional conclusions in the following areas:
The changing patterns of trauma incidence, symptoms and symptom severity during enrollment in a jail diversion program;
The relationship of trauma symptom severity and trauma with other mental health and criminal justice outcomes during the participation period;
The relationship between trauma experience and PTSD symptoms in the jail diversion population;
The role of evidence-based interventions in mitigating trauma symptoms and re-trauma across time.
The extent to which trauma symptoms influence the treatment needs and responses of individuals relative to other mental health symptoms assessed in the evaluation (e.g., Colorado Symptom Index);
Informing treatment plans in jail diversion programs – Information obtained through administering the two trauma sections at each of the interviews would help improve jail diversion program design (and hence benefit future jail diversion program participants) in the following ways:
DC Trauma Screen – Research has shown that earlier trauma, especially childhood sexual abuse, increases risk of later trauma and symptoms of posttraumatic stress over the lifetime (1, 2, 3). Including the DC Trauma Screen in the follow-up interviews will allow for the identification of individual patterns of trauma and re-trauma across the participation period. This information will be essential in making intervention, program and policy changes designed to prevent further trauma.
PCL – While trauma history screening is an important first step in identifying potential needs of consumers, it does not have the ability to speak to specific problems or symptoms that cause distress. The PCL assesses all 17 symptoms of posttraumatic stress. As a result the PCL has the ability to identify the type and severity of trauma-related symptoms that jail diversion participants generally experience; symptoms that can be linked directly to specific treatment plans. For example, symptoms such as nightmares or avoidance of others can directly translate into treatment plans using evidence-based interventions such as relaxation techniques (to lessen anxious arousal) and trauma-specific cognitive-behavioral therapies (to reduce nightmares and intrusive thoughts). Additionally, the literature suggests that experiencing PTSD symptoms may worsen the course of other symptoms of serious mental illness (Mueser et al., 2002) which implies that the targeting of trauma symptoms in treatment will lead to improvements in other areas of functioning. Hence, collecting the PCL will allow jail diversion programs to make trauma-informed intervention, program and policy changes involving population-specific, evidence-based interventions resulting in individual improvement in functioning across a variety of areas.
17) Display of Expiration Date
The expiration date will be displayed.
18) Exceptions to Certification Statement
This collection complies with the requirements in 5 CFR 1320.9. The certifications are included in this submission.
1) Respondent Universe and Sampling Methods
The TCE jail-diversion Grantees will be attempting to collect data about all individuals screened for diversion as well as about all jail diversion program enrollees. The starting point for data collection is everyone that the programs screen for possible entry into the jail diversion program. Based on data received from the previous rounds of TCE Initiative Grantees, on average approximately 1,200 individuals per year, per Grantee are screened at least once for possible jail diversion enrollment. (This number is variable across Grantees, however, due to differences in the size and types of programs.) Although this is a large number, the evaluation requires basic information on these individuals in order for SAMSHA/CMHS to obtain an accurate representation of the population that the enrollees are part of and to credit the programs with the extra level of planning and work required in order to enroll individuals into the jail diversion programs.
Based on data received from the previous rounds of TCE Initiative Grantees, on average, 50 individuals are enrolled in jail diversion by one Grantee per year. (Again this number is variable across Grantees due to differences in the size and types of programs.) Hence, compared to the census of everyone screened, the number of individuals deemed eligible and enrolled in the jail-diversion programs is quite small. Also program enrollees may decline to participate in the evaluation and evaluation participants may decline to participate in one or both follow-up interviews. Furthermore, it should be noted that prior to the addition of the grant mandated evaluation, no formal sample size was formulated. The goal for the program evaluation is to collect data on as many jail diversion program enrollees as possible at each site. For these reasons a universal sampling frame will be required to obtain the largest n possible.
Based on Grantees already collecting data, there is an expectation that the newer Grantees will meet or exceed the required minimum evaluation participation rate of 80 percent which should insure a sufficient sample size for the purposes of this evaluation. FY 2003 Grantees achieved a participation rate is 89.3%. FY 2004 Grantees have only just begun enrolling clients and conclusive data on their participation rate is not available.
Once the evaluation is completed, the TAPA Center will perform basic demographic comparisons between those enrolled in the evaluation and those who refused participation to determine whether any selection bias occurred due to refusals. The TAPA Center will also examine whether 6 and 12 month attrition resulted in any bias. If bias has occurred, adjustments will be made in analysis in order to account for this.
2) Information Collection Procedures
As discussed earlier, there are three primary data sources that comprise all of the data collected through this evaluation:
Interview Data composed of the following primary measures
GPRA measures
DC Trauma Collaboration Study Violence and Trauma Screening,
Posttraumatic Stress Checklist (FY2006 and subsequent Grantees only)
Colorado Symptom Index 1991
Perceived Coercion Scale (from the Mac Arthur Mandated Community Treatment Survey)
Mental Health Statistics Improvement Program (MHSIP)
Tracking Data
Events Tracking (demographic and eligibility information)
Person Tracking (demographics, diagnosis, charges, diversion point and condition and target arrest/incident and enrollment information)
Record Review Data
Service Use (type and number of services received)
Arrest History (type and jail days of arrests)
The starting point for data collection is everyone that the programs screen for possible entry into the jail diversion program. Once a person is deemed eligible for enrollment, and the court accepts the diversion plan (if applicable), then the person is enrolled in the jail diversion program. All of the data gathered about those individuals screened for possible diversion are collected and entered by Grantee staff using information obtained from the potential diversion program enrollees.
All diversion program enrollees are eligible to participate in the evaluation and unless there are extenuating circumstances (such as difficulties locating the person or hospitalization) all enrollees are approached for consent to participate. Attachment F provides an example of a consent form. Of those enrollees agreeing to participate in the evaluation, all are expected to receive a baseline interview within seven days of enrollment which can be conducted by Grantee staff. The participants are then called back for 6 and 12 month follow-up interviews, which must be administered within 30 days on either side of the due date and are conducted by Grantee staff only. Attachment H provides an example of a follow-up interview reminder letter. Note that while program staff may administer baseline interviews, only Grantee staff who is not in any way involved in providing services to program participants administers follow-up interviews. This is done to protect the privacy of the participant and to ensure that no adverse effects result from a refusal to participate in the evaluation or from any responses given.
Participants completing a baseline interview as well as a 6 and/or 12 month interview will receive service use and arrest history record reviews, which are conducted by Grantee staff. Person Tracking data are collected on all enrollees regardless of enrollment in the evaluation and contain information obtained either directly or indirectly from the potential diversion program enrollees. For those enrollees participating in the evaluation, the Person Tracking data also include information about interview completion statuses (when due and whether/when completed).
Most Grantee staff will record participant information through a pencil and paper method. This evaluation will not interfere with ongoing program operations. Grantee staff will submit completed interview forms monthly, electronic tracking data extracts bimonthly and completed record review forms at least once annually. All data, except the non-identifiable Events Tracking data, are matched using a unique client identifier created by the Person Tracking program.
3) Methods to Maximize Response Rates
Each jail diversion program has an Access-based software database distributed by the TAPA Center to assist in collecting required demographic and background information to assist in tracking participants and providing demographic information for the evaluation. The tracking software program has the capability to provide information on the current addresses for the participants and when they are due for a follow-up interview. The Baseline Person Tracking Program Information Form (Attachment G) is part of the tracking software program. It is intended to help interviewers contact participants for follow-up interviews.
The participants are contacted in several ways in order to assure that they return for the 6 and 12 month interviews. These methods vary from site to site and include mailing a reminder letter (one example is included in Attachment H), contacting friends, family, case managers, and/or therapists (with permission), dropping in on commonly visited locations (e.g., soup kitchens, shelters, AA meetings), and coordinating follow-up interviews with other scheduled appointments at the program.
Remuneration is used to encourage those participants that are part of hard to reach populations to complete the interviews. For many of the original 20 Grantees, remuneration is one important method for insuring high evaluation participation rates among jail diversion enrollees. Respondents are given remuneration of bus passes, grocery store and restaurant vouchers, phone calling cards, and clothing, not to exceed $30 in value. Each jail diversion program determines its own remuneration practices. The use of incentives varies by site and therefore it is difficult to determine the expected response rate with and without remuneration.
4) Tests of Procedures
The measures included in the TCE Initiative’s interviews are a combination of mandated GPRA items and additional non-GPRA mental health scales. Once compiled, the original (FY2002) TCE Grantees had the opportunity to review the instruments and agreed to the inclusion of all items approved by the OMB. Once consensus was reached, all of the measures were implemented and have been successfully administered by all rounds of the TCE Grantees (except the FY2006 and subsequent Grantees who have not started data collection). All of the non-GPRA measures contained in the instruments have been pilot tested and/or are well established data collection tools tested for validity and reliability. The four main non-GPRA measures along with their developmental background are as follows:
DC Trauma Collaboration Study Violence and Trauma Screening – This screening was originally developed by Community Connections in Washington, DC for use by clinicians in determining if an individual's trauma history warranted a clinical follow-up and in providing a sense of the recentness of the trauma and of the urgency of concern. This screen was developed to be used as a descriptive clinical tool only and, as such, has no psychometric properties. For the TCE Initiative, its inclusion is intended to provide basic descriptive information about individual trauma levels. Its use as an evaluation tool began with the SAMSHA Women, Co-Occurring Disorders & Violence Study (WCDVS) where the screen was recommended by leading trauma experts from the few available trauma measures. As part of the WCDVS planning process, this screen was subjected to a lengthy review by a group of consumers, clinicians and researchers. The screen was also pilot tested as part of the WCDVS study. Furthermore this screen is currently used by the prior rounds of the TCE Initiative Grantees.
Perceived Coercion Scale – This scale is a subscale of the MacArthur Admission Experience Survey that was developed as part of the MacArthur Mandated Community Treatment Survey. This subscale was developed to measure patient’s perceptions of coercion to enter treatment. An examination of its psychometric properties using principal components (factor) and correspondence analyses found strong evidence that the individual items represent a single construct (a patient’s perception of treatment coercion) and that the combined scale accurately summarizes this construct (4). Since its release, this scale has been used as an evaluation tool in many projects including previous rounds of the TCE Initiative’s evaluation.
Mental Health Statistics Improvement Program (MHSIP) – This tool was developed by a policy group interested in improving the quality and use of collected mental health information. The Official MSHIP Adult Survey on Consumer Satisfaction (released in 2000) includes 28 items with five domains. The TCE Initiative has incorporated only the Consumer Perceptions of Outcomes (“Outcomes”) domain for its interviews. Psychometric properties of the full scale and its domains are only available for an older 40-item version of the scale. These analyses found the “Outcome” domain to have an inter-item reliability greater than 0.50 with the items loading well in factor analyses (5). Both the “Report on the MHSIP Consumer Survey Workgroup”(6) and “The Final Report of the Mental Health Statistics Improvement Program (MHSIP) Task Force on a Consumer-Oriented Mental Health Report Card” (7) suggest that the final 28 item version of the scale has comparable reliability and validity to the 40 item version. Furthermore, the survey was pilot tested by the MHSIP Task Force and has been used by the previous rounds of TCE Initiative Grantees. It is also currently in use in many states across the country.
Colorado Symptom Index (CSI) – This scale was developed as a measure of psychiatric symptom status and was originally used in a major study of treatment outcomes in Colorado (8). Results from this study found the scale to have strong internal consistency (with alphas ranging from 0.77 to 0.85). Since then this scale has become a commonly used evaluation tool for measuring individual mental health status and has been part of the evaluation for the previous rounds of TCE Initiative Grantees.
The non-interview forms (for events tracking, person tracking, service use and arrest history data collection) collect commonly used descriptive and/or publicly available information. As with the interview forms, the original (FY 2002) TCE Grantees provided feedback about the information in, and format of, the tracking and record review forms. Once consensus was reached, these forms were implemented and successfully completed by all rounds of the Grantees.
The FY2006 Grantees had the opportunity to review the revised interviews that include the Posttraumatic Stress Checklist (PCL) and the extension of the DC Trauma Screen to the follow-up interviews. They agreed to include all items previously approved (including the DC Trauma Screen) as well as the PCL. These revised interviews will be administered by all Grantees awarded in FY2006 and in subsequent years.
Posttraumatic Stress Checklist - The PCL is a 17-item self-report measure of Posttraumatic Stress Disorder (PTSD) symptoms based on DSM-IV criteria, with a 5-point Likert scale response format that rates the severity of each symptom over the past month. Continuous scores are used to assess symptom severity and a cut-point of 3 (moderate severity) is used per each PTSD symptom to derive a PTSD diagnosis. The PCL has good psychometric properties. It has been found to be highly correlated with the Clinician Administered PTSD Scale (r = .929), the “gold standard” measure of PTSD, has good diagnostic efficiency (> .70), and robust psychometric properties with a variety of trauma populations (9, 10). Among individuals with serious mental illness, high internal consistency of the PCL was reported (.94 coefficient alpha), along with moderate test-retest reliability (.66) and moderate convergent validity with the CAPS (κ=.67) (11). Based on the brevity of the scale, along with its validity and reliability, the TAPA Center agreed that this would be a useful measure of PTSD symptoms to potentially be included in the TCE Initiative’s evaluation for the FY2006 and subsequent Grantees.
5) Statistical Consultants
Contractors/Statistical Consultants:
Name/Title |
Address |
Contact Information |
Henry Steadman, Ph.D. Director National GAINS Center |
Policy Research Associates 345 Delaware Avenue Delmar, NY 12885 |
Phone: (518) 439-7415 x229
|
Dan Abreu, M.S. CRC LMHC Associate Director National GAINS Center |
Policy Research Associates 345 Delaware Avenue Delmar, NY 12885 |
Phone: (518) 439-7415 x248 |
Laura Morris, M.S. Research Associate 2 |
TAPA Center 345 Delaware Avenue Delmar, NY 12885 |
Phone: (518) 439-7415 x364 |
Tariqul Islam, B.A. Research Assistant |
TAPA Center 345 Delaware Avenue Delmar, NY 12885 |
Phone: (518) 439-7415 x265 |
Steven Banks, Ph.D. Statistical Consultant |
The Bristol Observatory 521 Hewitt Road Bristol, Vermont 05443 |
Federal Project Officers/Statistical Consultants
Name/Title |
Address |
Contact Information |
David Morrissette, Ph.D. Government Project Officer
|
Center for Mental Health Services, SAMHSA |
Phone: (240) 276-1912
|
List of Attachments:
Baseline Interview (with revisions)
6-Month Interview (with revisions)
12-Month Interview (with revisions)
Arrest Data Collection Form
Mental Health and Substance Abuse Service Use Data Collection Form
Example of a Consent Form
Baseline Person Tracking Information Form
Follow-up Interview Reminder Letter
1 Burnam, M.A., Stein, J.A., Golding, J.M., & Seigel, J.M. (1988). Sexual assault and mental disorders in a community population. Journal of Consulting and Clinical Psychology, 56(6), 843-850.
2 Nishith, P., Mechanic, M.B., & Resick, P.A. (2000). Prior interpersonal trauma: The contribution to current PTSD symptoms in female rape victims. Journal of Abnormal Psychology, 109(1), 20-25.
3 Polusny, M.A. & Folette, V.M. (1995). Long term correlates of child sexual abuse. Applied & Preventive Psychology, 4(3), 143-166.
4 Gardner W., Hoge, S., Bennett, N., Roth, L., Lidz, C., Monahan, J., and Mulvey, E. (1993). Two scales for measuring patients' performance perceptions of coercion during hospital admission. Behavioral Sciences and the Law, 20, 307-321.
5 “Psychometric Properties of Report Card Instrument” from http://www.mhsip.org/library/browse_library.asp
6 Available at http://www.mhsip.org/library/browse_library.asp
8 Shern, D.L., Wilson, N.Z., & Coen, A.S. (1994). Client Outcomes II: Longitudinal Client Data from the Colorado Treatment Outcome Study. The Millbank Quarterly, 72 (1), 123-148.
9 Andrykowski, M.A., Cordova, M.J., Studts, J.L., & Miller, T.W. (1998). Posttraumatic stress disorder after treatment for breast cancer: Prevalence of diagnosis and use of the PTSD Checklist–Civilian Version (PCL–C) as a screening instrument. Journal of Consulting and Clinical Psychology, 6, 586–590.
10 Blanchard, E.B., Jones-Alexander, J., Buckley, T.C., & Forneris, C.A. (1996). Psychometric properties of the PTSD Checklist (PCL). Behaviour Research and Therapy, 34, 669-673.
11 Mueser, K.T., Rosenberg, S.D., Fox, L., Salyers, M.P., Ford, J.D., & Carty, P. (2001). Psychometric evaluation of trauma and posttraumatic stress disorder assessments in persons with severe mental illness. Psychological Assessment, 13(1), 110-117.
| File Type | application/msword |
| File Title | THE SUPPORTING STATEMENT |
| Author | Sarah Fleury |
| Last Modified By | Morrissette |
| File Modified | 2007-01-29 |
| File Created | 2007-01-25 |