CSAT GBHI OMB Supporting Statement FINAL_3_16_11

The Cross-Site Evaluation For the Benefit of Homeless Individuals (GBHI)

Supporting Statement

A. JUSTIFICATION

1. Circumstances of Information Collection

The Substance Abuse and Mental Health Services Administration’s (SAMHSA) Center for Substance Abuse Treatment (CSAT) is requesting approval from the Office of Management and Budget (OMB) for data collection activities for the Cross-Site Evaluation of the Grants for the Benefit of Homeless Individuals (GBHI) program. These activities include administration of the following surveys:

• CSAT GBHI Client Interview – Baseline

• CSAT GBHI Client Interview – 6-Month Follow-up

• CSAT GBHI Stakeholder Survey

The CSAT GBHI program is authorized under Section 506 of the Public Health Service Act (42 U.S.C. 290aa-5), as amended. The program also addresses Healthy People 2010, Volume II (Part B: Focus Area 18 (Mental Health and Mental Disorders) and Focus Area 26 (Substance Abuse)).

Homelessness affects more than 3.5 million people in the United States (National Law Center on Homelessness & Poverty, 2009) and about 38% of those homeless are alcohol dependent and 26% abuse other drugs (Burt et al., 1999; National Coalition for the Homeless, 2009). In a general homeless population, about 32% of men and 36% of women are estimated to have co-occurring mental and addictive disorders (North, Eyrich, Pollio, & Spitznagel, 2004). Overall, in a national sample, about three-quarters (74%) reported any alcohol, drug, or mental health problem in the year before shelter admission (Burt et al., 1999). The literature is replete with evidence suggesting that homelessness, substance use, and mental illness are closely associated and that the prevalence rates for the latter two problems are high (Hiday, Swartz, Swanson, Borum, & Wagner, 1999; Mallett, Rosenthal, & Keys, 2005; Shelton, Taylor, Bonner, & Bree, 2009; Vangeest & Johnson, 2002). Several populations, including veterans, families, victims of trauma, and criminal justice populations, are at particular risk for homelessness and alcohol and drug abuse (Greenberg & Rosenheck, 2008; HUD, 2009; McNeil, Binder & Robinson, 2005; Moore, Gerdtz, & Manias, 2007; National Law Center on Homelessness and Poverty, 2009; Rukmana, 2008; Veterans Administration, 2009). Eighty-five percent of the chronic homeless—those who have either been continuously homeless for one year or more or have had at least four episodes of homelessness in the past three years—have co-occurring mental and addictive disorders (Joseph & Langrod, 2004). Across two national samples, the National Survey of Homeless Assistance Providers and Clients and the U.S. Department of Housing and Urban Development’s (HUD) 2008 Annual Homeless Assessment Report, between 10% and 23% of respondents were veterans (Burt et al., 1999; U.S. HUD, 2009). The U.S. Department of Veterans Affairs (2009) estimates that about 45% of homeless veterans have mental illness, more than 70% suffer from alcohol or other drug abuse problems, and many are comorbid for these conditions. Persian Gulf and Middle East–era returning veterans are at increased risk for homelessness compared with prior service era veterans (Kline et al., 2009). Substance use, mental illness, and victimization are primary predictors of homelessness in longitudinal studies (Shelton et al., 2009; van den Bree et al., 2009) and criminal justice involvement has a bidirectional relationship with homelessness (Caton, Wilkins, & Anderson, 2007; Greenberg & Rosenheck, 2008; Martell, Rosner, & Harmon, 1995).

Effectiveness of substance abuse treatment (e.g., Modified Therapeutic Communities, Motivational Enhancement Therapy, Service Outreach and Recovery) in producing abstinence and a number of positive outcomes like employment stability, treatment adherence, and reduced unprotected sex has been well established (Ball et al., 2007; Borsari & Carey, 2000; Brown & Miller, 1993; Conrad et al., 1997; Drake, Yovetich, Bebout, Harris, & McHugo, 1997; Kertesz, Crouch, Milby, Cusimano, & Schumacher, 2009; Miller, Benefield, & Tonigan, 1993; Project MATCH Research Group, 1997; Rosenblum, Magura, Kayman, & Fong, 2005; Stephens, Roffman, & Curtin, 2000), as have the effects of integrated treatment for co-occurring disorders (e.g., Integrated Dual Disorders Treatment) on substance abuse, mental health, hospitalization, violence, and homelessness (Drake, McHugo, & Noordsy, 1993; Mueser, Drake, & Miles, 1997).

Housing interventions are most effective when combined with other services (Caton et al., 2007; Nelson, Aubry, & Lafrance, 2007), however, it is unclear which combination of housing models, services, and treatment yields the most robust outcomes with respect to housing stability, substance use, psychiatric symptomology, employment, and other important outcomes. In addition to combining treatment and housing strategies, various common structural characteristics or services, systems and program organization have been found in effective programs. In their review, Cheng and Kelly (2008) describe structural characteristics generally found in effective programs: interagency coalitions; interagency service delivery teams; interagency management information systems and client tracking systems; interagency agreements that formalize collaborative relationships; interagency application for funds; uniform application, eligibility criteria, and intake assessments; and co-location of services.

Gaps in the research include: studies that include non-HUD–funded programs to better describe prevalence of substance abuse, mental illness, and co-occurring problems; evaluation of subgroups of homeless individuals within a single study (with similar definitions, measures, and procedures); information on the needs of subpopulations across the continuum of homelessness; implementation and effectiveness of EBP’s specifically in homeless populations; fidelity of treatment and housing models implemented; cost-effectiveness in complex sites that employ multiservice interventions; and the value that comprehensive initiatives, such as those implemented by GBHI grantees, add to the overall treatment systems. There is also a dearth of empirical studies that look at performance measurement of homeless programs, benchmarking, and efficiency measures. Finally, there are few multisite studies of the sustainability of programs after cessation of federal funding and factors associated with sustainability.

Treatment providers know how to serve many of these individuals successfully and cost-effectively, but barriers to effective treatment exist; even after entering substance abuse treatment programs, many people who are homeless do not complete them. Recognizing the enormous societal costs of these persons’ failure to get needed treatment services, SAMHSA/CSAT was funded by Congress to establish the GBHI program, one of the grant programs in the Co-Occurring and Homeless Activities Branch (CHAB). CSAT GBHI is a competitive, discretionary grant program initiated in 2001 with the following goals: (1) to link substance use and mental health treatment services with housing programs and other services, (2) to expand and strengthen treatment services for people who are homeless who also have substance use disorders, mental disorders, or co-occurring substance use and mental disorders; and (3) to increase the number of homeless people who are placed in stable housing and who receive treatment services for alcohol, substance use, and co-occurring disorders.

Between 2001 and 2008, GBHI awarded 169 grants to provide services to the target population. An additional 25 Grantees were funded in 2009. Some Grantees serve priority populations, including criminal justice populations, chronically homeless persons, returning veterans, and chronic public inebriates; others focus on serving families, women, native Alaskans, Native Americans/Indians, other minority populations, or youth. Although all are required to, at a minimum, provide outreach, case management, substance abuse or co-occurring disorders treatment (integrated, sequential, or parallel), and wraparound and recovery services, many augment these services by adopting or adapting additional evidence-based practices (EBPs) from one of the SAMHSA toolkits (e.g., Assertive Community Treatment (ACT), Illness Management and Recovery, Supportive Employment), CSAT’s Treatment Improvement Protocols, or the National Registry of Evidence-Based Programs and Practices (NREPP). The models for service delivery vary and include primarily by referral, direct provision of treatment and other services, or a mix of direct service provision and referral to other community-based organizations. Service models are implemented in an array of settings, including on the street through outreach; in drop-in settings, shelters, and hospitals; at medical, substance abuse, or mental health clinics; in residential treatment communities; or in any of these settings or other nonoffice settings through mobile crisis units or ACT teams.

All clients are assessed by Grantees at intake, 6-months follow-up to intake and at program discharge with the CSAT GPRA Client Outcome Measures for Discretionary Programs (OMB control number 0930-0208) which data is provided to SAMHSA via the web and stored in the Services Accountability Improvement System (SAIS). Since the inception of the GBHI program, CSAT homeless grants have served 33,171 individuals, a majority minority men aged 18 to 54 (Le Fauve, 2009). In 2010 the active portfolio has served over 22,000 individuals. Per the FY2010 President Obama’s budget, outcomes data available for a subset of clients served by the program through 91 active GBHI Grantees was cited indicating that individuals demonstrate: 1) 122% increase in employment or engaging in productive activities; 2) 166% increase in persons with a permanent place to live in the community; 3) 52% increase in no past months substance use; and 4) 36% improvement in no/reduced alcohol or illegal drug related health, behavioral or social consequences.

The CSAT GBHI cross-site evaluation represents the most comprehensive assessment of CSAT GBHI ever undertaken and will provide evidence on the effects of CSAT GBHI project activities on client outcomes, treatment services, and treatment systems. This information will allow SAMHSA to determine the extent to which CSAT GBHI has met its objectives of implementing a program to provide substance abuse and integrated co-occurring mental and addictive disorders treatment and other wraparound services to meet the needs of homeless individuals and end homelessness among those with substance use and co-occurring problems. To achieve this overarching goal, the evaluation will identify the barriers, challenges, and facilitators of successful CSAT GBHI project implementation. This evaluation will also examine the feasibility, utility, and sustainability of future CSAT GBHI cohorts and make recommendations to SAMHSA of ways to improve future initiatives within the CSAT GBHI portfolio. The cross-site evaluation will specifically help SAMHSA achieve the goals of its Capacity Performance Goals and Matrix Priorities (see Attachment 1). The CSAT GBHI program is designed to help SAMHSA meet the following cross-cutting principles, Science to Services/Evidence-based Practices and Data for Performance Measurement & Management, within the following programs, Homelessness, Substance Abuse Treatment Capacity, and Co-occurring Disorders.

The purpose of the evaluation is formative with an intent to identify and measure post-program participation findings across the broad array of outcomes expected to be influenced by the range of services provided by GBHI Grantees either directly or through referral. These services, which are provided following assessed need, include treatment for substance abuse and mental health disorders (which includes screening, assessment, and active treatment), outreach, case management, and wraparound services (which can include, for example, relapse prevention, crisis care, education or vocational services, transportation, medical care, housing readiness training, benefits application, housing application, peer support services) and aftercare. As much as possible the intent is for each of these services to be based on models and practices that have an evidence base in peer-reviewed literature. Thus, outcomes directly relevant to the GBHI program include those related to substance use, mental health, employment and education, as well as to additional behavioral outcomes and housing. The evaluation does not intend to draw causal inferences with respect to the GBHI program participation and outcomes, but to measure (more explicitly than the current CSAT GPRA Outcomes Measures for Discretionary Programs allow) a variety of outcomes directly related to the specific services included in the GBHI programs. These are programmatic outcomes that are used to monitor the provision of services, understand the way services are tailored to clients with different needs and to better understand how the implemented service models match the models described in the efficacy and effectiveness literature.

This evaluation will examine structural, process, outcome, and cost components. The first component, structure, encompasses the resources available in a treatment delivery system; it can apply to individual practitioners, groups of practitioners, and to organizations and agencies. They represent the capacity to deliver quality care, but not the care itself. Process, the second component, represents what is done to and for the client. Process measurement can also focus on individual practitioners, groups, organizations, agencies or systems of care. Measures of structure and process will characterize the grantee organization and its partnerships, the system within which the program is embedded, the grantee’s relationships with stakeholders, the target population, services provided and received, program planning and implementation. These measures will provide important information on the nature of CSAT GBHI programs and will be used in outcome models as variables that are expected to be associated with client and program outcomes.

The outcome evaluation is the third component, which focuses on addressing the utility of the CSAT GBHI programs, or, the value in terms of the benefits produced. The outcome evaluation, using a formative, not causal approach, will focus on the program effects of the CSAT GBHI programs on client outcomes, accounting for grantee characteristics, treatment system and community contexts.

The fourth and final component of the cross-site evaluation is an economic evaluation of CSAT GBHI. The economic evaluation connects significantly to all other aspects of the evaluation by incorporating results from the process, structure, and outcome evaluations. The economic evaluation questions are focused on measuring the cost and cost-effectiveness of CSAT GBHI at the client, grantee, and system levels; on obtaining cost metrics that allow CSAT GBHI and CSAT GBHI components to be compared with other services; and on determining factors that affect the cost and cost-effectiveness of CSAT GBHI.

In summary, we present the evaluation framework and SAMHSA’s intent for the evaluation to address SAMHSA’s questions in terms of this formative evaluation. The purpose of the evaluation is formative with an intent to identify and measure post-program participation findings across the broad array of outcomes expected to be influenced by the range of services provided by GBHI Grantees either directly or through referral. These are programmatic outcomes that are used to monitor the provision of services, understand the way services are tailored to clients with different needs and to better understand how the implemented service models match the models described in the efficacy and effectiveness literature (see Attachment 11).

2. Purpose and Use of Information

The purpose of the CSAT GBHI Client Interview – Baseline and the CSAT GBHI Client Interview – 6-Month Follow-up is to collect client-level data that can be utilized to assess program impact on client outcomes and to provide descriptive information about clients. The data collected through the CSAT GBHI Stakeholder Survey will provide descriptive information about stakeholders involved with the CSAT GBHI grant program and their relationship with the grantee program. The information collected through all three surveys will provide the data necessary to conduct a complete structure, process, outcome, and cost evaluation, as described above. Detailed descriptions and purpose of the surveys are presented in the following paragraphs.

CSAT GBHI Client Interview – Baseline & CSAT GBHI Client Interview – 6-Month Follow-up: (See Attachments 2 & 3)

The CSAT GBHI Client Interview – Baseline and 6-Month Follow-up were developed to assess program impact on client outcomes on the basis of consultation with CHAB; discussion and feedback during the CSAT GBHI Cross-site Evaluation Expert Panel meeting; written comments received after expert panelist review of the preliminary evaluation plan, draft data collection tables, and an extensive literature review; and a review of protocols used in SAMHSA’s Homeless Families and Supportive Housing initiatives as well as by HUD. Additional areas highlighted for measurement include co-occurring mental disorders, history of homelessness, housing (placement/satisfaction), perception of coercion and choice in treatment and housing, readiness for change, service need, perception of care, and client burden. Three additional domains were also consistently advocated by the expert consultants: services received, trauma, and veterans’ service era and combat information.

The CSAT GBHI Client Interview – Baseline and the CSAT GBHI Client Interview – 6-Month Follow-up are composed of the following sections:

• Military Service Questions—Given the high prevalence of homelessness among returning veterans and differentially by service era (Kline et al., 2009), baseline collection of military service was recommended. These questions are included to collect basic information about the military background of clients, specifically branch of service, years of service, and service in a combat zone. This information is adapted from the Center for Mental Health Services (CMHS) Jail Diversion and Trauma Recovery Evaluation. This information is collected for descriptive purposes and is only collected at baseline.

• Employment—One question was developed to assess employment in the previous six month period to assess the impact of treatment services on this outcome measure. Employment is viewed as important in the ability to attain and maintain housing (Burt et al., 1999; Pickett-Schenk et al., 2002; Shaheen & Rio, 2007). This item will be asked at baseline and 6-month follow-up.

• Criminal Justice Involvement—Two questions were developed to assess number of arrests and number of nights incarcerated for the previous 6‑month period. Criminal justice involvement has been strongly associated with homelessness and with substance use (e.g., Greenberg & Rosenheck, 2008). These items will be asked at baseline and 6-month follow-up.

• Co-occurring disorders—At the request of CHAB, questions were developed to capture self-report data on severity and extent of co-occurring disorders among clients served by CSAT GBHI grantees. Those with co-occurring mental disorders are at increased risk for homelessness and co-occurring mental disorders are prevalent among chronically homeless individuals (e.g., Drake et al., 1997; Joseph & Langrod, 2004). This information will be both descriptive and used in sub-group analyses of client outcomes. These questions will only be asked at baseline.

• Housing and Homeless History—These questions assess the client’s current residence and residential history in the past 6 months including places stayed, time spent homeless, and problems encountered finding housing. The questions on past 6-month residential history are adapted from the HUD “Life After Transitional Housing” Study (Burt, 2009). Other questions include age of first homeless episode and the frequency and length of time a client has been homeless in the past three years, the amount of time necessary to measure homeless chronicity. These questions have been adapted from the CMHS/CSAT Homeless Families Study. Questions about homelessness in the past three years will be asked at baseline only. Questions about current residence and residential history in the past 6 months will be asked at both baseline and 6-month follow-up.

• Housing Satisfaction and Choice—This measure provides information on client satisfaction with various aspects of his or her housing, as well as, the amount of choice the client had over the place where he or she currently resides, which both have been associated with positive client outcomes (Greenwood, Schaefer-McDanile, Winkel, & Tsemberis, 2005; Srebnik, Livingston, Gordon, & King, 1995; Tsemberis, Moran, Shinn, Asmussen, & Shern, 2003). The measure was developed for the CMHS Supportive Housing Initiative. This set of questions will be asked at the baseline and the 6-month interviews.

• Perception of Housing Coercion—These questions were modified from Robbins, Callahan, and Monahan’s (2009) study of perceived coercion to treatment and client housing satisfaction among clients in Housing-First and Supportive Housing Programs. These questions are included to assess explicit treatment requirements and the extent to which clients feel they must participate in services to remain in their housing. These will be asked at the baseline and 6-month interviews.

• Readiness to Change Questionnaire (RTCQ)—This was developed by Rollnick et al. (1992) to use in brief interventions among problem drinkers. The questionnaire will be used to assess readiness to change among individuals who abuse alcohol and other drugs. These measures will be asked at both baseline and 6-month interviews and will allow the contractor to compare changes over time as well as include as a mediator variable in the client outcome analyses.

• Services Needed and Received—This section is designed to obtain information from the client’s perspective on the types of services he or she needed and types of services he or she received. These questions are adapted from the CMHS/CSAT Homeless Families Initiative and the CMHS Jail Diversion and Trauma Recovery Evaluation. These additional questions will allow the contractor to document all services received by clients, including those services provided outside of the CSAT GBHI Grantee Program; document differences between client and program reporting; and assess whether and how service receipt changed over time. These data will be important in improving ability to test whether treatment produces abstinence and housing stability. These items will be asked at the baseline and 6-month interviews.

• Perception of Care—These questions include a subset of items from the full Mental Health Statistics Improvement Program (MHSIP) Consumer Survey to assess cultural sensitivity to care, quality of treatment, general satisfaction, and the degree to which services focus on consumer recovery and self-management (Ganju, 1999). The MHSIP Consumer Survey was designed to obtain the subjective evaluation from the consumer on issues related to access, quality, appropriateness and outcomes. The questions have been adapted by the CMHS National Outcomes Measures. This information will be both descriptive and used as a mediator in outcome analyses. These questions will be asked at the baseline and 6-month interview.

• Treatment Choice—These questions are included to determine the extent to which clients feel coerced into treatment participation. The types of coercion covered include: income benefits, housing benefits, child custody, court ordered-treatment, and abstinence from substance use. There is also one question designed to assess whether clients are aware of other similar services in their community. Although developed specifically for this evaluation, the literature indicates these are areas of coercion for substance abuse treatment clients (Robbins et al., 2009). These items will be assessed at baseline and 6-month follow-up.

• Client Treatment Burden—These questions are posed to determine the financial burden treatment could potentially place on an individual. In addition to economic impediments, there are other practical impediments to participation in treatment services (Tucker et al., 2004; Rapp et al., 2006). The sources of financial burden were adapted from the Client Drug Abuse Treatment Cost Analysis Program (DATCAP) which was designed to assess the costs incurred by patients who attend inpatient or outpatient treatment services (DATCAP.com). This information will be collected at the 6-month interview only and will be used in the cost evaluation.

• Abbreviated Posttraumatic Stress Disorder Checklist (PCL-C)—Expert panelists recommended measuring trauma symptoms given that trauma is prevalent in the homeless population (e.g., Browne & Bassuk, 1997; Goodman, 1991; Bassuk et al., 1996; Burt et al., 1999; HUD, 2009; Shelton et al., 2009) and without intervention consistently predicts negative substance abuse, employment, housing and criminal justice outcomes. This 6-item measure is an abbreviated version of the PCL-C (Weathers, Litz, Huska, & Keane, 1994) which was developed to use as a screening instrument by primary care doctors (Lang & Stein, 2005). This information will be collected at both baseline and 6-month follow-up to assess changes in trauma symptoms.

A chart is presented in Attachment 11, for each of the CSAT GBHI Client Interview questions noting the domain relevant to the item or measure, justification for use in the evaluation, the population for which the measures were developed, additional literature citations and as relevant the corresponding OMB approval number for the items used in previous OMB approved cross-site evaluations and SAMHSA performance monitoring measures.

The target population for the CSAT GBHI Client Interview – Baseline and the 6-Month Follow-up is all accepted and enrolled clients receiving services under the CSAT GBHI grant in the 2009 (FY2010) cohort.

CSAT GBHI Stakeholder Survey: (See Attachment 4)

The CSAT GBHI Stakeholder Survey is a 22-item questionnaire that will be administered, per voluntary consent, via the web, to CSAT GBHI grantee stakeholder partners for projects funded from 2004 – 2009 (FY2005-FY2010). This is the main method through which the contractor will collect primary data from stakeholders. The questionnaire is designed to address SAMHSA’s GBHI cross-site objectives regarding service provision, impact on local treatment systems, implementation lessons learned and project sustainability. The questions specifically gather background information about the partner agency, the services provided, and experience partnering on the implementation and sustainability efforts of the local CSAT GBHI program. This information is necessary to (a) assess important aspects of the CSAT GBHI program related to partnering, (b) measure characteristics of the local treatment system in which the grantee is located, and (c) identify moderating or mediating variables of client outcomes. Questions regarding partner agency characteristics and services offered were developed for the cross-site evaluation based on a review of CSAT GBHI grantee documents submitted to SAMHSA (e.g., grantee applications) and per CSAT TOO and CHAB review feedback. Implementation and collaboration questions were adapted from the cross-site evaluation survey of Weed and Seed funded by the Department of Justice (Trudeau, Barrick, & Roehl, 2010) and from the SAMHSA CMHS Jail Diversion TCE cross-site qualitative study on program sustainability (Broner, 2010a, 2010b). Experience of the contractor in implementing this type of survey is described in Attachment 11.

3. Use of Information Technology

The CSAT GBHI Client Interview – Baseline and the CSAT GBHI Client Interview – 6-Month Follow-up are designed as a paper and pencil interview. The interview form will use electronically scannable TeleForm technology to reduce data entry burden and errors. The client interview will be administered onsite by either the grantee program or the grantee’s local evaluator. Once the interview is complete, the administrator will place the completed survey into a sealed, postage-paid envelope and return it to the contractor. Once received by the contractor, the form will be scanned into a dataset. Scanning these forms will eliminate the need for data entry, thereby reducing cost and the potential for data error. Further discussion of the TeleForm is provided in Attachment 11.

The CSAT GBHI Stakeholder Survey will be administered via the web. Each survey respondent will be issued a username and password to access the web-based survey for their program. To complete the survey, each respondent will login to a secure web-based form to fill out the survey. The web-based survey will reduce burden on the respondent and minimize potential for measurement error. For example, skip patterns and automatic data quality checks (e.g., range checks) can be coded into the online survey form to improve data quality.

4. Effort to Identify Duplication

SAMHSA monitors the performance of CSAT GBHI programs by requiring the grantees to collect and submit data through the Government Performance and Results Act (GPRA) (OMB No. 0930-0208). The CSAT GBHI Client Interview – Baseline and the CSAT GBHI Client Interview – 6-Month Follow-up cover some of the same domains as the GPRA data (e.g., employment and criminal justice) but there is no duplication of data that will be collected from the CSAT GBHI Client Interview – Baseline and the 6-Month Follow-up that can be obtained from the GPRA data. The GPRA data cover a previous 30-day timeframe which is not robust enough to accurately assess the impact of treatment services on outcome measures or establish best practices, which are both primary objectives of the evaluation. The CSAT GBHI Client Interview – Baseline and the 6-Month Follow-up questions are unique from the GPRA questions in that timeframes are extended from assessing the previous 30 days to assessing the previous six months. This timeframe extension was strongly endorsed by expert panelists at the CSAT GBHI Expert Panel meeting.

The contractor conducted an extensive literature review to confirm that the data collected through the CSAT GBHI Client Interview – Baseline, 6-Month Follow-up, and the CSAT GBHI Stakeholder Survey would not be duplicative of any ongoing national or state-level data collection efforts. Panelists at the CSAT GBHI Expert Panel meeting and contractor staff who have expertise in CHAB technical assistance have also confirmed this data collection will not be duplicative. Data collected in this evaluation is not available from other sources and will be unique because of the scale and breadth of the initiative’s implementation: nationwide, across a spectrum of provider settings, and across a broad cross-section of populations.

5. Involvement of Small Entities

CSAT GBHI grantees are required to administer a baseline and 6-month follow-up GPRA Client Outcome Measures interview to all clients admitted to their program under the CSAT GBHI grant. The grantee can elect to have its own staff or local evaluators administer the interview. A detailed description of the interviewers is provided in Attachment 11; all GPRA interviewers have been trained by SAMHSA through face-to-face and online training in interview administration and client tracking. Grantees also provide additional training to the interviewers in administrating supplemental assessments and tracking procedures. The contractor will ask the administrator of the GPRA interview to also administer the CSAT GBHI Client Interview – Baseline and the CSAT GBHI Client Interview – 6-Month Follow-up immediately following the GPRA interview. Since they will already be interviewing the client and are receiving funds for data collection under the CSAT GBHI grant, the additional interview will not add a significant amount of burden to the grantee staff or the local evaluators. In accepting CSAT GBHI funds, grantees agreed to participate fully in all SAMHSA-approved cross-site evaluation activities. The contractor has designed the client interview to include only the most pertinent information needed to be able to effectively carry out this cross-site evaluation. There will be no significant impact on these small entities.

6. Consequences If Information Collected Less Frequently

CSAT GBHI Client Interview – Baseline & CSAT GBHI Client Interview – 6-Month Follow-up: A client-level interview will be administered on a voluntary basis to clients who receive services under the CSAT GBHI grant. Only those clients who complete the initial baseline interview will be asked to complete a 6-month follow-up interview. Data collection at these follow-up points is necessary to measure the short- and longer-term outcomes of the CSAT GBHI program.

Following up at six months is optimal for producing useful outcome data. Waiting until six months after the initial receipt of services allows enough time for effects of the CSAT GBHI program to develop, including changes in housing status and stability, substance use behavior, mental health symptoms, and secondary outcomes, such as criminal justice involvement, employment, and trauma symptoms. Alternatively, waiting more than 6 months jeopardizes the validity of the data collected. As time passes, self-reported data become less accurate. Moreover, follow-up response rates, especially among much of the population to which CSAT GBHI is being delivered, decrease over time. Attrition is often systematically correlated with client characteristics, which may bias the measurement of changes in outcomes and preclude the generalization of those outcomes to a broader population.

CSAT GBHI Stakeholder Survey: This is a one time collection.

7. Consistency with the Guidelines in 5 CFR 1320.5(d)(2)

This information collection fully complies with the guidelines in 5 CFR 1320.5(d)(2).

8. Consultation Outside the Agency

The notice required by 5 CFR1320.8(d) was published in the Federal Register on August 2, 2010 (75 FR 45123-45124). No comments were received.

SAMHSA has made extensive use of experts in the area of homeless research, including current and previous CSAT GBHI grantees, to provide guidance on the design and analysis of the cross-site evaluation. An expert panel meeting was held in December 2009 to review the various aspects of the cross-site evaluation, including the preliminary evaluation plan, data collection procedures, economic analysis methods, and literature review. The experts provided feedback on all aspects of the evaluation and their comments and suggestions were incorporated into the development of the surveys. The list of experts is provided in Exhibit 1.

Exhibit 1: Expert Panel Members

Expert	Affiliation	Contact Information
Peggy Bailey, M.A.	Senior Policy Advisor Corporation for Supportive Housing 1731 Connecticut Ave, NW, 4th Floor Washington, DC 20009	Phone: 917-596-6337 E-mail: [email protected]
Gary Bond, Ph.D.	Professor of Psychiatry Dartmouth Psychiatric Research Center 2 Whipple Place, Suite 202 Lebanon, NH 03766	Phone: 603-448-0263 E-mail: [email protected]
Brian Dates, M.A.	Director of Evaluation and Research Southwest Counseling Solutions 1700 Waterman Detroit, MI 48209	Phone: 313-841-7442 E-mail: [email protected]
Anne Fletcher	Social Science Analyst U.S. Department of Housing & Urban Development Office of Policy Development & Research 451 7th Street, SW Room 8120 Washington, DC 20140	Phone: 202-402-4347 E-mail: [email protected]
Linda Frisman, Ph.D.	Director of Research Connecticut Department of Mental Health and Addiction Services 410 Capitol Avenue, MS # 14 RSD Hartford, CT 06134	Phone: 860-418-6788 Email: [email protected]
William McAllister, Ph.D.	Senior Research Fellow, Institute for Social and Economic Research and Policy Associate Director, Center for Homelessness Prevention Studies Columbia University 420 West 118th St, MC 3355 New York, NY 10027	Phone: 212-854-5781 Email: [email protected]
Stephen Metraux, Ph.D.	Associate Professor Department of Health Policy & Public Health University of the Sciences in Philadelphia 600 South 43rd Street Philadelphia, PA 19104-4495	Phone: 215-596-7612 E-mail: [email protected]
Roger H. Peters, Ph.D.	Chair and Professor Department of Mental Health Law and Policy University of South Florida Louis de la Parte Florida Mental Health Institute 13301 N. Bruce B. Downs Blvd. Tampa, Florida 33612-3807	Phone: 813-974-9299 E-mail: [email protected]
Laura Prescott, B.A.	President and Founder Sister Witness International 50 Union Street Greenfield, MA 01301	Phone: 413-212-4343 E-mail: [email protected]
Michael Rowe, Ph.D.	Associate Clinical Professor Program for Recovery & Community Health Department of Psychiatry Yale University School of Medicine 319 Peck Street, Building 1 New Haven, CT 06513	Phone: 203-764-8690 E-mail: [email protected]
David L. Shern, Ph.D.	President and Chief Executive Officer Mental Health America 2000 North Beauregard, Sixth Floor Alexandria, VA 22311	Phone: 703-813-1747 E-mail: [email protected]
David Smelson, Psy.D.	Professor, Vice Chair of Clinical Research Department of Psychiatry University of Massachusetts Medical School 55 Lake Avenue North Worcester, MA 01655-0002	Phone: 508-713-5420 E-mail: [email protected]
Mary E. Smith, Ph.D.	Associate Director Decision Support, Research and Evaluation Division of Mental Health Illinois Department of Human Services 160 North LaSalle Street, 10th Floor Chicago, IL 60601	E-mail: [email protected]
Sally J. Stevens, Ph.D.	Executive Director Southwest Institute for Research on Women University of Arizona 925 North Tyndall Avenue Tucson, AZ 85721-0438	Phone: 520-626-9558 E-mail: [email protected]

9. Payment to Respondents

CSAT GBHI Client Interview – Baseline & CSAT GBHI Client Interview – 6-Month Follow-up: CSAT GBHI clients, who are homeless individuals with substance use and/or mental health disorders, are typically a hard-to-reach transient population. To increase response rates, all clients who agree to participate in the client interview at baseline will receive a cash equivalent incentive worth a $10 value (e.g., gift card). Participants who complete the baseline will be asked to complete a 6-month follow-up interview. Clients who agree to participate in the 6-month follow-up will receive a cash equivalent incentive worth a $25 value (e.g., gift card). Respondents will not be penalized if they wish to skip questions or stop the interview during either the baseline or 6-month follow-up. Survey research literature suggests that monetary incentives have a strong positive effect on response rates and no known adverse effect on reliability. Research has shown improved response rates when remuneration is offered to respondents. Results from the 2001 National Household Survey on Drug Abuse (NHSDA) incentive experiment were reported by Wright, Bowman, Butler, & Eyerman (2005); key conclusions from their analyses are summarized below:

The $20 and $40 incentive payments each produced about a 10-point gain in overall response rates when compared with the $0 control group. The overall response rate was significantly higher for $40 than the $20 incentive within many of the subgroups addressed in the analysis. Both incentive payment groups more than paid for themselves due to decreased costs of follow-up and more productive screening resulting from the improved response rates. Incentives motivate (or obligate) respondents to admit to substance use that they might not have admitted without the incentive.

CSAT GBHI Stakeholder Survey: No cash incentives or gifts will be given to respondents.

10. Assurance of Confidentiality

Concern for privacy and protection of respondents’ rights will play a central part in the implementation of all study components. The contractor is developing the cross-site evaluation surveys and analyzing the data and has extensive experience protecting and maintaining the privacy of respondent data.

CSAT GBHI Client Interview – Baseline & CSAT GBHI Client Interview – 6-Month Follow-up: The process of administering the CSAT GBHI Client Interview – Baseline and 6-Month Follow-up is designed to protect client privacy, reduce client discomfort and burden, and ensure that the collected data are of the highest quality. The contractor is asking the CSAT GBHI grantee staff or local evaluators to collect the CSAT GBHI Client Interview – Baseline and 6-Month Follow-up data immediately following the administration of the SAMHSA-required GPRA interview. The contractor will hold training sessions with all FY2010 grantees to detail the steps involved in administering the client interview and the procedures to follow to ensure protection of respondent’s rights and safeguarding of client data. Grantee programs will be provided with a Client Interview Script (See Attachment 5), a Client Interview Informed Consent (See Attachment 6), and a Frequently Asked Questions (FAQ) handout (in development).

To begin the CSAT GBHI Client Interview – Baseline or 6-Month Follow-up, the interview administrator (hereafter referred to as ‘administrator’) will provide the client with a brief introduction to the interview and ask the client if they will agree to hear more. If the client agrees to proceed, the administrator will read the informed consent for the client interview to the client, who will sign it if he or she understands and agrees with its contents. The consent form will explain the purpose of the cross-site evaluation and the interview, describe the interview length and procedures, describe risks or benefits and steps the evaluation is taking to protect the client’s privacy, inform the client of the incentive, and inform them that the interview is voluntary and that he or she may refuse to answer a question or stop the interview at any point without penalty. The consent form will also include the OMB approval expiration dates, the statement of survey burden, and the statement that the study is federally sponsored. This process will take place in a private location to protect client privacy. The administrator will write the CSAT GBHI site number, the client’s GPRA ID number, and the Interviewer ID number on the first page of the interview. This is the only identifying information the evaluation will have access to; the evaluation will not know the client’s name or be able to connect client interview answers to a particular client.

The CSAT GBHI Client Interview – Baseline and 6-Month Follow-up each have two parts. In the first part of each interview, the administrator will read the questions to the client and mark the answers on the scantron form. This part of the interview is comprised of sections related to military service, employment, criminal justice, co-occurring disorders, housing and homeless history, housing satisfaction and choice, perception of housing coercion, readiness to change, services needed and received, client treatment burden, and trauma symptoms. The second part of the CSAT GBHI Client Interview – Baseline and 6-Month Follow-up includes sections related to perception of care, treatment coercion, and treatment choice. These sections will be completed by the client without the administrator present. The client will be provided information about the kinds of questions they will be answering and assistance in the correct way to use the scantron. The client will again be reminded he or she can refuse to answer questions or stop the interview completely. He or she will also be instructed not to write any identifying information on the form, like their name. If a client is illiterate, the administrator can assist the client in two ways. First, before the client answers anything, the administrator can explain how to answer yes/no questions or Likert scale questions by pointing out what those answers look like or explain which directions imply ‘better’ or ‘worse’. Second, the administrator may remain in the room with the client but in a location that prevents the administrator from seeing the client’s responses. While in the room the administrator may read each question to the client using a blank copy of the instrument that is not the instrument the client is filling out. As needed, the administrator may remind the client of the answer format and may point out what the answer options look like using the blank instrument. In the event this happens, the administrator will be instructed to follow two rules: 1) consistently remind the client to protect or hide their instrument or answers while the administrator is helping them using the blank instrument and 2) always point out or describe all possible answer choices for a given question to reduce the potential for bias. Once the client completes this portion of the survey, he or she will place the survey into a tamper proof/evident, postage-paid envelope and return it to the administrator who will mail both sections to the contractor for processing. Once received, they will both be scanned into a secure dataset.

All clients who complete the CSAT GBHI Client Interview – Baseline will be asked to participate in the CSAT GBHI Client Interview – 6-Month Follow-up. If they agree, the client will be given another informed consent outlining the same content as the baseline consent form. Again, they will be informed that participation is voluntary and they will not be penalized for non-participation. The 6-month follow-up will be administered by the grantee staff or the local evaluator in the same scantron format as the baseline following the same procedures outlined above. Client interviews will be identified only with the client GPRA number which will be necessary to link the baseline data with the 6-month follow-up data and to link the GPRA data with the CSAT GBHI Client Interview – Baseline and 6-Month Follow-up data; no personally identifying information will be given to the contractor.

CSAT GBHI Stakeholder Survey: The contractor will obtain limited contact information for stakeholders, including full name and e-mail address, to notify them of the survey. Stakeholders will be contacted through e-mail and issued a username and password to access the web-based survey for their grantee program. Each respondent will login to a secure web-based form to complete the survey. They will also be given the grantee program’s identification number which they will be asked to enter during the web survey. This will be the only identifying information linked to the stakeholder’s responses which will be used to link the responses to the appropriate grantee program. The stakeholders will be required to give electronic informed consent (see Attachment 7) before they begin answering questions. At no point will survey responses be linked to a specific stakeholder.

For all data collection activities, the contractor will use passwords to safeguard all project directories and analysis files containing completed survey data to ensure that there is no inadvertent disclosure of study data. Contractor staff will also be trained on handling sensitive data and the importance of privacy. All contractor staff will sign a privacy pledge (See Attachment 8). In addition, the three interviews, all informed consents and the client interview script have been reviewed and approved by the contractor’s Institutional Review Board (IRB) (Federal Wide Assurance Number 3331), approval #12612 (see Attachments 9 and 10). In keeping with 45 CFR 46, Protection of Human Subjects, the CSAT GBHI procedures for data collection, consent, and data maintenance are formulated to protect respondents’ rights and the privacy of information collected. Strict procedures will be followed for protecting the privacy of respondents’ information and for obtaining their informed consent. The IRB-approved model informed consents meet all Federal requirements for informed consent documentation. This template will be customized by each grantee to obtain informed consent for participation in the study. Any necessary changes to the surveys will be reviewed by the contractor’s IRB.

Data from the CSAT GBHI client interviews will be safeguarded in compliance with the Privacy Act of 1974 (5 U.S.C. 552a). The privacy of data records will be explained to all respondents during the consent process and in the consent forms.

11. Questions of a Sensitive Nature

CSAT GBHI Client Interview – Baseline & CSAT GBHI Client Interview – 6-Month Follow-up: The client interviews, by necessity, will collect sensitive information about homelessness, substance abuse, mental health, and criminal justice involvement as these are all outcomes of interest to SAMHSA. Also, the CSAT GBHI Expert Panel strongly endorsed including a measure regarding trauma symptoms. The client interview will ask clients about trauma symptoms they may be experiencing but they will not be asked about specific traumatic events. If these questions cause any distress for the client, the interview administrator will connect them with someone from the grantee program who they can speak with. Also, two sections included in both the CSAT GBHI Client Interview – Baseline and the 6-Month Follow-up interviews, Perception of Care and Treatment Choice, will be self-administered to eliminate discomfort a client may feel in giving their feedback about the program to program staff. Sensitive information of this nature is always regarded as private, and privacy for clients in federally assisted treatment programs is assured through strict adherence to Federal Regulation 42 CFR, Part 2. All client interviews will be conducted in a private space and the administrator will first obtain consent for participation. Respondents will be informed about the purpose of the data collection and that responding to all interview questions is voluntary. They will be assured that they may stop taking the interview at any time without forfeiting the incentive and without penalty from the grantee program. In addition, specific assurances will be provided to respondents concerning the safety and protection of data collected from them. Respondents’ names or other personally identifying information will not be linked to data collected.

CSAT GBHI Stakeholder Survey: No sensitive information will be collected from the grantee stakeholders.

12. Estimates of Annualized Hour Burden

Estimate the annualized hour burden of the collection of information from clients. The total client sample size for the CSAT GBHI cross-site data collection effort is estimated to be a maximum of 7,356 respondents based on each of the 25 FY2010 grantee target enrollment numbers from year two through year five. The baseline survey is expected to have a response rate of 80%, therefore resulting in 5,885 respondents completing the baseline survey. The 6-month follow-up survey is expected to have a response rate of 80% of the baseline sample, leaving 4,708 respondents with baseline and follow-up data. Exhibit 2 presents estimates of annualized burden based on preliminary testing. As evidenced from the testing, including time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information, the total estimated time to complete the baseline survey is 20 minutes. The 6-month follow-up survey drops two small sections from the baseline survey but adds one longer section and it is estimated that it will take 24 minutes to complete.

Exhibit 2. Cross-Site Data Collection Burden for the CSAT GBHI Client Interview – Baseline, CSAT GBHI Client Interview – 6-Month Follow-up, & CSAT GBHI Stakeholder Survey

Instrument/Activity	Number of Respondents	Responses per Respondent	Total Number of Responses	Hours per Response	Total Burden Hours	Hourly Wage	Total Respondent Costa
Baseline data collection (Clients)	5,885	1	5,885	.33	1,942	$20.76	$40,316
6-month follow-up data collection (Clients)	4,708	1	4,708	.40	1,883	$20.76	$39,091
Client Subtotal	10,593		10,593		3,825	$20.76	$79,407
Stakeholder Survey	648	1	648	.28	181	$31.88	$5,770
TOTAL	11,241		11,241		4,006		$85,177

^aTotal respondent cost is calculated as hourly wage × time spent on survey × total number of responses.

Estimate the annualized hour burden of the collection of information from grantee stakeholders. The total stakeholder sample size is estimated to be 648 (8 responses per the 25 FY2010 grantee sites and 4 responses per the 112 previous grantee sites). The stakeholder web survey is estimated to take 17 minutes to complete. Exhibit 2 presents estimates of annualized burden based on preliminary testing.

Estimate the annualized cost burden to the respondent for the collection of information from clients. There are no direct costs to respondents other than their time to participate in the interview. The total cost of the time respondents spend completing these surveys is $79,407 (number of total baseline client respondent hours plus follow-up respondent hours × $20.76, the estimated average hourly wages for adults as published by the Bureau of Labor Statistics (2008) inflated to 2010 value). The annualized cost is approximately $19,852.

Estimate the annualized cost burden to the respondent for the collection of information from stakeholders. There are no direct costs to respondents other than their time to participate in the study. The total cost of the time respondents spend completing these surveys is $5,770 (number of stakeholder respondent hours × $31.88, the estimated average hourly wages for individuals working in professional managerial occupations as published by the Bureau of Labor Statistics (2008) inflated to 2010 value). The annualized cost is approximately $1,442.

13. Estimates of Annualized Cost Burden to Respondents

There are no respondent costs for capital or start-up or for operation or maintenance.

14. Estimates of Annualized Cost to the Government

The estimated cost to the government for the data collection is $261,220.  This includes approximately $250,000 for cost of materials, programming, incentives, trainings, contractor labor, housing and maintaining data, and approximately $2,805 per year represents SAMHSA costs to manage/administer the survey for 2% of one employee (GS-15).  The annualized cost is approximately $65,305.

15. Changes in Burden

This is a new collection of information.

16. Time Schedule, Publications, and Analysis Plan

Time Schedule: Exhibit 3 outlines the key time points for the study and for the collection of information. The requested period also allows for training and start-up activities associated with the preparation for data collection.

Exhibit 3. Time Schedule for Entire Project

Activity	Time Schedule
Obtaining OMB approval for data collection	Winter 2010 (Q2 FY2011)
CSAT GBHI Client Interview - Baseline Implementation Begins	Spring 2011 (Q3 FY2011)
CSAT GBHI Stakeholder Survey Data Collection Begins	Summer 2011 (Q4 FY2011)
CSAT GBHI Client Interview - 6-month Follow-up Implementation Begins	Fall 2011 (Q1 FY2012)
All CSAT GBHI Client Interview Data Collection Ends	Spring 2014 (Q3 FY2014)
CSAT GBHI Stakeholder Survey Data Collection Ends	Spring 2014 (Q3 FY2014)
Data analysis	Fall 2011 (Q1 FY2012) - Summer 2014 (Q4 FY2014)
Dissemination of findings Interim reports, presentations, manuscripts, final report	Fall 2011 (Q1 FY2012) - Summer 2014 (Q4 FY2014)

Publications: The CSAT GBHI cross-site evaluation is designed to produce knowledge about the implementation and impact of CSAT GBHI programs. It is therefore important to prepare and disseminate reports, concept papers, documents, and oral presentations that clearly and concisely present project results so that they can be appreciated by both technical and nontechnical audiences. The contractor will:

• Produce rapid-turnaround analysis papers, briefs, and reports;

• Prepare and submit monthly technical progress reports, semi-annual briefings and annual progress reports;

• Prepare special reports in concert with CHAB and expert panel input. For example, the contractor plans to submit a “portrait” of the CSAT GBHI grantee and client characteristics;

• Prepare final cross-site findings report, including an executive summary;

• Deliver presentations at professional and federally sponsored conventions and meetings; and

• Disseminate reports and materials to entities inside and outside CSAT.

Analysis:

CSAT GBHI Client Interview – Baseline & CSAT GBHI Client Interview – 6-Month Follow-up:

The outcome evaluation component focuses on examining the utility of future CSAT GBHI cohorts through the review of planned and actual outcomes. Within the context of the social-ecological framework, the outcome evaluation will focus on the effects of the CSAT GBHI programs on client outcomes, accounting for grantee characteristics and treatment system and community contexts. HLM will be used to estimate the mean change in client-level outcomes between baseline and follow-up. HLM is appropriate for these analyses because this modeling approach allows the contractor to control for the clustering of clients within grantee. Within the HLM framework, the contractor will adjust for client characteristics. These adjusted mean changes will provide a rigorous, yet easy-to-understand, assessment of program impact. Separate analyses will address the impact of grantee characteristics (such as program model) on client-level outcomes, controlling for client characteristics. As appropriate, subgroup analyses will be conducted in which the data will be stratified by program type or client type to assess whether outcomes differ among the different types of programs or for different types of client (e.g., veterans or women).

As described in Attachment 11, the GBHI cross-site evaluation supplemental data from the CSAT GBHI Client Interview – Baseline and 6-Month Follow-up (contingent on OMB approval) will be combined with data from the CSAT GPRA Client Outcome Measures, information gathered from the Grantees describing their program components, and data from secondary sources such as SAIS GPRA and Technical Assistance data, the National Survey of Substance Abuse Treatment Services (N-SSATS) and the Treatment Episode Data Set (TEDS) to develop a comprehensive portrait of the GBHI client populations, the needs of these populations, the services provided to address those needs, and the outcomes across a multitude of domain areas for those participating in GBHI programs. These supplemental data will provide mediating and moderating variables, as well as information on client characteristics not covered by the CSAT GPRA survey. The areas addressed by the supplemental data collection include service need, burden, satisfaction/perception of care, the form of care or individually tailored care, model adaption, homelessness, housing (placement/safety/perceived choice/perceived value), readiness for change, and co-occurring mental disorders. Three additional domains (services, trauma and veteran’s service era and combat information) were added in response to the recommendations of the expert panel and SAMHSA and confirmed with the GBHI Grantees. The additional services data will improve our ability to describe the relationships between treatment plans and abstinence and housing stability including measuring the extent to which models of matching services to needs are being used and the appropriate dosage of services as described in the literature for these models. As CSAT GPRA data includes administrative data on services received only at discharge, it impossible to assess whether and how service receipt changes over time using only GPRA data alone. The GPRA data does not collect this information from the client or address perceived need and service matching. The supplemental data will address this limitation. Additionally, the panelists recommended measuring trauma symptoms given that trauma is prevalent in the homeless population (e.g., Browne & Bassuk, 1997; Goodman, 1991; Bassuk et al., 1996; Burt et al., 1999; HUD, 2009; Shelton et al., 2009) and without intervention consistently predicts negative substance abuse, employment, housing and criminal justice outcomes. Finally, given the high prevalence of homelessness among returning veterans and differentially by service era (Kline et al., 2009), along with there being several Grantee programs focused solely on veterans, baseline collection of veteran service era was recommended.

We conducted a literature review that helped advance our thinking about likely influences on client-, grantee-, and system-level outcomes (Broner et al., 2010). As we developed our data collection and analysis plans, we used information from the review to strengthen the evaluation’s ability to provide insightful findings on what works for whom, under what approaches, and in what systems and contexts. At the client level, demographic characteristics (sex, age, race or ethnicity), parental status, educational attainment, veteran status (for recent cohorts), disability, social supports, and involvement with the criminal justice system can be important with respect to understanding the appropriateness and expected effectiveness of specific approaches. Client differences in substance abuse, mental illness, and co-morbidity are of central importance to GBHI. Our data collection and analyses will allow us to describe how client populations differ on these factors across study sites and test whether these factors are associated with differential program choices, components and successful provision of services, including housing the clients. For example, by collecting gender at the client level, we will assess whether programs are better able to provide appropriate services for female clients than for male clients. Clients will also differ in their levels of participation, program completion, and treatment compliance. Information from the supplemental data collection will enhance the CSAT GPRA information from the SAIS discharge data to allow us to estimate what client characteristics are significantly associated with participation at 6 month follow-up and to test whether participation mediates the programs’ ability to carry out full services objectives.

The outcome evaluation component focuses on addressing the “utility” element of the evaluation’s Objective 1, which per SAMHSA’s RFA was is to “examine the feasibility, utility, and sustainability of future Treatment of Homeless cohorts through the review of planned and actual outcomes.” The outcome evaluation will focus on the changes in client outcomes that are associated with differences in grantee models. The findings will be framed in a pre-post quasi-experimental design that will allow us to examine the relationship of both intent-to-treat and service receipt from to outcomes. HLM will be used to estimate the mean change in client-level outcomes between baseline and follow-up. HLM is appropriate for these analyses because this modeling approach allows us to control for the clustering of clients within grantee. Within the HLM framework, we will adjust for client characteristics and other contextual factors. These adjusted mean changes will provide easy-to-understand estimates of possible program impact. Although these estimates are not intended to be causally interpreted, we do intend to compare them to estimates for similar models and populations in the scientific literature to confirm that they are within ranges that we would expect, conditional on the level of adherence to the models that we observe for each grantee. These estimates form a baseline for exploring how program decisions and characteristics alter service delivery and outcomes. In this way, variation among the 25 Grantees will serve as experimental variation for analyzing ‘key ingredients’ of models for achieving different outcomes, such as linking clients to certain types of housing. As appropriate, subgroup analyses will be conducted in which the data will be stratified by program type or client type to assess whether outcomes differ among the different types of programs or for different types of client (e.g., veterans or women). Sample data analysis shells are presented in Attachment 11.

CSAT GBHI Stakeholder Survey:

Along with existing documents (narrative reports produced by grantees consisting of the original grant application, quarterly reports to project officers, and the annual continuation application and progress reports), key informant interviews, and site visits to the FY2010 grantee programs, stakeholder responses to the web-survey will provide crucial information on grantee structure and process including information on barriers, solutions and innovative strategies for successful implementation. Systematic qualitative analyses will be conducted using the software package ATLAS.ti. Key results will be identification of prominent themes regarding model choice and implementation success across the grantees. Rank ordered themes will be presented as well as descriptive statistics on web survey responses (means, medians, ranges). When appropriate for answering evaluation hypotheses, pairwise or ANOVA/ANCOVA statistical tests will be conducted to determine differences in responses across programs and model types. Finally, web survey responses will provide program-level independent variables that will be incorporated into the client-level outcome model described above. Specifically, the contractor will test whether these program elements are significant moderators or mediators of client outcomes.

17. Display of Expiration Date

OMB approval expiration dates will be displayed on the CSAT GBHI Client Interview – Baseline, CSAT GBHI Client Interview – 6-Month Follow-up, the client interview consent forms, and on the informed consent screen of the CSAT GBHI Stakeholder Survey.

18. Exceptions to Certification for Statement

There are no exceptions to the certification statement. The certifications are included in this submission.

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

CSAT GBHI Client Interview – Baseline & CSAT GBHI Client Interview – 6-Month Follow-up: The targeted universe for the CSAT GBHI client interview is all enrolled and accepted clients who receive services under the CSAT GBHI FY2010 grant. Eligibility for the receipt of CSAT GBHI services is limited to homeless individuals with substance use disorders, mental health disorders, or co-occurring substance use and mental health disorders. Based on the target enrollment numbers in Years 2-5 for the combined 25 FY2010 grantees, the expected total number of clients receiving services is 7,356.

CSAT GBHI Stakeholder Survey: Stakeholder names will be generated based on data extraction from grantee-submitted SAMHSA documents (e.g., grantee applications) and reviewed through follow-up with grantees for all grantee programs funded from 2004-2009 (FY2005-FY2010). The numbers of grantee partnerships vary widely so it is expected that grantees will recommend between 3 and 10 essential partners who have provided services under their CSAT GBHI initiative for the contractor to contact. These stakeholder partners will be contacted through email and provided a link to the web survey with a username and password for secure log-in.

2. Information Collection Procedures

CSAT GBHI Client Interview – Baseline & CSAT GBHI Client Interview – 6-Month Follow-up:

As described in Section A.6, the CSAT GBHI client interview will collect data from individuals at baseline and at 6-month follow-up. Data collection at the follow-up point is necessary to measure the short- and longer-term outcomes of the CSAT GBHI programs implemented by the grantees. Because measuring these outcomes is one of the primary objectives of the CSAT GBHI initiative, less frequent than semiannual data collection would greatly endanger the utility of the CSAT GBHI initiative to all clients.

The process of administering the CSAT GBHI Client Interview – Baseline and 6-Month Follow-up is designed to protect client privacy, reduce client discomfort and burden, and ensure that the collected data are of the highest quality. The contractor is asking the CSAT GBHI grantee staff or local evaluators to collect the CSAT GBHI Client Interview – Baseline and 6-Month Follow-up data immediately following the administration of the SAMHSA-required GPRA interview. As described in detail in Attachment 11, these grantee interviewers are trained interviewers who have received training on interview administration, participant engagement, participant protection, and tracking procedures, from the grantee as well as from SAMHSA per OMB approved procedures (OMB control number 0930-0208) developed for the GPRA Client Outcome Measures. The contractor will hold training sessions with all FY2010 grantees to detail the steps involved in administering the client interview and the procedures to follow to ensure protection of respondent’s rights and safeguarding of client data. Grantee programs will be provided with a Client Interview Script, a Client Interview Informed Consent, and a Frequently Asked Questions (FAQ) handout.

To begin the CSAT GBHI Client Interview – Baseline or 6-Month Follow-up, the administrator will provide the client with a brief introduction to the interview and ask the client if they will agree to hear more. If the client agrees to proceed, the administrator will read the informed consent for the client interview to the client, who will sign it if he or she understands and agrees with its contents. The consent form will explain the purpose of the cross-site evaluation and the interview, describe the interview length and procedures, describe risks or benefits and steps the evaluation is taking to protect the client’s privacy, inform the client of the incentive, and inform them that the interview is voluntary and that he or she may refuse to answer a question or stop the interview at any point without penalty. The consent form will also include the OMB approval expiration dates, the statement of survey burden, and the statement that the study is federally sponsored. This process will take place in a private location to protect client privacy. The administrator will write the CSAT GBHI site number, the client’s GPRA ID number, and the Interviewer ID number on the first page of the interview. This is the only identifying information the evaluation will have access to; the evaluation will not know the client’s name or be able to connect client interview answers to a particular client.

The CSAT GBHI Client Interview – Baseline and 6-Month Follow-up each have two parts. In the first part of each interview, the administrator will read the questions to the client and mark the answers on the scantron form. This part of the interview is comprised of sections related to military service, employment, criminal justice, co-occurring disorders, housing and homeless history, housing satisfaction and choice, perception of housing coercion, readiness to change, services needed and received, client treatment burden, and trauma symptoms. The second part of the CSAT GBHI Client Interview – Baseline and the 6-Month Follow-up includes sections related to perception of care, treatment coercion, and treatment choice. These sections will be completed by the client without the administrator present. The client will be provided information about the kinds of questions they will be answering and assistance in the correct way to use the scantron. The client will again be reminded he or she can refuse to answer questions or stop the interview completely. He or she will also be instructed not to write any identifying information on the form, like their name. If a client is illiterate, the administrator can assist the client in two ways. First, before the client answers anything, the administrator can explain how to answer yes/no questions or Likert scale questions by pointing out what those answers look like or explain which directions imply ‘better’ or ‘worse’. Second, the administrator may remain in the room with the client but in a location that prevents the administrator from seeing the client’s responses. While in the room the administrator may read each question to the client using a blank copy of the instrument that is not the instrument the client is filling out. As needed, the administrator may remind the client of the answer format and may point out what the answer options look like using the blank instrument. In the event this happens, the administrator will be instructed to follow two rules: 1) consistently remind the client to protect or hide their instrument or answers while the administrator is helping them using the blank instrument and 2) always point out or describe all possible answer choices for a given question to reduce the potential for bias. Once the client completes this portion of the survey, he or she will place the survey into a tamper proof/evident, postage-paid envelope and return it to the administrator who will mail both sections to the contractor for processing. Once received, they will both be scanned into a secure dataset.

All clients who complete the CSAT GBHI Client Interview – Baseline will be asked to participate in the CSAT GBHI Client Interview – 6-Month Follow-up. If they agree, the client will be given another informed consent outlining the same content as the baseline consent form. Again, they will be informed that participation is voluntary and they will not be penalized for non-participation. The 6-month follow-up will be administered by the grantee staff or the local evaluator in the same scantron format as the baseline following the same procedures outlined above. Client interviews will be identified only with the client GPRA number which will be necessary to link the baseline data with the follow-up data and to link the GPRA data with the CSAT GBHI Client Interview – Baseline and the 6-Month Follow-up. The contractor will not have any contact with the clients between baseline and follow-up. However, follow-up success will benefit from the fact that CSAT GBHI grantees are providing case management or other services that keep them in ongoing engagement with the clients. Furthermore, they are conducting their own administrative data collection that requires them to maintain contact with the clients. A detailed description of grantee processes for maintaining client contact and rates of retention are detailed in Attachment 11.

As described above in B.1, the contractor intends to administer the CSAT GBHI Client Interview – Baseline and the 6-Month Follow-up to all clients that the CSAT GBHI grantees intake and in all programs. The only challenge to representativeness is that the programs began providing services to clients in the beginning of 2010—prior to the evaluation’s data collection effort. Nonetheless, evaluation data will reflect the outcomes associated with mature and stable programs and will cover the majority of clients served over the performance period of the grants. Fortunately, GPRA data is collected on all clients served (as required by SAMHSA) which will allow the contractor to analyze any differences in client baseline characteristics between clients who responded to the CSAT GBHI Client Interview – Baseline and 6-Month Follow-up and clients who were already enrolled prior to implementation.

CSAT GBHI Stakeholder Survey: The contractor will obtain limited contact information for stakeholders, including full name and e-mail address, to notify them of the survey. Stakeholders will be contacted through e-mail and issued a username and password to access the web-based survey for their grantee program. Each respondent will login to a secure web-based form to complete the survey. They will also be given the grantee program’s identification number which they will be asked to enter during the web survey. This will be the only identifying information linked to the stakeholder’s responses which will be used to link the responses to the appropriate grantee program. The stakeholders will be required to give electronic informed consent before they begin answering questions. At no point will survey responses be linked to a specific stakeholder. The stakeholder web survey will only be administered one time.

3. Methods to Maximize Response Rates

CSAT GBHI Client Interview – Baseline & CSAT GBHI Client Interview – 6-Month Follow-up: The ability to gain the cooperation of potential respondents is key to the success of this endeavor. All grantees are required by SAMHSA to administer the GPRA interview to 100% of clients who enter treatment under the CSAT GBHI grant. In addition, a minimum of 80% of clients must also receive the 6-month follow-up GPRA interview. In order to increase the likelihood of client response and ease the burden placed on both client and grantee, the client interview will be administered immediately following the GPRA interview. The contractor anticipates an 80% to 85% response rate for the CSAT GBHI Client Interview – Baseline and a 15% to 20% attrition rate for the CSAT GBHI Client Interview –6-Month Follow-up (see Attachment 11). The contractor will employ several strategies to maintain high response rates in the CSAT GBHI Client Interview – Baseline and 6-Month Follow-up:

• Stress the importance of the project as well as the contractor’s commitment to respondent privacy.

• Train survey staff for handling sensitive information collection in a respectful manner.

• Administer the survey immediately following the administration of the required GPRA interview.

• Offer cash equivalent incentives (e.g., gift cards) for survey response.

CSAT GBHI Stakeholder Survey: To recruit participants for the stakeholder survey, the contractor will ask grantee project directors to nominate representatives from each of their key partner agencies or organizations and will then contact the nominees to ask that they participate in the survey. In a recent study that also used a web survey of partners of grantees, the evaluation team achieved a response rate of 60% (1,353 respondents of 2,278 invited), with an average of 8 respondents in each of 169 sites. As in that study, it is anticipated that all nominees will have access to the web because they represent agencies or organizations and their involvement in partnering with the grantee is part of their job – and can therefore access the web via a computer in their office. Additional findings regarding prior response rates to similar stakeholder web-based surveys implemented by the contractor is presented in Attachment 11. Although web survey respondents will not be provided incentives, nominees are nominated because they are actively involved in the partnership and therefore are typically motivated to share their experiences and perspectives. To be successful and useful, the stakeholder web survey does not need to achieve response rates at the same level of the client interview. The main consideration is that some partners from each site respond; it is not necessary that all, or even most, partners in a site respond. The contractor will use several strategies to achieve sufficient response rates in the stakeholder survey:

• Ask grantee project directors to inform their nominated partners about the survey and encourage them to participate.

• Send nominees an initial email invitation that explains the study and its importance, why they are being asked to participate, how they can contact the contractor for additional information, and how to access the web survey.

• Send reminder emails to non-respondents and, if approved by CSAT, ask grantee project directors to also encourage non-respondents to participate.

• Keep the survey to a reasonable length that encourages participation and will not lead to “word of mouth” comments among nominees that discourage participation.

• If needed, allow respondents some other way to take the survey other than over the web (e.g. mailed hard copy or conducted over the telephone).

4. Test of Procedures

CSAT GBHI Client Interview – Baseline & CSAT GBHI Client Interview – 6-Month Follow-up: A pencil-and-paper version of the baseline and 6-month follow-up client interviews were tested with six respondents (using contractor staff, including previous homeless consumers) and found that the baseline interview, including informed consent, takes approximately 20 minutes to complete. The 6-month follow-up interview, including informed consent, takes approximately 24 minutes. The practice tests were timed using a variety of answer patterns; the time required to complete the surveys varies with client characteristics, particularly history of homelessness, housing stability and substance use. The range of times from the baseline interview testing was 16 minutes to 23 minutes and the range of times from the 6-month follow-up interview was 20 minutes to 31 minutes.

In preparation for potential implementation of the CSAT GBHI Client Interviews, two other pilot tests were implemented each replicating these findings regarding timing and acceptability of the test procedures with active clients (see Attachments 11 and 12). Further, as described in Attachments 11 and 12, the surveys and accompanying procedures were reviewed with each of the 25 grantees who confirmed the utility of the potential data and that they believed the supplemental surveys would be acceptable to clients and not overly burdensome to the program and specifically the GPRA interviewers.

CSAT GBHI Stakeholder Survey: A pencil-and-paper version of the web survey was tested with six respondents (using contractor staff, including previous stakeholders) and it was found to take approximately 17 minutes to complete, including the informed consent. The web survey contains a number of skip patterns and response times will vary based on the services offered by the stakeholder. The practice tests were completed using hypothetical stakeholders who offered a range of services from none (e.g., a stakeholder who supplies funding only) to full services (e.g., a stakeholder who offers housing, substance abuse and mental health services) which will be uncommon. It is also likely that the web-based form will take less time than the paper version as the skip patterns will run automatically for the respondent. The range of times from the testing was 13 minutes to 20 minutes.

5. Statistical Consultants

As noted in Section A.8, SAMHSA has consulted extensively with an expert panel who will continue to provide expert advice throughout the course of the evaluation. In addition, the contractor team is comprised of several experts who will be directly involved in the data collection and statistical analysis. Also, contractor in-house experts will be consulted throughout the program on various statistical aspects of the design, methodological issues, economic analysis, database management, and data analysis. Exhibit 4 provides details of these team members and advisors.

Exhibit 4. Data Collection and Analysis Team Members and Advisors

Expert	Affiliation	Contact Information
Nahama Broner, Ph.D. Cross-site Evaluation Project Director	Senior Research Psychologist Crime, Violence and Justice Research Program RTI International 121 West 27th Street, Suite 1001 New York, NY 10001	Phone: 212-367-1990 Fax: 212-367-8060 E-mail: [email protected]
Alexander Cowell, Ph.D. Cross-site Evaluation Cost Analysis Team Leader	Economist Behavioral Health Economics Program RTI International 3040 Cornwallis Road Research Triangle Park, NC 27709	Phone: 919-541-8754 Fax: 919-485-5555 E-mail: [email protected]
John Hornik, Ph.D. Cross-site Evaluation Data Analysis Co-Team Leader	Director of Research Advocates for Human Potential 59 Carriage Lane Amherst, MA 01002	Phone: 413-687-5246 E-mail: [email protected]
Georgia Karuntzos, Ph.D. Advisor	Acting Director Risk Behavior and Family Research Program Associate Director Behavioral Health Research Division RTI International 3040 Cornwallis Road Research Triangle Park, NC 27709	Phone: 919-541-6159 Fax: 919-485-5555 E-mail: [email protected]
Pamela Lattimore, Ph.D. Cross-site Evaluation Data Analysis Co-Team Leader	Principal Scientist Crime, Violence and Justice Research Program RTI International 3040 Cornwallis Road Research Triangle Park, NC 27709	Phone: 919-485-7759 Fax: 919-485-7700 E-mail: [email protected]
Karl Maxwell, M.P.A. Cross-site Evaluation Task Order Officer	Public Health Advisor Center for Substance Abuse Treatment Substance Abuse and Mental Health Services Administration Room # 5-1062 1 Choke Cherry Road Rockville, MD 20857	Phone: 240-276-2824 E-mail: [email protected]
James Trudeau, Ph.D. Cross-site Evaluation Data Collection Team Leader	Director Crime, Violence and Justice Research Program RTI International 3040 Cornwallis Road Research Triangle Park, NC 27709	Phone: 919-485-7751 Fax: 919-485-7700 E-mail: [email protected]
Gary A. Zarkin, Ph.D. Advisor	Director Behavioral Health Research Division RTI International 3040 Cornwallis Road Research Triangle Park, NC 27709	Phone: 919-541-5858 Fax: 919-541-6683 E-mail: [email protected]

REFERENCES

Ball, S. A., Martino, S., Nich, C., Frankforter, T. L., van Horn, D., Crits-Christoph, P., et al. (2007). Site matters: Multisite randomized trial of motivational enhancement therapy in community drug abuse clinics. Journal of Consulting and Clinical Psychology, 75, 556–567.

Bassuk, E. L., Weinreb, L. F., Buckner, J. C., Browne, A., Salomon, A., & Bassuk, S. S. (1996). The characteristics and needs of sheltered homeless and low-income housed mothers. Journal of  the American Medical Association, 276, 640–646.

Borsari, B., & Carey, K. B. (2000). Effects of a brief motivational intervention with college student drinkers. Journal of Consulting and Clinical Psychology, 68, 728–733.

Broner, N. (2010, March). Sustaining jail diversion: A qualitative study of strategic planning by SAMHSA’s jail diversion targeted expansion capacity grants. Presented at the CMHS National GAINS Center Conference, Orlando, FL.

Broner, N. (2010, April). Using program evaluation to improve outcomes and inform program management and sustainability. Presented at the SAMHSA CMHS Annual Services for Supportive Housing Grantee Conference, Alexandria, VA.

Brown, J. M., & Miller, W. R. (1993). Impact of motivational interviewing on participation and outcome in residential alcoholism treatment. Psychology of Addictive Behaviors, 7, 211–218.

Browne, A., & Bassuk, S. S. (1997). Intimate violence in the lives of homeless and poor housed women: prevalence and patterns in an ethnically diverse sample. American Journal of Orthopsychiatry, 72, 261–277.

Burt, M. R. (2009). Life after transitional housing: Tracking homeless families after they leave HUD-assisted transitional housing. Urban Institute.

Burt, M. R., Aron, L. Y., Douglas, T., Valente, J., Lee, E., & Iwen, B. (1999). Homelessness: Programs and the people they serve. Washington, DC: Interagency Council on the Homeless.

Caton, C., Wilkins, C., & Anderson, J. (2007). People who experience long-term homelessness: Characteristics and interventions. In D. Dennis, G. Locke, & J. Khadduri (Eds.), Toward understanding homelessness: The 2007 National Symposium on Homelessness Research (pp. 4-1 through 4-44). Washington, DC: U.S. Department of Housing and Urban Development and U.S. Department of Health and Human Services. Retrieved October 26, 2009, from http://aspe.hhs.gov/hsp/homelessness/symposium07/

Cheng, A. L., & Kelly, P. J. (2008). Impact of an integrated service system on client outcomes by gender in a national sample of a mentally ill homeless population. Gender Medicine, 5, 395–404.

Conrad, K. J., Hultman, C. I., Pope, A. R., Lyons, J. S., Baxter, W. C., Daghestani, A. N., et al. (1997). Case managed residential care for homeless addicted veterans: Results of a true experiment. Medical Care, 36, 40–53.

Drake, R. E., McHugo, G. J. & Noordsy, D. L. (1993). Treatment of alcoholism among schizophrenic patients: Four-year-outcomes. American Journal of Psychiatry, 150, 328–329.

Drake, R. E., Yovetich, N. A., Bebout, R. R., Harris, M., & McHugo, G. J. (1997). Integrated treatment for dually diagnosed homeless adults. Journal of Nervous and Mental Disease, 185, 298–305.

Ganju, V. (1999). The MHSIP Consumer Survey: History, development, revisions, applications, commonly-asked questions. Washington, DC: National Institute of Mental Health, Mental Health Statistics Improvement Program. Planning, Research & Evaluation, Texas Department of MHMR. Retrieved February 2, 2010 from http://www.mhsip.org/mhsiptest/documents/MHSIPConsumerSurvey.pdf.

Goodman, L., Saxe, L., & Harvey, M. (1991). Homelessness as psychological trauma. American Psychologist, 46, 1219-1225.

Greenberg, G. A., & Rosenheck, R. A. (2008). Jail incarceration, homelessness and mental health: A national study. Psychiatric Services, 59, 170–177.

Greenwood, R. M., Schaefer-McDanile, N. J., Winkel, G., & Tsemberis, S. J. (2005). Decreasing psychiatric symptoms by increasing choice in services for adults with histories of homelessness. American Journal of Community Psychology, 36, 223–238.

Hiday, V., Swartz, M., Swanson, J., Borum, R., & Wagner, H. (1999). Criminal victimization of persons with severe mental illness. Psychiatric Services, 50, 62–68.

Joseph, H., & Langrod, J. G. (2004). The homeless. In J. H. Lowinson, P. Ruiz, R. B. Millman, & J. G. Langrod (Eds.), Substance abuse: A comprehensive textbook (4th ed.; pp. 1141–1168). Philadelphia: Lippincott, Williams, & Wilkins.

Kertesz, S. G., Crouch, K., Milby, J. B., Cusimano, R. E., & Schumacher, J. E. (2009). Housing First for homeless persons with active addiction: Are we overreaching? The Milbank Quarterly, 87, 495–534.

Kline, A., Callahan, L., Butler, M., St. Hill, L., Losonczy, M. F., & Smelson, D. A. (2009). The relationship between military service era and psychosocial treatment needs among homeless veterans with a co-occurring substance abuse and mental health disorder. Journal of Dual Diagnosis, 5, 357–374.

Lang, A. J., & Stein, M. B. (2005). An abbreviated PTSD checklist for use as a screening instrument in primary care. Behaviour Research and Therapy, 43, 585–594.

Mallett, S., Rosenthal, D., & Keys, D. (2005). Young people, drug use, and family conflict: Pathways into homelessness. Journal of Adolescence, 28, 185–199.

Martell, D. A., Rosner, R., & Harmon, R. B. (1995). Base-rate estimates of criminal behavior by homeless mentally ill persons in New York City. Psychiatric Services, 46, 596–601.

Miller, W. R., Benefield, G., & Tonigan, J. S. (1993). Enhancing motivation for change in problem drinking: A controlled comparison of two therapist styles. Journal of Consulting and Clinical Psychology, 61, 455–461.

Mueser, K. T., Drake, R. E., & Miles, K. M. (1997). The course and treatment of substance use disorder in persons with severe mental illness. In L. S. Onken, J. D. Blaine, S. Genser, & A. M. Horton (Eds.), Treatment of drug-dependent individuals with comorbid mental disorders (pp. 86–109). Rockville, MD: National Institute on Drug Abuse.

National Coalition for the Homeless. (2009). Substance abuse and homelessness. Retrieved October 17, 2009, from http://www.nationalhomeless.org/factsheets/addiction.html

National Law Center on Homelessness and Poverty; National Coalition for the Homeless. (2009). Homes not handcuffs: The criminalization of homelessness in U.S. cities. Washington, DC: Authors. Retrieved November 10, 2009, from http://www.nationalhomeless.org/publications/crimreport/crimreport_2009.pdf

Nelson, G., Aubry, T., & Lafrance, A. (2007). A review of the literature on the effectiveness of housing and support, Assertive Community Treatment, and intensive case management interventions for persons with mental illness who have been homeless. American Journal of Orthopsychiatry, 77, 350–361.

North, C. S., Eyrich, K. M., Pollio, D. E., & Spitznagel, E. L. (2004). Are rates of psychiatric disorders in the homeless population changing? American Journal of Public Health, 94, 103–108.

Pickett-Schenk, S. A., Cook, J. A., Grey, J. A., Banghart, M., Rosenheck, R. A., & Randolph, F. (2002). Employment histories of homeless persons with mental illness. Community Mental Health Journal, 38, 199-211.

Project MATCH Research Group. (1997). Matching alcoholism treatments to client heterogeneity: Project MATCH posttreatment drinking outcomes. Journal of Studies on Alcohol, 58, 7–29.

Rapp, R., Xu, J., Carr, C., Lane, D., Wang, J., & Carlson, R. (2006). Treatment barriers identified by substance abusers assessed at a centralized intake unit. Journal of Substance Abuse Treatment, 30(3), 227-235. doi:10.1016/j.jsat.2006.01.002

Robbins, P. C., Callahan, L., & Monahan, J. (2009). Perceived coercion to treatment and housing satisfaction in housing-first and supportive housing programs. Psychiatric Services, 60, 1251–1253.

Rollnick, S., Heather, N., Gold, R., & Hall, W. (1992). Development of a short “readiness to change” questionnaire for use in brief, opportunistic interventions among excessive drinkers. British Journal of Addiction, 87, 743–754.

Rosenblum, A., Magura, S., Kayman, D. J., & Fong, C. (2005). Motivationally enhanced group counseling for substance users in a soup kitchen: A randomized clinical trial. Drug and Alcohol Dependence, 80, 91–103.

Shaheen, G., & Rio, J. (2007). Recognizing work as a priority in preventing or ending homelessness. Journal of Primary Prevention, 28, 341-358.

Shelton, K., Taylor, P., Bonner, A., & van den Bree, M. (2009). Risk factors for homelessness: Evidence from a population-based study. Psychiatric Services, 60, 465–472. doi:10.1176/appi.ps.60.4.465

Srebnik, D., Livingston, J., Gordon, L., & King, D. (1995). Housing choice and community success for individuals with serious and persistent mental illness. Community Mental Health Journal, 31(2), 139-152.

Stephens, R. S., Roffman, R. A., & Curtin, L. (2000). Comparison of extended versus brief treatments for marijuana use. Journal of Consulting and Clinical Psychology, 68, 898–908.

Trudeau, J., Barrick, K., & Roehl, J. (2010). Independent Evaluation of the National Weed and Seed Strategy. (Draft Report). Washington, DC: U.S. Department of Justice, Community Capacity Development Office.

Tsemberis, S. J., Moran, L., Shinn, M., Asmussen, S. M., & Shern, D. L. (2003). Consumer preference programs for individuals who are homeless and have psychiatric disabilities: A drop-in center and a supported housing program. American Journal of Community Psychology, 32, 305-317.

Tucker, J., Vuchinich, R., & Rippens, P. (2004). A factor analytic study of influences on patterns of help-seeking among treated and untreated alcohol dependent persons. Journal of Substance Abuse Treatment, 26(3), 237-242. doi:10.1016/S0740-5472(03)00209-5

U.S. Department of Housing and Urban Development, Office of Community Planning and Development. (2009). The 2008 annual homeless assessment report to Congress. Washington, DC: Author.

U.S. Department of Veterans Affairs, Veterans Administration. (2009). Homeless veterans: Overview of homelessness. Washington, DC: Author. Retrieved on November 17, 2009, from http://www1.va. gov/homeless/page. cfm?pg=1

van den Bree, M. B. M., Shelton, K., Bonner, A., Moss, S., Thomas, H., & Taylor, P. J. (2009). A longitudinal population-based study of factors in adolescence predicting homelessness in young adulthood. Journal of Adolescent Health, 45, 571–578.

Vangeest, J. B., & Johnson, T. P. (2002). Substance abuse and homelessness: Direct or indirect effects? Annals of Epidemiology, 12, 455–461.

Weathers, F., Litz, B., Huska, J., & Keane, T. (1994). The PTSD Checklist–Civilian Version (PCL-C) for DSM-IV. Boston: National Center for PTSD. Weathers, F., Litz, B., Huska, J., & Keane, T. (1994). The PTSD Checklist–Civilian Version (PCL-C) for DSM-IV. Boston: National Center for PTSD.

Wright, D., Bowman, K., Butler, D., & Eyerman, J. (2005). Non-response bias from the national household survey on drug abuse incentive experiment. Journal of Economic and Social Measurement, 30, 219-231.

Attachments

1. SAMHSA Priority Program Goals

2. CSAT GBHI Client Interview

3. CSAT GBHI Client Interview - 6-Month Follow-up

4. CSAT GBHI Stakeholder Survey

5. CSAT GBHI Client Interview Script

6. CSAT GBHI Client Interview Consent Form

7. CSAT GBHI Stakeholder Survey Consent Form

8. Privacy Pledge

9. IRB Approval Client Survey

10. IRB Approval Stakeholder Survey and other non-client primary data

31