Form EBP Self-Assessmen EBP Self-Assessmen EBP Self-Assessment Part 1 and Part 2

Attachment 3: EBP Self-Assessment Part 1 & Part 2

OMB No. 0930-XXXX

Expiration Date XX/XX/XXXX

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number.  The OMB control number for this project is 0930-XXXX.  Public reporting burden for this collection of information is estimated to average XX hours per respondent, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.  Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

Substance Abuse and Mental Health Services Administration (SAMHSA)

National Evaluation of SAMHSA’s Homeless Programs

EBP Self-Assessment Part 1 – General Implementation Questions

Instructions

The cross-program evaluation team is interested in learning more about the primary evidence-based service practices (EBPs) being implemented by SSH/GBHI/CABHI program grantees. We know some grantee projects are implementing multiple EBPs. Primary EBPs are defined as those that are received by the largest number of consumers or clients served by the SSH/GBHI/CABHI project. During the grantee Project Director interview, information was collected on the primary EBPs being implemented in your site, as well as who is delivering and receiving these EBPs.

The cross-program evaluation team will be seeking to confirm the extent to which key components of certain EBPs¹ are being implemented, degree of implementation fidelity, and specific modifications that may have been made for use by local grantee programs. Information on practice-specific EBP implementation for these select EBPs will be collected from qualifying projects through a separate web-based self-assessment, and may also be explored and verified during key informant interviews and/or grantee site visits.

Here, we want to learn more generally about implementation of your site’s primary EBPs, and about factors that may serve as barriers or facilitators to implementation fidelity within grantee projects, such as readiness to implement the EBP, leadership, funding, training and supervision, quality improvement, and outcomes. Some of the questions are focused on the grantee agency and/or the overall grant project, and others are focused on the provider implementing the EBP, which may or may not be different from the grantee agency. Each SSH/GBHI/CABHI grantee project should have a key respondent which is typically the grantee Project Director or his/her appropriate designee (e.g., local site evaluator or other project staff familiar with EBP implementation at the site) or Program Manager/Supervisor at the provider agency implementing the primary EBP(s) complete the self-assessment. If needed, the key respondent may ask questions of staff familiar with the characteristics and implementation of your project’s EBP(s).

Primary EBP Information [PREPOULATED FROM PD INTERVIEW & VERIFIED]

Questions

Response Options

During the Project Director interview, the primary EBPs identified for this grantee program included:

EBP	Proposed for implementation in grant application?	Status of implementation	% program participants that receive	Who provides (grantee or other agency); SAMHSA grant funds used	Where provided	If grant has ended, still implementing?
1.
2.
3.

Respondent Information

Name/Title of Respondent #1:_____________________________ Primary Role in SAMHSA Grantee Project: (check all that apply) Project Director Project Coordinator Program Manager Local Evaluator Housing Provider Mental Health Counselor/Treatment Provider/Supervisor Substance Abuse Counselor/Treatment Provider/Supervisor Integrated Treatment (Mental Health & Substance Abuse) Counselor Trauma Specialist Case Manager Benefits Specialist Peer Specialist/Consumer Housing Specialist Vocational Specialist Educational Specialist Other: _______________________________________________

Respondent Agency/Organization:__________________________ Agency’s Primary Role in SAMHSA Grantee Project: (check all that apply) Grantee agency Administrative/Project Coordination/Oversight Research/Evaluation Substance abuse treatment provider Mental health treatment provider Integrated treatment (Mental Health & Substance Abuse) provider Shelter Housing provider Case management provider Medical (primary/specialized) care provider Benefits assistance provider Education provider Employment or job training provider Veterans Administration (VA) services provider Justice/criminal justice services provider Child and family services provider Other:______________________________________

Basic Program Information [PREPOPULATED FROM PD INTERVIEW & VERIFIED]

Questions	Response Options
The target populations for this grantee program who is receiving this EBP includes: (Check all that apply)	EBP :________________
	Mental Disorders Only Substance Abuse/Dependence Only Co-Occurring Mental and Substance Use Disorders Veterans Youth (under 18 years old) Young adults (e.g., ages 18-21) Older adults (e.g., 55 and over) Immigrants Criminal justice (e.g., previously incarcerated, reentry/diversion or on probation/adjudication) Families Persons at risk or living with HIV/AIDS Chronic public inebriates Domestic violence victims Lesbian, gay, bisexual, transgender, questioning individuals and allies (LGBT/LGBTQA) Pregnant Developmentally or physically disabled Other, specify:       None of the above specifically targeted If not correct, explain:      
The homeless populations that participate in this grantee program & therefore receive this EBP includes:	At Risk for Becoming Homeless Acute (first time) Homeless Episodically Homeless Chronically homeless Homeless, Not Specified If not correct, explain:      

EBP:___________________
Readiness to Implement EBP
Why was this EBP selected by the grantee project? (check all that apply)	Fit with population(s) served Fit with overall organization philosophy Already had the practice in place Outcomes align with program goals Required by SAMHSA grant Other, specify: ___________________
How long has the provider agency been implementing this EBP?	Haven’t started implementing yet Less than one year 1-2 years 3-4 years 5 or more years
Which of the following best describes the current stage of implementation of this EBP for program participants?	Preparation (e.g., hiring staff, conducting initial training, creating new operation polices & procedures, developing/finalizing strategic implementation plan) Early Implementation (e.g., referrals, screening & assessments occurring, services are underway) Full Implementation (e.g., staff skillful in service delivery, new policies & procedures are routine, practice is fully integrated into agency/program) Sustainability (e.g., sustainability plan developed & underway, continuous staff training & funding secured for future, outcomes used for program improvement) Other, specify:_______________
How is the priority the implementing agency places on this EBP demonstrated? (check all that apply)	There is an agency plan to implement the EBP Leadership frequently talks about the EBP Recruitment/selection of staff to implement the EBP Allocation of funding/other resources for the EBP Other, specify­­­­_______
Does the implementing agency have a formal plan to guide implementation of this EBP?	No Don’t know Yes If yes, which is true of the agency’s plan? (check all that apply) It is a written document It is discussed at staff meetings or meetings devoted to the plan All project staff are fully aware of the plan It has specific short- and long-term objectives regarding EBP implementation It identifies strategies for stakeholder outreach/consensus building for the EBP It identifies sources of funding for the EBP It identifies training resources for EBP implementation It identifies strategies for EBP implementation and outcomes evaluation Other, specify­­­­_______
Is leadership within the implementing agency supportive of this EBP’s implementation?	Extremely supportive Somewhat supportive Not at all supportive If supportive, at what leadership level(s) within the agency is this demonstrated? (check all that apply) Executive Management (e.g., agency executive director) Program Management Clinical/Front Line Supervisors Other, specify­­­­_______ If supportive, how is this demonstrated? (check all that apply) Leadership is actively involved in EBP implementation Barriers that impede implementation or effectiveness are addressed Support exists for coaching/ active supervision of staff directly implementing EBP Other, specify­­­­_______
Has a staff person at the implementing agency been assigned to lead implementation of the EBP?	No Yes If yes, what percent of his/her time is dedicated to the EBP’s implementation? 100% 76-99% 51-75% 25-50% less than 25% If yes, which of the following is true? (check all that apply) S/he has the necessary authority to lead implementation S/he has adequate training/expertise in the EBP S/he has a good relationship with staff directly implementing the EBP His/her leadership of EBP implementation is perceived positively by others
Would you say the implementing agency’s interest in this EBP is:	Limited to this SAMHSA-funded grant program/project only Extends beyond this program/project Other, specify­­­­_______
Are there any explicit policies the implementing agency has that support implementation of this EBP?	No Yes If yes, explain:      
Are there any explicit policies the implementing agency has that serve as barriers to implementation of this EBP?	No Yes If yes, explain:      
Are there any state or local (e.g., mental health or substance abuse authority) regulations or policies that support implementation of this EBP?	No Yes If yes, explain:      
Are there any state or local regulations or policies that serve as barriers to implementation of this EBP?	No Yes If yes, explain:      
Are there state or local standards that have to be followed in implementing the EBP? For example, some states have specific implementation guidelines related to staffing, fidelity checks, satisfaction surveys, etc.	No Yes If yes, describe       If yes, how are these standards established and enforced? Contracting Licensing Other, specify       If yes, which of the following consequences may occur for not meeting standards? Corrective action plan Financial consequences Other, specify      
Funding
How is this EBP funded? (check all that apply)	Medicaid (fee-for-service, Waiver, etc.) State agency funding, specify:_______________ SAMHSA grant funds, specify:__________________ Other special grant funds, specify:________________ Other, specify­­­­_______ Don’t know
How have start up or conversion costs associated with this EBP (e.g., lost productivity for training, hiring staff before clients enrolled, changing medical records and/or computer systems, etc.) been financed?	Costs were covered within the implementing agency’s own operating budget There was a discreet funding source that covered all costs (specify___________) There was a discreet funding source that covered some costs (specify_________) Don’t know
Which of the following best describes the financing for this EBP?	No components of service are reimbursable Some costs are reimbursable Most costs are reimbursable Service pays for itself (i.e. all costs covered adequately, or funding of covered components compensates for non-covered components) Service pays for itself and reimbursement rates are attractive relative to competing non-EBP services Don’t know
Is there a plan to continue the EBP once SAMHSA grant funding has ended? (Or if grant funding has already ended has the practice continued?)	Yes Don’t know No If no, why not? (check all that apply) Plan not developed yet but intend to continue the EBP Insufficient funding Lack of support from partnering agencies Too many barriers to implementation Insufficient numbers of eligible participants Model was not viewed as successful Other, specify:­­­­______
Hiring, Training & Supervision
Did the implementing agency receive expert advice/consultation regarding strategies to support implementation of this EBP?	No Don’t know Yes, initially only Yes, initially & ongoing If yes, who received this consultation? (check all that apply) Agency Administrators Program Directors/Supervisors Other, specify ___________ If yes, who supported/funded this consultation? (check all that apply) SAMHSA Other, specify ___________ If yes, who provided this consultation? Specify:__________________
Did staff selection/recruitment include attention to ensuring staff have the pre-requisite skills and/or credentials required by this EBP?	No Don’t know Yes
Was initial skills training provided to practitioners to support implementation of this EBP?	No Yes If yes, which of the following was true of this training? (check all that apply) Trainer was an expert who is experienced or certified in the EBP Training comprehensively addressed all elements of the EBP Active learning strategies were used (e.g., role play, group work, feedback) Teaching aides (e.g., worksheets, manuals, handouts) were used A SAMHSA Took Kit was utilized or referenced as part of the training
Is ongoing or refresher training available for practitioners to reinforce application of this EBP & help staff deal with emerging practice issues?	No Yes If yes, how often is this made available? (check all that apply) Monthly or more frequently Quarterly Annually Only as needed/requested
Which of the following training methods are used? (check all that apply)	Computer assisted training In-person training workshops Staff provided with training materials to “self-teach” Staff observe/shadow experienced staff person(s) Other, specify _________
Does all staff implementing this EBP receive the same training?	Yes No If no, explain:      
Do all practitioners delivering this EBP receive ongoing supervision and oversight?	No Yes If yes, which of the following is true? (check all that apply) Practitioners receive structured face-to-face supervision on a weekly basis Practitioners receive supervision but less than weekly (specify:_______) Supervision is provided by a practitioner experienced in this EBP Supervision includes observation of EBP implementation, coaching & feedback Supervision is provided but is not specific to the practice Other, specify ___________
Is there support/buy-in for implementation of this EBP among practitioners?	No Yes If yes, which of the following is true? (check all that apply) Practitioners voice support for the EBP Practitioners can describe how they’ve used the EBP Practitioners can describe how the approach benefits/helps clients Other, specify_______
Fidelity/Outcomes Monitoring & Performance Improvement
Are all clients screened to determine whether they qualify for receiving this EBP using standardized tools or admission criteria?	Yes No If no, why not? (check all that apply) All clients receive the intervention No standardized tool or admission criteria available Other, specify_________
To date, how many clients participated in this EBP during the grant period?	_______
How many clients were eligible to participate during the grant period?	_______
How is fidelity to this EBP monitored? (check all that apply)	Regular use of a standardized fidelity tool/checklist, specify:­­­­________ Direct observation Document review Focus groups or interviews with program participants Key informant interviews Tape/video recorded sessions/groups Other, specify:_________ We do not monitor fidelity to this EBP (Skip 32 – 37)
How often is fidelity data collected/assessed for this EBP?	Ongoing Every six months Annually Other, specify:____________
Who conducts fidelity assessments for this EBP? (check all that apply)	Staff internal to provider agency Staff external to provider agency Grant program evaluator Consultant Other, specify:____________
When was the last fidelity assessment done and what were the results?	Date conducted:       Measure Used:       Score/results:      
To what degree have the core components of this EBP been implemented to fidelity so far?	Low – Less than 50% of components implemented to fidelity Moderate 50-80% of components implemented to fidelity High – 81-100% of components implemented to fidelity
If this EBP has been implemented with moderate to low fidelity so far, why?	NA – Implemented with high fidelity All components planned but not yet fully implemented Some components were purposefully modified If modified, describe how and why (e.g., why certain components were not implemented or revised or new components added)     
Which of the following is true regarding the use of fidelity performance data? (check all that apply)	Data is shared with program staff Data is shared with internal advisory groups, board members, etc. Data is shared publicly via the web, agency annual reports, etc. Data is used for quality improvement Implementation adjustments have been made based on fidelity data
Are there any plans to maintain fidelity to this practice beyond the grant period?	No Don’t know Yes If yes, describe     
Are outcome data (e.g. changes in client functioning, access to treatment, housing/homeless status) related to this EBP collected?	No Yes If yes, how are these data used? (check all that apply) Don’t know Data are shared with practitioners to help them track/monitor client progress. Data are shared with agency leadership to help inform implementation of the EBP. Data are shared with stakeholders to solicit support (e.g. additional funding/ resources) for EBP implementation. Other, specify:      
Overall Barriers/Facilitators
Overall, what factors have served as barriers to implementation of this EBP during this project (i.e. have hindered successful implementation)? (check all that apply)	Lack of clear strategic plan for implementing the EBP Inadequate financing for the EBP Limited staff time/staff resources for EBP implementation Lack of on-going training, supervision, and consultation on the EBP Lack of positive practitioner attitudes toward the EBP Lack of prior experience with this EBP Lack of prior experience with other EBPs State or local policy/regulations Grantee or partner agency policies or practices Lack of support for implementation from key leaders at grantee or partner agency Lack of support for implementation from key external stakeholders Other, specify­­­­_______ Other, specify­­­­_______ None
Overall, what factors have served as facilitators to implementation of this EBP during this project (i.e. have helped with successful implementation)? (check all that apply)	Clear strategic plan for implementing the EBP Adequate financing for the EBP Adequate allocation of staff time/staff resources for EBP implementation Access to on-going training, supervision, and consultation on the EBP Positive practitioner attitudes toward the EBP Prior experience with this EBP Prior experience with other EBPs Supportive state or local policy/regulations Supportive grantee or partner agency policies or practices Support for implementation from key leaders at grantee or partner agency Support for implementation from key external stakeholders Other, specify­­­­_______ Other, specify­­­­_______ None

[**Repeat same questions for up to 2 more primary EBPs identified through the Project Director (PD) Interview]

Substance Abuse and Mental Health Services Administration (SAMHSA)

National Evaluation of SAMHSA’s Homeless Programs

EBP Self-Assessment Part 2 – Practice Specific Questions

Instructions

The cross-program evaluation team is interested in learning more about the primary evidence-based service practices (EBPs) being implemented by SSH/GBHI/CABHI program grantees. We know some grantee projects are implementing multiple EBPs. Primary EBPs are defined as those that are received by the largest number of consumers or clients served by the SSH/GBHI/CABHI project. During the grantee Project Director interview, information was collected on the primary EBPs being implemented in your site, as well as who is delivering and receiving these EBPs.

Through a separate web-based self-assessment, data is being collected from all grantees about general implementation of their site’s primary EBPs, and factors that may serve as barriers or facilitators to implementation fidelity within grantee projects, such as readiness to implement the EBP, leadership, funding, training and supervision, quality improvement, and outcomes.

Here, we are interested in confirming the extent to which key components of certain EBPs² are being implemented, degree of implementation fidelity, and specific modifications that may have been made for use by local grantee programs. This self-assessment should only be responded to by SSH/GBHI/CABHI grantees that identified one or more (up to 3) of the selected EBPs as their primary EBP(s) being implemented. Grantees meeting this criteria should have a key respondent which is typically the grantee Project Director or his/her appropriate designee (e.g., local site evaluator or other project staff familiar with EBP implementation at the site) or Program Manager/Supervisor at the provider agency implementing the primary EBP(s) complete the self-assessment. If needed, the key respondent may ask questions of staff familiar with the characteristics and implementation of your project’s EBP(s).

Practice-specific EBP implementation may also be explored and verified during key informant interviews and/or grantee site visits.

Basic Grantee/Program Information [PREPOPULATED FROM PD INTERVIEW & VERIFIED]

Questions

Response Options

During the Project Director interview, the primary EBPs identified for this grantee program included:

EBP	Proposed for implementation in grant application?	Status of implementation	% program participants that receive	Who provides (grantee or other agency); SAMHSA grant funds used	Where provided	If grant has ended, still implementing?
1.
2.
3.

Respondent Information

Name/Title of Respondent #1:_____________________________ Primary Role in SAMHSA Grantee Project: (check all that apply) Project Director Project Coordinator Program Manager Local Evaluator Housing Provider Mental Health Counselor/Treatment Provider/Supervisor Substance Abuse Counselor/Treatment Provider/Supervisor Integrated Treatment (Mental Health & Substance Abuse) Counselor Trauma Specialist Case Manager Benefits Specialist Peer Specialist/Consumer Housing Specialist Vocational Specialist Educational Specialist Other: _______________________________________________

Respondent Agency/Organization:__________________________ Agency’s Primary Role in SAMHSA Grantee Project: (check all that apply) Grantee agency Administrative/Project Coordination/Oversight Research/Evaluation Substance abuse treatment provider Mental health treatment provider Integrated treatment (Mental Health & Substance Abuse) provider Shelter Housing provider Case management provider Medical (primary/specialized) care provider Benefits assistance provider Education provider Employment or job training provider Veterans Administration (VA) services provider Justice/criminal justice services provider Child and family services provider Other:______________________________________

Assertive Community Treatment (ACT)/Intensive Case Management (ICM) Module

Dimension	Measure	Question	Response
(Not visible to respondents)
Human Resources: Small caseload	ACT consumer/ provider ratio = 10:1	What is the average case load size per ACT team member/ICM staff?	50 consumers or more 35 to 49 consumers 21 to 34 consumers 11 to 20 consumers 10 or fewer consumers
Human Resources: Team approach	Provider group functions as a team; team members know and work with all consumers.	Do ACT/ICM clients see the same staff person over and over (i.e. staff carry individual caseloads) or do they see different people (i.e. team shares caseload and members work with all clients)?	Staff members carry individual caseloads Staff members share caseload and members work with all clients
		In a typical 2-week period, what percentage of consumers has face-to-face contact with more than one member of the team?	90% - 100% 64 - 89% 37 - 63% 11 - 36% 0 - 10%
Human Resources: Program meeting	Program meets frequently to plan and review services for each consumer.	How often do the ACT team/ICM staff members meet as a full group to review services provided to consumers?	At least 4 days/week At least 2 days/week but less than 4 times/week 1 day per week At least twice per month but less than 1day/ week Once per month or less Staff do not meet as a full group to discuss consumers
		How many consumers are reviewed at each meeting?	Each consumer reviewed at each meeting, even if briefly Each consumer is not discussed each time staff meet Staff do not meet as a full group to discuss consumers
Human Resources: Practicing ACT lead	Supervisor of front-line ACT team members provides direct service.	Does the ACT team leader/ICM supervisor provide direct services to consumers?	Yes No
		What percentage of the ACT team leader/ICM supervisor’s time is devoted to direct services?	Over 50% of the time 25- 50% of the time Less than 25% of the time or routinely as back-up No regular percentage; only on rare occasions as back-up Team leader/Supervisor does not provide direct services
Human Resources: Continuity of staffing	Program maintains the same staffing over time.	What is the total number of staff positions on the ACT team/in the ICM program?	___________
		How many staff people have left the team/program?	If team/program has been existence for at least 2 years: _____(#) staff who have left over the last 2 years If team/program has been existence for less than 2 years: _____(#) staff who have left over the last _____ (# months) since the team/program began
Human Resources: Staff capacity	Program operates at full staffing.	Which of the following best represents ACT team/ICM program staffing capacity over the past 12 months?	Operated at 95% or more of full staffing Operated at 80-94% of full staffing Operated at 65-79% of full staffing Operated at 50-64% of full staffing Operated at less than 50% of full staffing
Human Resources: Psychiatrist on staff	For 100 consumers, at least 1 full-time psychiatrist is assigned to work with the program.	How many consumers are served by the ACT/ICM program?	_____# consumers served by ACT team/ICM program
		How many full-time equivalent (FTE) psychiatrists are assigned to work with the ACT/ICM program?	_____ FTE A psychiatrist is not assigned to work with the program
Human Resources: Nurse on staff	At least 2 full-time nurses are assigned to work with a 100 consumer program.	How many full-time equivalent (FTE) nurses are assigned to work with the ACT/ICM program?	_____ FTE A nurse is not assigned to work with the program
Human Resources: Substance abuse specialist on staff	At least 2 staff members with at least 1 year of training or clinical experience in substance abuse treatment per 100 consumer program.	How many full-time equivalent (FTE) substance abuse specialists are assigned to work with the ACT/ICM program?	_____ FTE A substance abuse specialist is not assigned to work with the program
		What types of training or clinical experience are assigned substance abuse specialists required to have? (check all that apply)	At least one year of substance abuse training Less than one year of substance abuse training At least one year of supervised substance abuse treatment experience Less than one year of supervised substance abuse treatment experience A substance abuse specialist is not assigned to work with the program
Human Resources: Vocational specialist on staff	At least 2 team members with 1 year training/ experience in vocational rehabilitation and support.	How many full-time equivalent (FTE) vocational specialists are assigned to work with the ACT/ICM program?	_____ FTE A vocational specialist is not assigned to work with the program
		Are assigned vocational specialists required to have at least one year of training/experience in vocational rehabilitation and support?	Yes No A vocational specialist is not assigned to work with the program
Human Resources: Program size	Program is of sufficient size to consistently provide necessary staffing diversity and coverage.	How many full-time equivalent (FTE) staff does the program have?	At least 10 FTE staff 7.5- 9.9 FTE staff 5.0- 7.4 FTE staff 2.5- 4.9 FTE staff Less than 2.5 FTE staff
Organizational Boundaries: Explicit admission criteria	Clearly identified mission to serve a particular population; has and uses measureable, operationally defined criteria to screen out inappropriate referrals.	Are there formal admission criteria the ACT/ICM program uses to screen potential consumers?	No Yes If yes, which of the following criteria are used (check all that apply)? Diagnosis of serious mental illness Diagnosis of co-occurring substance use disorder Pattern of frequent hospital admissions Frequent use of emergency services Consumers discharged from long-term hospitalization Homelessness Involvement with the criminal justice system Not adhering to medications as prescribed Not benefitting from usual mental health services (e.g. day treatment) Other, specify:________________
		Do all consumers served by the program meet the admission criteria you indicated in your response to Question 19?	Yes, all cases comply with this admission criteria Sometimes we accept clients who do not meet these criteria We accept most referrals There are no formal admission criteria for the program
Organizational Boundaries: Intake rate	Takes consumers in at a low rate to maintain stable service environment.	On average, how many new consumers has the ACT/ICM program taken on per month during the last six months?	6 or fewer consumers per month 7-9 consumers per month 10-12 consumers per month 13-15 consumers per month 16 or more consumers per month
Organizational Boundaries: Full responsibility for treatment services	In addition to case management, directly provides psychiatric services, counseling/ psychotherapy, housing support, substance abuse treatment, employment and rehabilitative services.	Which of the following services are delivered to ACT/ICM program consumers directly by program staff, and which are delivered by another department or agency? (check all that apply)	Directly by program staff: Case management Medication prescription, administration, monitoring, and documentation Counseling/individual supportive therapy Housing support Substance abuse treatment Employment or other rehabilitative services (e.g., ADLs)	By other department/agency: Case management Medication prescription, administration, monitoring, and documentation Counseling/individual supportive therapy Housing support Substance abuse treatment Employment or other rehabilitative services (e.g., ADLs)
Organizational Boundaries: Responsibility for crisis services	Has 24-hour responsibility for covering psychiatric crises.	What is the ACT team/ICM program staff role in providing 24 hour emergency services?	Provides 24 hour crisis coverage directly (i.e. a staff member is on-call at all times) Provides back-up support to emergency/on-call service (e.g., crisis program is called first, makes decision about need for direct ACT/ICM program involvement) Is available by phone, mostly in consulting role Emergency service has program-generated protocol to follow for program consumers Has no responsibility for handling crises after hours
Organizational Boundaries: Responsibility for hospital admissions	Is closely involved in hospital admissions	How often are program staff involved in the decision to admit consumers for psychiatric hospitalization?	Program staff are involved in 95% or more of admissions Program staff are involved in 65-94% of admissions Program staff are involved in 35-64% of admissions Program staff are involved in 5-34% of admissions Program staff are involved in less than 5% of admissions
Organizational Boundaries: Responsibility for hospital discharge planning	Is involved in planning for hospital discharges	How often is program staff involved with discharge planning when consumers are hospitalized for psychiatric or substance abuse reasons?	95% or more of discharges planned jointly with program staff 65-94% of discharges planned jointly with program staff 35-64% of discharges planned jointly with program staff 5-34% of discharges planned jointly with program staff Less than5% of discharges planned jointly with program staff
Organizational Boundaries: Time-unlimited services	Rarely closes cases; remains the point of contact for all consumers indefinitely as needed.	Which of the following happens when a ACT/ICM consumer’s need for services is reduced?	They continue to be served on a time-unlimited basis They are discharged because they have graduated from services They are stepped down to less intensive services (specify:______) Other, specify:_______________________
		What percentage of consumers is expected to graduate from the program within the next 12 months?	Less than 5% 5-17 % 18-37% 38-90% More than 90%
Nature of Services: Community-based services	Program works to monitor status, develop community living skills in community rather than in office.	What percentage of face-to-face contacts with program consumers occur in the community (vs. in an office setting)?	80% or more 60-79% 40-59% 20-39% Less than 20%
Nature of Services: No dropout policy	Program retains high percentage of consumers.	How many consumers dropped out of the program over the last 12 months for the following reasons? Do not include consumers who graduated because their services needs were reduced.	_____# who refused services _____# who cannot be located _____# who have been closed because staff determined they could not serve them _____#who dropped out for other reasons (specify:_________)
Nature of Services: Assertive engagement mechanisms	Program uses street outreach, legal mechanisms, or other techniques to ensure ongoing engagement.	What happens if a consumer continually refuses or does not comply with (e.g., misses appointments for) program services? (check all that apply)	They are immediately discharged from the program Staff initially attempts to engage but may eventually discharge Staff attempt to engage using assertive techniques as much as possible Staff consistently use assertive techniques to keep consumers involved in services Other, specify:__________ None of the above
		What methods do program staff use to keep consumers involved in services? (check all that apply)	Outpatient commitment Representative payee services Contacts with probation/parole Street/Shelter outreach after enrollment Other, specify:__________ None of the above
Nature of Services: Intensity of service	High amount of face-to-face service time as needed.	On average, how much face-to-face time do program staff have with consumers per week?	2 hours/week or more 85-119 minutes/week 50-84 minutes/week 15-49 minutes/week Less than 15 minutes/week
Nature of Services: Frequency of contact	High amount of face-to-face service contacts as needed.	On average, how many face-to-face contacts do program staff have with consumers per week?	5 or more contacts/week 3-4 contacts/week 1-2 contacts/week No contacts/week
Nature of Services: Work with informal support system	Program provides support and skills for consumers’ informal support network.	On average, how often do program staff work with the family, landlord, employer, or other informal support network members for each consumer with a support system in the community?	5 or more contacts/month 3-4 contacts/month 1-2 contacts/month No contacts/month
Nature of Services: Individualized substance abuse treatment	One or more team members provide direct substance abuse treatment for consumers with substance use disorders.	Do program consumers with substance use disorders receive formal individual counseling for substance use from a team/program staff member?	Yes, on weekly basis or more Yes, but not regularly No
Nature of Services: Co-occurring disorder treatment groups	Program uses group modalities as a treatment strategy for consumers with dual disorders.	What percentage of consumers with substance use disorders attend at least one substance abuse treatment group per month that is run by program staff?	50% or more 35-49% 20-34% 5-19% less than 5%
Nature of Services: Co-occurring disorders model	Program uses no-confrontational, stage wise treatment model, follows behavioral principles, consider interactions of mental illness and substance use, has gradual expectations for abstinence	Which of the following principles and approaches does the program use to treat consumers with substance use issues? (check all that apply)	Confrontation Abstinence only Reduction of use (i.e. harm reduction) Stage wise approach Referrals to rehab Referrals to detox - only when medically necessary Referrals to detox for other purposes Referrals to AA, NA, etc. Other, specify:_________
Nature of Services: Role of consumers on team	Consumers are members of the team who provides direct services.	How are consumers involved as team/program staff members? (check all that apply)	As full-time paid employees As part-time paid employees As volunteers As full professional team members/staff As case managers with reduced responsibilities As aides to the team/program staff In consumer-specific roles (e.g., self-help) Not at all
		Were any components of the ACT program model difficult to implement?	No Yes If yes, which ones? (check all that apply) Small caseload size (10:1) Team approach Frequent program meetings to review each consumer Practicing program lead Continuity of staffing Operating at full staff capacity 1 FTE psychiatrist on staff per 100 consumers 2 FTE nurses on staff per 100 consumers 2 substance use specialists on staff per 100 consumers 2 vocational specialists on staff per 100 consumers Program size (appropriate # of FTE staff) Explicit admission criteria Low intake rate Full responsibility of treatment services 24 hour responsibility for crisis services Responsibility for hospital admission Responsibility for hospital discharge planning Time-unlimited services Services delivered in community (vs. office based settings) No dropout policy Assertive engagement mechanisms used High intensity of services High frequency of contacts Work with informal support system Direct provision of individualized substance abuse treatment Co-Occurring disorder treatment groups provided Co-occurring disorder model used Consumers provide direct services
		Did you make any adjustments or modifications to the program model?	No Yes If yes, please describe     
		Were any of the following types of evidence-based service interventions fully imbedded within your implementation of the ACT/ICM program model?	Motivational Interviewing Cognitive Behavioral Therapy (CBT) Motivational Enhancement Therapy (MET) Peer Support Strengths-Based Case Management/Approach SSI/DI Outreach, Access & Recovery (SOAR) Trauma-Specific Intervention (specify:___________) Other (specify:___________________)

Integrated Dual Disorders Treatment (IDDT) Module

Dimension	Measure	Question	Response
(Not visible to respondents)
Multidisciplinary team (MDT)	Case managers, psychiatrist, nurses, residential staff, employment specialists, and rehab specialists work collaboratively on mental health treatment team.	Do staff work with consumers individually or as part of a multidisciplinary team (MDT)?	Individually (Skip to Q #4) As a MDT Other (explain:___________________)
		What staff members comprise the MDT? (check all that apply)	Psychiatrist Nurse Case manager Employment specialist(s) Integrated treatment specialist Clinicians (e.g. psychologist, licensed social worker, etc.) Practitioners of other ancillary rehabilitation services Other (specify:_____________________)
		Are all members of the MDT required to attend treatment team meetings?	Yes No
Integrated treatment specialists	Integrated treatment specialists work collaboratively with the MDT, modeling integrated treatment skills and training other staff in evidence-based practice principles and practice.	Does your agency assign integrated treatment specialists to the program or are consumers referred to integrated treatment specialists (e.g., through a separate program within the agency)?	Integrated treatment specialists are assigned to program Consumers are referred to integrated treatment specialists No integrated treatment specialists connected with the agency
		How often do integrated treatment specialists attend MDT meetings?	Always Frequently Sometimes Rarely Never NA
		How involved are integrated treatment specialists in treatment planning with other members of the treatment team?	Very involved Somewhat involved Not at all involved NA
Stage-wise interventions	All services are consistent with and determined by each consumer’s stage of treatment. The stages of treatment include the following: Engagement Persuasion Active treatment Relapse prevention	Which of the following philosophies or goals are used by staff when treating individuals with co-occurring disorders?	Confrontation Abstinence Stages of change Reduction of use Relapse prevention Other (specify:____________)
		How often would you say that interventions are consistent with the individual’s stage of treatment?	80-100% of the time 61-79% of the time 41-60% of the time 21-40% of the time 0-20% of the time
		Are program staff offered training on stages of change and the stages of treatment?	Yes No
Access to comprehensive services	Individuals in the program have access to comprehensive services including: Residential services SE Family interventions IMR ACT	Which of the following services do program consumers have genuine access to at the agency? (check all that apply)	Residential Services Supported Employment (SE) Family Intervention Illness Management and Recovery (IMR) Assertive Community Treatment (ACT) Other (specify:_______________)
Time-unlimited services	Individuals in the program are treated on a time-unlimited basis with intensity modified according to each person’s needs.	Does the program graduate consumers after they have completed a certain number of sessions or groups?	Yes No
		Which of the following happens when a consumer’s need for services is reduced?	They are closed out of services after a defined period of time (Skip to Q#13) They continue to be served indefinitely and the intensity of services is modified based on individual consumer need. If yes, how often is this true? 80-100% of the time 61-79% of the time 41-60% of the time 21-40% of the time Less than 20% of the time
Outreach	Integrated treatment specialists demonstrate consistently well-thought out outreach strategies and connect consumers to community services, whenever appropriate, to keep consumers engaged in the program.	What happens if a consumer continually refuses or does not comply with (e.g., misses appointments for) program services? (check all that apply)	They are immediately discharged from the program Staff initially attempts to engage but may eventually discharge Staff attempt to engage using assertive outreach techniques as much as possible Staff consistently use assertive techniques to keep consumers involved in services Other, specify:__________ None of the above
		What types of assistance do integrated treatment specialists offer to connect consumers with as a means of engagement? (check all that apply)	Housing assistance Legal aid Meals or other food resources Clothing Medical care Crisis management Other (specify:____________)
Motivational interventions	All interactions with consumers in the program are based on motivational interventions: Expressing empathy Developing discrepancy Avoiding argumentation Rolling with resistance Instilling self-efficacy and hope	Are integrated treatment specialists offered training in motivational interventions?	Yes No
		Which of the following techniques are used by integrated treatment specialists with program consumers? (check all that apply)	Expressing empathy Developing discrepancy Avoiding argumentation Rolling with resistance Instilling self-efficacy and hope Other (specify:__________)
		How often do staff use a motivational approach in their interactions with consumers?	80-100% of the time 61-79% of the time 41-60% of the time 21-40% of the time 0-20% of the time
Substance abuse counseling	Individuals who are in the active treatment or relapse prevention stages receive substance abuse counseling that includes: How to manage cues to use and consequences of use Relapse prevention strategies Drug and alcohol refusal skills training Problem-solving skills training to avoid high-risk situations Coping skills and social skills training Challenging consumers’ beliefs about substance abuse	During which phase(s) of treatment are program consumers offered some form of substance abuse counseling? (check all that apply)	Engagement: while forming a trusting working alliance/relationship Persuasion: while helping engaged consumers become motivated to participate in recovery Active Treatment: while helping motivated consumers acquire skills/supports for managing illness and pursuing goals Relapse Prevention: while helping consumers in stable remission develop/use strategies to maintain recovery
		Which of the following knowledge/skills are taught to consumers who receive substance abuse counseling in the program? (check all that apply)	How to manage cues to use and consequences of use Relapse prevention strategies Drug and alcohol refusal skills Problem-solving skills training to avoid high-risk situations Coping skills and social skills training to deal with symptoms or negative mood states Relaxation Other (Specify:______________)
Group treatment for co-occurring disorders	All consumers in the program are offered group treatment specifically designed to address both mental health and substance use problems.	Which of the following best describes the types of group treatment offered by the program?	No group treatment is offered (Skip to Q#21) Substance use or mental health specific groups are offered only (Skip to Q#21) Groups that address both mental health and substance use are offered
		What proportion of program consumers regularly attend group treatment focused on both mental health and substance use?	65-100% 50-64% 35-49% 20-34% Less than 20%
Family interventions for co-occurring disorders	With individuals’ permission program involves consumers’ family members (or other supports) provide education about co-occurring disorders, offer coping skills training and support to reduce stress in the family, and promote collaboration with the treatment team.	Are family interventions offered to consumers in the program?	No (Skip to Q#25) Yes
		Are all consumers asked permission to involve family members or other supporters in family interventions?	No Yes
		What proportion of consumers’ family members or other supporters receive family interventions for co-occurring disorders?	65-100% 50-64% 35-49% 20-34% Less than 20%
Alcohol and drug self-help groups	Individuals in the active treatment or relapse prevention stages attend self-help programs in the community.	Does the program ever refer consumers to self-help groups in the community (e.g., AA, NA, etc)?	No (Skip to Q# 28) Yes
		During which phase(s) of treatment do referrals to self-help groups occur? (check all that apply)	Engagement: forming a trusting working alliance/relationship Persuasion: helping engaged consumers become motivated to participate in recovery Active Treatment: helping motivated consumers acquire skills/supports for managing illness and pursuing goals Relapse Prevention: helping consumers in stable remission develop/use strategies to maintain recovery
		How many consumers in your program regularly attend self-help programs in the community?	65-100% 50-64% 35-49% 20-34% Less than 20%
Pharmacological treatment	Prescribers for consumers in the program are trained in the evidence-based model & use the following: Prescribe despite active substance use Work closely with consumers and treatment team Focus on increasing adherence to psych meds Avoid prescribing meds that may be addictive Prescribe meds that help reduce addictive behavior	Are prescribers (e.g., physicians or nurses) who work with consumers in the program trained in the evidence-based model?	No Yes
		Are psychotropic medications prescribed to consumers with active substance use problems?	No Yes
		How often is the treatment team in contact with program consumers’ prescribers?	Always Frequently Sometimes Rarely Never
		What types of strategies do prescribers typically use for consumers who do not take psychiatric medications as prescribed?	Encourage consumers’ right to refuse medications Encourage consumers’ adherence to medications Other (specify:___________)
		Are consumers in the program prescribed medications that may be addictive?	Always Frequently Sometimes Rarely Never
		Are consumers in the program prescribed medications known to be effective in reducing addictive behavior?	Always Frequently Sometimes Rarely Never
Interventions to promote health	Integrated treatment specialists promote health by encouraging consumers with co-occurring disorders to do the following: Avoid high-risk behavior and situations that can lead to infectious diseases Find safe housing Practice proper diet and exercise	Do integrated treatment specialists offer consumers interventions to promote health?	No Yes
		Which of the following areas do integrated treatment specialists typically address with program consumers? (check all that apply)	Switching to less harmful substances Finding safe housing Proper diet and exercise Safe sex practices The risk of losing friends and family Other (specify:_______________)
		How many program consumers receive interventions to help them reduce the negative consequences of substance abuse?	80-100% 50-79% Less than 50%
Secondary interventions for non-responders	Program has a protocol to identify consumers who do not respond to basic treatment for co-occurring disorders, to evaluate them, and to link them to appropriate secondary interventions.	Does your program have a protocol to identify consumers who do not respond to basic treatment?	No Yes
		How often are individuals assessed to determine if they are progressing toward recovery?	There is no evaluation or assessment process Annually At a minimum of every 6 months At a minimum of every 3 months
		What percentage of consumers who do not respond to basic treatment are referred for secondary interventions?	80-100% 61-79% 41-60% 21-40% Less than 20%
		Were any components of this program model difficult to implement?	No Yes If yes, which ones? (check all that apply) Staff work as a multidisciplinary team (MDT) Integrated Treatment Specialists work collaboratively w/MDT Services are consistent with consumers’ stage of treatment Consumers have access to comprehensive services Time-unlimited services Outreach strategies used to keep consumers engaged Motivational interventions used Substance abuse counseling at appropriate stage Group treatment for co-occurring disorders offered Family interventions for co-occurring disorders offered Alcohol & drug self-help groups offered at appropriate stage Pharmacological treatment consistent with EBP Interventions to promote health used Secondary interventions for non-responders used
		Did you make any adjustments or modifications to the Integrated Treatment model?	No Yes If yes, please describe.     
		Were any of the following types of evidence-based service interventions fully imbedded within your implementation of the Integrated Treatment program model?	Motivational Interviewing Cognitive Behavioral Therapy (CBT) Motivational Enhancement Therapy (MET) Peer Support Strengths-Based Case Management/Approach SSI/DI Outreach, Access & Recovery (SOAR) Trauma-Specific Intervention (specify:___________) Other (specify:___________________)

Illness Management and Recovery (IMR) Module

Dimension	Measure	Question	Response
(Not visible to respondents)
Staffing: Number of people in a session/group	IMR is taught individually or in groups of eight or fewer consumers	Are IMR sessions taught individually, in a group format, or both?	Individually In Groups Both individually and in groups
		How many people typically participate in an IMR session or group?	15 or more consumers 13-15 consumers 11-12 consumers 9-10 consumers 8 or fewer consumers IMR is only taught individually
Program length	Consumers receive at least 3 months of weekly IMR sessions or an equivalent number of IMR sessions	How often and for what length of time do consumers typically attend IMR sessions? Note: Exclude from consideration consumers who drop out prematurely.	______total # of sessions attended ______total length of time attended (in months) Are sessions held: Weekly Bi-weekly Once per month Other (specify:___________)
Comprehensiveness of the curriculum	Curriculum is comprehensive & includes: Recovery strategies Practical facts about MI Stress-Vulnerability Model & tx strategies Building social support Using medication effectively Drug & alcohol use Reducing relapses Coping with stress Coping with problems and persistent symptoms Getting your needs met in the mental health system.	Is there an established curriculum for the IMR sessions?	No Yes
		Which of the following topics are covered in IMR sessions? (check all that apply)	Recovery strategies Practical facts about mental illnesses Stress-Vulnerability Model and treatment strategies Building social support Using medication effectively Drug and alcohol use Reducing relapses Coping with stress Coping with problems and persistent symptoms Getting needs met in the mental health system Other (specify:________________)
Provision of educational handouts	All consumers participating in IMR receive IMR handouts	Do IMR consumers receive educational handouts as part of the program?	No Yes If yes, is this true: 90-100% of the time 70-89% of the time 40-69% of the time 20-39% of the time Less than 20% of the time
Involvement of significant others	Developing and enhancing natural support is one of IMR’s goals. Social support helps people generalize information and skills learned in sessions to their natural environment.	Does the IMR program involve consumers’ significant others (e.g. family, friends, other non-paid supports)?	No (Skip to Q#9) Yes
		How are significant others involved: (check all that apply)	IMR practitioners have regular contact with significant others Significant others assist consumers in pursuing IMR goals Other (specify:________________) Is this type of involvement true for: At least 50% of IMR consumers 30-49% of IMR consumers Less than 30% of consumers
Assignments: IMR goal setting	One of the objectives of the IMR program is to help consumers establish personally meaningful goals.	To what extent do IMR consumers have personally established goals that are realistic and measurable?	90-100% of consumers have at least one such goal 70-89% of consumers have at least one such goal 40-69% of consumers have at least one such goal 20-39% of consumers have at least one such goal Less than 20% of consumers have at least one such goal
Assignments: IMR goal follow-up	Practitioners and consumers collaboratively follow up on goals identified above.	How often is progress toward achieving consumers’ IMR goals reviewed?	At every session Some other frequency (e.g. every other session, monthly, etc.) Infrequently/only as needed Progress is not reviewed Is the above true for: All IMR consumers Most IMR consumers Some IMR consumers
Assignments: Motivation-based strategies	Practitioners regularly use motivation-based strategies.	Which of the following strategies are used in IMR sessions? (check all that apply)	Teaching new information and skills to achieve goals Encouraging positive perspectives of past experiences Exploring the pros and cons of change Instilling hope and belief in self-efficacy Other (specify ________)
		How often are motivation based strategies used in IMR sessions?	They are used in at least half of the sessions They are used in some sessions They are used in a few sessions They are never used in sessions
Assignments: Educational techniques	Practitioners embrace the concept of and regularly apply educational techniques.	Which of the following educational techniques are used in IMR sessions? (check all that apply)	Interactive teaching Checking for understanding Breaking down information Reviewing information Other (specify ________)
		How often are educational techniques used in IMR sessions?	They are used in at least half of the sessions They are used in some sessions They are used in a few sessions They are never used in sessions
Assignments: Cognitive-behavioral techniques	Practitioners regularly use cognitive-behavioral techniques to teach IMR information and skills.	Which of the following techniques are used in IMR sessions? (check all that apply)	Reinforcement Shaping Modeling Role playing Cognitive restructuring Relaxation training Other (specify _______)
		How often are cognitive-behavioral techniques used in IMR sessions?	They are used in at least half of the sessions They are used in some sessions They are used in a few sessions They are never used in sessions
Assignments: Coping skills training	Practitioners embrace the concept of and systematically provide, coping skills training.	Are IMR practitioners familiar with the principles of coping skills training?	No Some are familiar The majority are familiar All practitioners are familiar
		How frequently do IMR practitioners use coping skills principles in their IMR sessions?	Regularly Moderately Not often Never
Assignments: Relapse prevention training	Practitioners embrace the concept of relapse prevention training and systematically apply it.	Are IMR practitioners familiar with the principles of relapse prevention training?	No Some are familiar The majority are familiar All practitioners are familiar
		How frequently do IMR practitioners use relapse prevention training in their IMR sessions?	Regularly Moderately Not often Never
Assignments: Behavioral tailoring for medication	Practitioners embrace the concept of and use behavioral tailoring for medication.	Are IMR practitioners familiar with the principles of behavioral tailoring for medication?	No Some are familiar The majority are familiar All practitioners are familiar
		How frequently do IMR practitioners use behavioral tailoring for medication techniques in their IMR sessions?	Regularly Moderately Not often Never
		Were any components of this program model difficult to implement?	No Yes If yes, which ones? (check all that apply) IMR taught individually or in groups of 8 or fewer consumers At least 3 months of weekly sessions or equivalent Comprehensiveness of curriculum Provision of educational handouts Involvement of significant others IMR goal setting IMR goal follow-up Motivation-based strategies used Educational techniques used Cognitive-behavioral techniques used Coping skills training provided Relapse prevention training provided Behavioral tailoring for medications used
		Did you make any adjustments or modifications to the IMR model?	No Yes If yes, please describe.     
		Were any of the following types of evidence-based service interventions fully imbedded within your implementation of the IMR model?	Motivational Interviewing Cognitive Behavioral Therapy (CBT) Motivational Enhancement Therapy (MET) Peer Support Strengths-Based Case Management/Approach SSI/DI Outreach, Access & Recovery (SOAR) Trauma-Specific Intervention (specify:___________) Other (specify:___________________)

Supported Employment (SE) Module

Dimension		Measure	Question	Response
(Not visible to respondents)
Staffing: Caseload size		Employment specialists (ES) manage caseloads of up to 25 consumers	What is the average caseload size for an employment specialist?	81 or more consumers 61 to 80 consumers 41 to 60 consumers 26 to 40 consumers 25 or fewer consumers
Staffing: Focus of vocational services staff time		ES provide only vocational services.	What services do employment specialists provide? (check all that apply)	Vocational services Case management Individual or group therapy Staffing for day or residential programming Other (specify:_____________) If only selected vocational services above, SKIP to Q#4
			How much of the time do employment specialists provide non-vocational services?	Less than 20% 20-39% 40-59% 60-79% 80% or more
Staffing: Vocational generalists role/responsibilities		Each ES carries out all phases of vocational service including engagement, assessment, job development, job placement, job coaching, and follow-along supports.	Which of the following most accurately describes the role of employment specialists (ES) in the program?	Each ES carries out all phases of vocational service, including engagement, assessment, job development, placement, and coaching, and follow-along supports. ES provides 2 or more phases of vocational service but not the entire service (e.g. some do engagement and assessment only while others do job development and placement, etc.) ES specializes in 1 aspect of vocational service ES maintain caseloads but refer consumers to other programs for vocational service ES do not carry caseloads and only provide vocational referrals to other vendors or programs Other (specify:_________________)
Organization: Integration of rehabilitation with mental health treatment		ES are part of the mental health treatment teams with shared decision making. They attend regular treatment team meetings and have frequent contact with treatment team members.	Do employment specialists interact with the mental health treatment team?	No Yes, but infrequently Yes, regularly If yes, how & how frequently is contact made: (check all that apply) Telephone contact ____ times per month Face-to-face contact ____ times per month Attendance at treatment team meetings ____ times per month
			Do employment specialists and case managers or case management teams participate in shared decision making about consumer services?	No Yes
Organization: Vocational unit functioning		ES function as a unit rather than a group of practitioners. They have group supervision, share information, and help each other with cases.	Do all employment specialists have the same supervisor?	No Yes If yes, how & how frequently do they receive supervision: Individually ____ times per month As a group ____ times per month
			Do employment specialists provide services for one another’s consumers?	No Yes
Organization: Zero-exclusion criteria		No eligibility requirements such as job readiness, lack of substance abuse, no history of violent behavior, minimal intellectual function, and mild symptoms	Must consumers meet certain eligibility criteria in order to receive supported employment services?	No Yes If yes, which of the following screening criteria are used (check all that apply): Job readiness Abstinence from substance use No history of violent behavior Other (specify:_________________)
			Where does the supported employment program accept referrals from?	Case Managers Therapists Psychiatrists Family members Self-referral Other (specify:_____________________)
Services: Ongoing, work-based vocational assessment		Vocational assessment is an ongoing process based on work experiences in competitive jobs.	Are vocational assessments that are conducted in the supported employment program primarily:	Office-based assessments done prior to job placement? Pre-vocational assessments conducted at a day program site? Carried out in a sheltered work environment? Based on a series of temporary job experiences? Ongoing assessments that occur in community jobs? Other (specify:___________)
Services: Rapid search for competitive jobs		The search for competitive jobs occurs rapidly after program entry.	Must consumers take any steps in the program before beginning a job search?	Yes, some pre-requisites exist (e.g. pre-vocational counseling, participation in an enclave or sheltered work, etc.) before search for a competitive job can begin. No, the job search begins as soon as a consumer expresses interest in competitive employment
			How soon after program entry does a consumer typically begin having contact with competitive employers (i.e. start their job search)?	Within 1 month 1-6 months 6-9 months 9-12 months More than 12 months
Services: Individualized job search		Employer contacts are based on consumers’ job preferences (relating to what they enjoy and their personal goals) and needs rather than the job market, that is, what jobs are readily available.	How are employer contacts selected? (Check all that apply)	Based on the local job market (i.e. which jobs are readily available) Based on the employment specialists decisions Based on the consumer’s preferences and needs Other (specify:___________)
			How often are employer contacts made based on consumer preferences and needs rather than the job market?	Most of the time About 75% of the time About 50% of the time About 25% of the time Never
Services: Diversity of jobs developed		ES provide job options that are in different settings.	What proportion of the types of job options and settings offered to consumers are:	The same/similar (e.g., all janitorial, or in food service settings)_____% Different (e.g., consist of all types of jobs/settings) ______%
			What percentage of consumers work in the same types of jobs or settings?	75-100% About 75% About 50% About 25% Less than 10%
Services: Permanence of jobs developed		ES provide competitive job options that have permanent status rather than temporary or time-limited status.	Do employment specialists ever suggest jobs to consumers that are temporary, time-limited, or volunteer?	Yes, always Yes, sometimes No, never
			How often do employment specialists provide options to consumers for permanent, competitive jobs?	75-100% of the time About 75% of the time About 50% of the time About 25% of the time Employment specialists do not provide options for permanent, competitive jobs
Services: Jobs as transitions		All jobs are viewed as positive experiences on the path of vocational growth and development. ES help consumers end jobs when appropriate and then find new jobs.	When a job has ended, do employment specialists offer to assist consumers in finding another job?	Not usually Yes always Depends on the situation If it depends, how often are they likely to assist? About 75% of the time About 50% of the time About 25% of the time Please provide an example of a reason an employment specialist might be less likely to assist a consumer in finding a new job? ____________________
Services: Follow along supports		Individualized, follow-along supports are provided to employer and consumer on a time-unlimited basis.	Are follow-along supports provided:	To consumers (e.g., job coaching/counseling, job support groups, etc.)? No not provided Yes provided to most Provided to less than half	To employers (e.g., education, guidance)? No not provided Yes provided to most Provided to less than half
			Is there a time limit for providing supports:	To consumers? No Yes If yes, what is the limit? ____	To employers? No Yes If yes, what is the limit? _____
Services: Community-based services		Vocational services such as engagement, job-finding, and follow-along supports are provided in community settings	What percentage the services employment specialists provide are in the community (vs. in an office or mental health facility)?	70-100% 60-69% 40-59% 11-39% 0-10%
Services: Assertive engagement and outreach		Assertive engagement and outreach are conducted as needed	Do employment specialists conduct outreach to engage consumers?	Yes, initially Avg. # of contacts: _____ OR frequency____ (e.g., once per week, month, etc.) Yes, if they stop attending vocational services Avg. # of contacts: ___ OR frequency____ (e.g., once per week, month, etc.) No (Skip to Q# 26)
			What types of outreach are typically used? (check all that apply)	Letters or other written materials sent to the consumer’s residence Phone calls to the consumer Phone calls to consumers’ case manager/other care provider (with consent) Community visits with consumers
			Where there components of the Supported Employment program model that were difficult to implement?	No Yes If yes, which ones? (check all that apply) Caseload size (1:25) ES provide only vocational services ES carry out all phases of vocational service Integrating ES with mental health treatment team ES share a supervisor and help each other with cases Zero-exclusion criteria Ongoing, work-based vocational assessments. Rapid search for competitive jobs Employer contacts based on consumer preferences/needs vs. job market Job options provided are in different settings. Providing permanent, competitive job options Helping consumers find new jobs Providing follow-along Providing vocational services in community settings Providing assertive engagement and outreach
			Did your agency make any adjustments or modifications to the Supported Employment model?	No Yes If yes, please describe.          
			Were any of the following types of evidence-based service interventions fully imbedded within your implementation of the Supported Employment model?	Motivational Interviewing Cognitive Behavioral Therapy (CBT) Motivational Enhancement Therapy (MET) Peer Support Strengths-Based Case Management/Approach SSI/DI Outreach, Access & Recovery (SOAR) Trauma-Specific Intervention (specify:___________) Other (specify:___________________)

Critical Time Intervention (CTI) Module

Component/Measure (not visible to respondents)	Question	Response
Program Structure/Staffing	Which settings are consumers who receive CTI services directly transitioning between?	Transitioning from: Hospital Shelter Housing setting (e.g., residential, transitional housing) specify:_______ Streets Prison Jail Other, specify_______	Transitioning to: Transitional housing Permanent housing Other, specify_______
	In what setting is the CTI program based?	Drop-in center Shelter Mental health impatient unit Other, specify_______
	What staff members comprise the CTI team?	Psychiatrist Nurse Team leader /coordinator (specify credentials, e.g., MSW___________) Housing case manager or specialist CTI case managers/workers (specify #_____) Other, specify_______
	What is the average case load size per CTI worker?	35 to 50 consumers 21 to 34 consumers 15 to 20 consumers 10 or fewer consumers Does caseload size vary by phase of service? If yes, explain:_____
	Does CTI staff meet as a team to discuss clients’ needs and care?	No Yes If yes, how often are team meetings held? Weekly Bi-weekly Monthly Only as needed Other, specify___________ If yes, who conducts the team meetings? ________ If yes, what percentage of CTI clients are reviewed at each team meeting: ____%
	How often are each CTI client’s needs and care reviewed and discussed by CTI program staff?	Weekly Bi-weekly Monthly Only as needed Other, specify___________
	What types of supervision and organizational support does CTI program staff receive?	Individual clinical supervision (specify frequency________) Field work observation/feedback Team case presentations/feedback Review/feedback of client case notes Resources to support work in the field (specify:_______) Other, specify___________
Early Engagement	Are CTI workers able to establish relationships and begin to engage consumers prior to their transition to a new setting in the community?	Yes No (SKIP to Q 11)
	What is the typical length of time between initial contact and a consumers’ discharge or move to the community (i.e. length of pre-CTI period)?	Less than 1 week 1-2 weeks 2-4 weeks More than 1 month Other, specify______
	How often do CTI workers typically meet with consumers during the ‘pre-CTI period’?	Once 2-3 times 4 times Other, specify_____
Assessment/Treatment Planning	Is a CTI intake assessment completed?	No (SKIP to Q 13) Yes If yes, when is it completed? _____________
	Which of the following are components of the intake assessment?	Demographic information Psychiatric history (diagnosis, symptoms, medications, hospitalizations) Substance use history (diagnosis, symptoms, treatment history) Homelessness/housing history Reasons for housing loss/risks to housing stability Financial supports Formal & informal supports ADL skills Strengths & interests of consumer Other, specify______
	Are CTI services delivered in phases?	No Yes If yes, how many phases? ____ If yes, how long does each phase last? _____
	Is a CTI treatment plan completed?	Yes, at the beginning of CTI services only Yes, for each phase of service Other, specify_____
	What is the typical timeframe for completion of the treatment plan?	Within two weeks prior to services/phase beginning Within two weeks after services/phase beginning 3-4 weeks after services/phase beginning Other, specify___________
	What focus areas do CTI treatment plans typically address? (check all that apply)	Psychiatric treatment & medication management Money management Substance abuse management Housing crisis management & prevention Family interventions Life skills training Other, specify:_________________
	How many of the focus areas selected in Question #16 typically comprise a CTI treatment plan at any one time?	More than 6 6 4-5 1-3
	Which of the following best describes how treatment plan focus areas are chosen:	Based on consumer ‘s history of risk of homelessness Based on goal attainment/new risk areas identified at end of previous phase of CTI service Other, specify______ Does this vary by phase of service? If yes, explain:_____
Outreach/Early Linking	During the first phase (i.e. first 1-3 months) of CTI services, how is contact maintained between CTI workers and consumers? (check all that apply)	Phone contact is made Home visits are made If home visits made, how soon after the start of Phase One do they occur? Within one week Within two weeks Within one month Other, specify______ Visits are made to clients at their treatment setting (e.g., day program) If clients visited at treatment setting, how soon after the start of Phase One do they occur? Within one week Within two weeks Within one month Other, specify______ Workers accompany consumers on appointments Other, specify:_______________
	How often do CTI workers typically make contact with consumers during the initial phase (1-3 months) of service?	Once per month 2-3 times per month 4 times per month Other, specify_____
	How often do CTI workers typically meet with primary mental health and/or substance use treatment providers during the initial phase (1-3 months) of service?	Once 2-3 times 4 times Other, specify_____
	How often do CTI workers typically meet with housing providers including landlords during the initial phase (1-3 months) of service?	Once 2-3 times 4 times Other, specify_____
	During the initial phase (1-3 months) of service, do CTI workers hold joint meetings between:	Consumers and their community linkages? Yes No Linkages from different agencies? Yes No
Nature/Length of Services	Which of the following principles and approaches do CTI staff use in their work with consumers? (check all that apply)	Confrontation Abstinence only Harm reduction Stage wise approach Office-based assessments Community-based assessment & skill building Other, specify:_________
	What is the total length of time consumers typically receive CTI services?	3 months 6 months 9 months 12 months Other, specify____
	Are consumers ever discharged from services early?	No Yes If yes, why? _______________
	Which of the following activities are most likely to occur during the initial phase (1-3 months) of CTI services?	CTI worker focuses with consumer on work accomplished and long-term goals CTI worker focuses on assessment and linkage with supports CTI worker accompanies consumer to appointments CTI worker observes consumer trying out skills and adjusts consumer support network CTI worker encourages consumer and caregivers to work out problems on their own CTI worker substitutes for caregivers when necessary CTI worker mediates conflicts between consumer and caregivers
	Which of the following activities are most likely to occur during the middle phase (e.g., months 4-6) of CTI services?	CTI worker focuses with consumer on work accomplished and long-term goals CTI worker focuses on assessment and linkage with supports CTI worker accompanies consumer to appointments CTI worker observes consumer trying out skills and adjusts consumer support network CTI worker encourages consumer and caregivers to work out problems on their own CTI worker substitutes for caregivers when necessary CTI worker mediates conflicts between consumer and caregivers
	Which of the following activities are most likely to occur during the final phase (e.g., months 7-9) of CTI services?	CTI worker focuses with consumer on work accomplished and long-term goals CTI worker focuses on assessment and linkage with supports CTI worker accompanies consumer to appointments CTI worker observes consumer trying out skills and adjusts consumer support network CTI worker encourages consumer and caregivers to work out problems on their own CTI worker substitutes for caregivers when necessary CTI worker mediates conflicts between consumer and caregivers
	How often do CTI workers typically have contact with consumers during the final phase (e.g., months 7-9) of CTI services?	Once per month 2-3 times per month 4 times per month Other, specify____
	Were any components of this program model difficult to implement?	No Yes If yes, specify_________________
	Did you make any adjustments or modifications to the CTI model?	No Yes If yes, please describe     
	Were any of the following types of evidence-based service interventions fully imbedded within your implementation of the CTI program model?	Motivational Interviewing Cognitive Behavioral Therapy (CBT) Motivational Enhancement Therapy (MET) Peer Support Strengths-Based Case Management/Approach SSI/DI Outreach, Access & Recovery (SOAR) Trauma-Specific Intervention (specify:___________) Other (specify:___________________)

1 Defined as those primary EBPs that are program-level models being implemented in 14 or more sites for which a fidelity toolkit/scale exists.

2 Defined as those primary EBPs that are program-level models being implemented in 14 or more sites for which a fidelity toolkit/scale exists.