Form Discussion Guide_C Discussion Guide_C Discussion Guide_Cost Questionnaire

Evaluation of Programs to Provide Services to Persons Who Are Homeless with Mental and /or Substance Use Disorders

Attachment 02_SV Discussion Guide_Cost Questionnaire

Project Director Interview

OMB: 0930-0339

Document [docx]

Download: docx | pdf

Attachment 2: Site Visit Discussion Guides and Cost Questionnaire

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this project is 0930-XXXX. Public reporting burden for this collection of information is estimated to average 3.5 hours per respondent, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

Opening Session/Project Director Discussion Guide

National Evaluation of SAMHSA’s Homeless Programs

The key informants for this session are the Project Director, Program Manager/Coordinator, upper management, evaluator, treatment staff, and primary partners as identified by the PD.

Shape1

Note for Site Visitors:

The purpose of this session is several fold: (1) Ensuring the grantee and key staff have an understanding of the multi-site evaluation and their involvement over the next few years; (2) that the site visitors understand the grantee agency, its relationship to the program and community homeless services; (3) to gain a solid overview of the program through a walkthrough of the Client Flow Chart, Organizational Chart and Program Logic Model; (4) to provide a detailed understanding of the treatment and other services implemented by the program, barriers and facilitators to program implementation, and lessons learned; and (5) to provide an overview of grant staffing and sustainability.

Prior to the interview, site visitors should review the Document Extraction and the Project Director Telephone Interview, as well as materials on the primary EBP(s). In addition, site visitors should pre-fill the sections of the protocol that refer to the Project Director Interview (e.g., services, EBPS, etc.). Please review and edit this data, as necessary, during the interview.

Make sure respondents are aware that consent will need to be completed before commencing the session and that the session is audio taped. Once consents and sign-in sheet are completed and collected, the session can begin. The Lead Site Visitor should describe the multi-site study and the Economic Advisor should lead the Client Flow Chart, Organization Chart, and Program Logic Model walk through. The priority is the Client Flow Chart with other charts clarified as time allows. Site visitors should make sure they are making corrections to the Client Flow Chart during the session (and subsequent sessions) as it will be used as a reference to ask questions and will be reviewed during other interviews to ensure accuracy and deeper our understanding of the program.

Following the site visit it will be the Economic Advisor’s responsibility, with the input/review of the other site visitors, to update and complete the Client Flow Chart, Program Logic Model and make any necessary changes to the grantee’s Organization Chart.

Shape2

In summary it is the Lead Site Visitor’s responsibility to:

Introduce site visitors and purpose
Hand out consents
review consent process, read consent and have respondents sign with only first name and last initial
Note audio taping (also included in the consent)
Have respondents complete sign-in sheets with only first name and last initial
Collect consents and sign-in sheets
Consents and sign-in sheets will be sent to RTI Data Manager following the interview along with submission of the site visit notes and audio recording.

Introductory Script:

Welcome and thank you for participating in the Homeless Programs site visit. We appreciate your taking the time out of your busy day to meet with us. This site visit is part of the evaluation of SAMHSA’s portfolio of Homeless Programs, including Projects for Assistance in Transition from Homelessness (PATH); Services in Supportive Housing (SSH); and Grants for the Benefit of Homeless Individuals (GBHI); which includes tracks focused on SSH, General GBHI grantees, and Cooperative Agreements to Benefit Homeless Individuals (CABHI).

[Site visitors should also discuss: (1) The purpose of the evaluation, (2) the overall sample (three-tiered strategy for data collection), (3) types of data collection, (4) types of products, and (5) answer any questions. More detail on these talking points will be in the site visit guide introduction.]

[Site visitors should pass around sign-in sheet]

I want to remind you that everything you say during this conversation will remain private; it’s very important for this study that you feel free to be candid with me. Of course you can refuse to answer any of my questions for any reason. This discussion will take approximately 3 to 4 hours, though we can certainly take a break during that time if you’d like.

[Site visitors should note audio taping and provide time for each respondent to read informed consent, complete, and ask questions as needed.]

I want to point out a couple of things before we begin. Throughout this interview, when I refer to the “SSH/GBHI/CABHI program” or “your program” I am specifically talking about the SSH/GBHI/CABHI program which is funded by the Substance Abuse and Mental Health Services Administration (SAMHSA).When I refer to “Consumers” or “Clients,” I am specifically referring to the individuals served by the SSH/GBHI/CABHI program.

Do you have any questions before we begin?

Overview of Grantee/Partner Agencies (Addresses EQ 1)

Instructions: The purpose of these questions is to provide an overview of the grantee/partner agencies, including how the grant fits within each agency, and to provide a summary of the central focus of the grant. Site visitors should ask these questions of representatives of both the grantee and partner agency(s) (if applicable).

How would you describe your agency in terms of its mission and its position or role in the local treatment and service system?
How does the program fit within your agency? How does it help fulfill its mission?
1. Review grantee and partner Organization Charts and the Project Organizational Chart (provided by the grantee prior to the site visit).
2. Is the grant part of your agency’s larger strategic plan? Or any other group’s plan?
3. Have there been any challenges to integrating the grant project within your agency?
Does your agency have any other local SAMHSA grants? If yes, how are the grants related?

Shape3

Community Context (Addresses EQ 2)

Instructions: The purpose of these questions is to provide site visitors with a better understanding of the larger community context or local treatment and service system in which the grant functions.

What are the characteristics of the local treatment and service system for the target population?
1. Resource poor? Rich?
2. How are treatment services typically accessed? Financed?
3. What are the major treatment/service gaps?
4. What are the gaps in wraparound services?
5. How is the program targeting these gaps?
What are the homeless and housing resources for the target population?
1. How is housing financed for the target population?
2. Are there housing policies that impact clients’ ability to secure/retain housing?
3. Are there any major housing gaps/barriers?
Relationship of grantee and community homeless prevention/intervention activities.
1. Does the grantee participate in city/county/state efforts to end homelessness? If yes, for how long?
2. Is the grantee aware of, and a part of, the city/county/state 10-year plan?) If yes, for how long?
3. Is the grantee aware of, and a part of, the homeless continuum of care? If yes, for how long?
Are there important community-level changes that we should understand (e.g., changes in funding, local events, etc.) that affect your project’s implementation or effectiveness?

Shape4

Brief Project Overview (Addresses EQ 2)

Instructions: The purpose of these questions is to provide site visitors with a brief overview of the program including partners and collaboration.

What is the central focus of the grant – the heart and soul of your approach?
How is your program organized to provide services? (e.g., partnering and service delivery approach)

Note to Site Visitors: If the program has a viable logic model, use as a basis for discussion on project organization, including how things fit together. For example, why providing service X is expected to affect outcome Y.

Why is the project organized this way?
What service gaps does the project respond to?
Have there been any notable changes in the general approach from what was proposed? If so, what are they and why did they occur?

Who are your service/housing partners for this project (if any) - both formal and informal? What is their role in the project?
1. What is their role in the treatment and service system?
Has the project led your agency to change how it partners with or relates to other agencies or systems?
1. Prior to this grant did your agency previously collaborate with partner agencies or stakeholders? If yes, with which partners and how?
2. Have you formed new linkages with partners in key housing and service areas?

Are there any stakeholders missing?
What kinds of external meetings does the project use for collaboration among partners (e.g., advisory groups, committees, meetings, client case meetings, and other info sharing) and how often are the various types of collaboration used?
1. Have there been any challenges to collaboration? How have you overcome them?
2. Is there a Stakeholder Committee/Community Consortium?
  1. What agencies are represented on it?
  2. What is its purpose? How often does it meet?
3. What types of committees/consortiums, relevant to the target population, existed prior to funding?
4. What was the grantee involvement in these committees/consortiums prior to and following grant funding?
What kinds of internal meetings does the project hold? How often and how long are these meetings?
1. Staffing meetings?
2. Case conferences (among staff, clients not present)?
3. Project management meetings?
4. Meetings with clients as part of treatment?
How do you coordinate with other service providers around clients? Describe communication, documentation, and other processes.
1. With housing providers to maintain housing? (Such as subsidy administrators, landlords, property managers, etc.)

Shape5

Target Population (Addresses EQ 2)

Instructions: The purpose of these questions is to provide site visitors with a better understanding of the target population, why it was chosen and project specific inclusion/exclusion criteria.

Note to site visitors: Prefill this section as necessary based on grant application and/or PD Engagement/Telephone Interviews.

As we understand it, the target population for your GBHI/SSH/CABHI program is __________.
Has the target population changed since submitting the proposal? If so, why?
What are the specific criteria for program enrollment?
Why are you using these criteria? Are there specific factors that influenced your choice of target population?
Are there specific exclusion criteria? (e.g., drug use)
[If not already covered] Was the target population previously served in the community? If yes, do services differ under the GBHI/SSH/CABHI grant?

Shape6

Client Flow (Addresses EQ 1)

Instructions: The purpose of these questions is to gain a solid overview of the program through a walkthrough of the Client Flow Chart.

Note to site visitors: This section should be led in collaboration with the Economic Advisor. The Client Flow Chart is to be used as a tool to ensure your understanding is complete and accurate – as well as a visual guide for the discussion. Carefully note the respondent’s role in each project component. Please use the prompts below as needed – we want to know the setting, staffing, time between events, whether each service is provided by the grantee, a partner or referral, etc. Update the Client Flow Chart as necessary to be used in later meetings.

How do you identify project clients?
1. Who does this? (i.e., agency, staff)
2. When and where is it done?
3. Do partners or other groups refer clients to your project? If yes, which partners and for what services?
4. Since project funding, has the process for how you identify clients changed?
How do you recruit project clients?
1. Who does this? (i.e., agency, staff)
2. When and where is it done?
3. Since project funding, has the process for how you recruit clients changed?
How do you screen/assess project clients?
1. Who does this? (i.e., agency, staff)
2. When and where is it done?
3. What processes or instruments are used?
4. Since project funding, has the process for how you screen or assess clients changed?
How do you engage project clients?
1. Who does this? (i.e., agency, staff)
2. When and where is it done?
3. What processes, tools or incentives do you use? Are staff trained in engaging clients?
4. Since project funding, has the process for how you engage clients changed?
For the typical client, how much time elapses between:
1. Identification and recruitment?
2. Recruitment and screening/assessment?
3. Screening/assessment and program acceptance?
4. Program acceptance and administration of GPRA (if not part of initial screening or assessment)?
5. Program acceptance and securing housing?
For the treatment services provided by the project, please specify:
1. What treatment services are provided by the project? (List all treatment services and ask follow up questions for each. Site visitors use separate Treatment Services Table for reporting)
  1. Who (agency/staff) provides the service (e.g., grantee agency, referral)?
  2. Who pays for the service (e.g., grant, in-kind, client)?
  3. When and where (setting & individual/group) is the treatment service provided?
  4. Is the treatment service curriculum/manual based? (Note to site visitors – more specific questions on treatment services in EBP section.)
2. What is the approximate client-to-counselor caseload for treatment providers for this project?
3. Since project funding, has the process for how you provide treatment services changed?
For the case management/wraparound services provided by the project please specify:
1. What specific services are provided? (List all case management/wraparound services and ask follow up questions for each. Site visitors – use the separate Case Management/Wraparound Services Table for reporting)
  1. Who (agency/staff) provides the service (e.g., grantee agency, referral)?
  2. Who pays for the service (e.g., grant, in-kind, client)?
  3. When and where (setting & individual/group) is the case management/wraparound service provided?
  4. Is the case management/wraparound service curriculum/manual based?
  5. Is the service provided as part of case management?
2. What goals and objectives does the project have for case management/wraparound services with respect to client outcomes?
3. Do case managers co-manage clients? Describe how many other case managers, who they are, all or a subset of clients, why/when/how you coordinate.
4. Is case management/wraparound provided differently with SSH/GBHI/CABHI clients than for other clients? If yes, how (e.g., approach, services, etc.)?
5. Since project funding, has the process for how you provide case management changed?
[If not already covered] How are mainstream benefits (e.g., SSI/SSDI, TANF, SNAP, etc.) accessed?
1. Who provides benefits assistance and how?
2. What type of benefits are clients connected with?
3. Are any specific models used, such as SOAR?
4. Has the process for accessing mainstream benefits changed as a result of this grant? If so, how?
For housing, please specify where it is accessed and who provides it. (Review where housing fits on flow chart to understand the focus of the program vis-a-vis housing)
1. Where are clients housed before they come to your program?
  1. Identify the types (e.g., shelter, transitional, etc.) and the approximate percentage in each type.
  2. Identify who provides it. How is it accessed?
2. Where are clients housed during the program?
  1. Identify the types (e.g., shelter, transitional, etc.) and the approximate percentage in each type.
  2. Identify who provides it. How is it accessed?
3. Where are clients housed after the program?
  1. Identify the types (e.g., shelter, transitional, etc.) and the approximate percentage in each type.
  2. Identify who provides it. How is it accessed?
4. If PSH provided, how many units are available for clients? (Note to site visitors – more detailed questions about PSH in fidelity section.)
5. How and when are project clients discharged from housing? Are housing services provided after discharge?
6. Since project funding, has the process for accessing housing changed?
For housing supports, how would you describe your project’s overall approach to client housing stability? Specifically, how does the project integrate, housing, treatment and wraparound services?
1. What specific housing support services is your agency providing project clients?
  1. How and why were those specific services selected?
2. Did your agency provide these services to this population prior to the grant? Has the grant impact the way these services are delivered?
3. What goals and objectives does the project have for housing support services and client outcomes?
  1. So far, how successful has the project been in meeting these goals and objectives?

Why did the project choose to provide the [treatment/case management/wraparound] services?
In the project case flow, are there particular bottlenecks or points where clients drop out or terminate services or housing?
How long can project clients receive services from your program? Specify for each service modality.
What is the attrition rate? Does the program have any special procedures to address attrition?
How and when are project clients discharged? Are services provided after discharge?

Shape7

Systems and Client Outcomes (Addresses EQ 4)

Instructions: The purpose of these questions is to better understand how project implementation has affected the grantee agency and the larger treatment system.

Has your project influenced any processes in the local treatment and services system? If yes, how? Probe areas not noted by respondent.
1. Client outreach and referral?
2. Intake and assessment?
3. Case management?
4. Treatment?
5. Housing access/supports?
6. Other mechanisms for service access, delivery, or retention?
If project is focusing on systems outcomes (e.g., changes in services, policies, systems integration, housing, workforce stability and competence, etc.):
1. What system outcomes is it focusing on?
2. Has there been progress to date? (i.e., specific, measurable evidence of improvement in system outcomes)
How does the program track client outcomes related to treatment, case management and housing services?
What do you think are the main client outcomes that should be tracked for this project?
If you are involved in data collection, do you feel the GPRA questions are useful measures for your client outcomes?
1. Are there limitations with GPRA/NOMS with regard to the description of your clients and their outcomes?
2. What should we take into account when we interpret GPRA outcomes for this site?

Shape8

Barriers, Facilitators & Innovations (Addresses EQ 2)

Instructions: The purpose of these questions is to better understand the barriers and facilitators to project implementation and service delivery, as well as any changes or innovations – among grantee, partners, and the larger treatment system – as a result of the grant.

Has the project encountered any barriers or facilitators to implementation or service delivery, including:
1. Barriers/Facilitators to identifying and enrolling the intended numbers of project clients?
2. Barriers/Facilitators to retaining target population clients?
3. Barriers/Facilitators to providing treatment and case management services to clients?
4. Barriers/Facilitators to obtaining housing for clients?
5. Barriers/Facilitators to collaboration among partners?
6. If a goal- has your project encountered specific barriers/facilitators to helping achieve systems change through this project?
7. Have you experiences any other barriers/facilitators we have not discussed?
Has the grant led to any innovations at your agency or your partners? For example, are there new processes, technologies, or tracking approaches?
Has the grant led to any changes/innovations in the treatment system? For example, are there new processes, technologies, or tracking approaches?

Shape9

Lessons Learned (Addresses EQ 2)

Instructions: The purpose of these questions is gather information on lessons learned, regarding project organization, implementation and outcomes; and to assess to what degree they are relatable across SSH/GBHI/CABHI grantees.

Do you have any lessons learned or recommendations regarding your project related to:
1. Project organization or approach?
2. Successfully serving this target population?
3. Improving client outcomes for your project?
4. Improving systems outcomes (pertinent to this project)?
5. The safety of your clients, treatment staff, and the supports provided to the latter? Have you changed the way you provide any services as a result of an incident?
What would you do differently if you were starting the project again?
What has been the biggest impact/value added by the grant?

Shape10

Sustainability (Addresses EQ 6)

Instructions: The purpose of these questions is to better understand progress towards (and intentions for) sustainability efforts.

Have you developed a formal written sustainability plan?
1. If yes, what are the sustainability goals, key elements for achieving them and timeline?
2. If not, please describe the informal efforts you are currently engaged in to sustain your program? What are your future plans?
Which program elements will you seek to sustain?
How do your sustainability efforts/plans for your project fit within the agency’s strategic plan?
How does the project sustainability plan fit in the community service system?
Is the sustainability plan/efforts related to planning efforts such as the local HUD Consolidated Plan or Continuum of Care planning?
Which partners are currently involved in sustainability planning/efforts, and how?
1. Who are the champions (or key stakeholders) for your project?
2. Who are you hoping to engage in future sustainability planning/efforts? (i.e., who do you believe is essential to ensure future funding and/or maintaining services?)
Are you using (or planning to use) evaluation data to support sustainability? If so, how?
What are the plans for sustaining financing and other resources?
What effect do you think not sustaining this program will have on clients, the community, or the local service systems?

Shape11

Shape12

Note to site visitors: The remaining question blocks should be asked of the PD only. All others present may leave at this time. Remember to thank them for their time and input.

Shape13

Project Staffing (Addresses EQ 1, 2)

Instructions: The purpose of these questions is to gain a better understanding of staff roles, hiring, turnover and retention.

Note to site visitors: Ask this question block of PD only. Please complete staff table in advance of meeting.

Briefly review the staff funded by the grant – prefill from grant application and PD interview.
1. Has this changed since the application? Why?
Have you faced any challenges in hiring staff? If yes, how did you address them?
How much turnover have you experienced among project staff? If turnover, explore reasons and lessons learned.
What measures are used to retain staff?
Does project staff reflect the project’s target population?
Does project staff include peers or clients? If so, describe their roles.

Shape14

EBPs/Best Practices & Training (Addresses EQ 1)

This section is to be asked of the PD and Program Manager only-and only asked if agency responsible for implemented the main Program EBP/Practice.

Instructions: The purpose of these questions is to gain a better understanding of the components/elements of the primary services being implemented by the site. The respondent should describe the components/elements of each EBP/Best Practice as well as any modifications. This section also asks about organizational support of the EBP implementation.

Note to site visitors: In advance of site visit, please identify primary EBPs/Practices and review fidelity measures. Be sure to use the appropriate fidelity scale components as reference for questions. You will score the worksheets after the site visit using the appropriate fidelity scale sheet(s).

What is the primary service EBP(s)/best practices being implemented by the program?
1. Why was this EBP/best practice selected?
2. Does your agency have prior experience with this EBP/best practice? How long?
Does the EBP/ best practice have adequate organizational support/resources?
1. How does your agency administration/leadership support the EBP/ best practice? Is this EBP in an agency plan? Is there an assigned leader? Are there state/local policies that support implementation?
How is the EBP/ best practice funded? How have start-up/conversion costs been covered? Is this EBP/ best practice reimbursable? How will it be covered after funding SAMHSA funding ends?
What are the staffing requirements for the EBP/ best practice (number, type, and structure)?
1. What are the staff qualifications/requirements?
2. Practitioner to client ratio?
What are the core components of the EBP/ best practice?
1. Specific eligibility criteria? Any issues with finding eligible clients? Have you made any modifications to criteria?
2. Modalities of clinical services provided, including by whom, frequency, session format, and intensity- content (Ask only if relevant to EBP/Practice)
3. Modalities of non-clinical services provided: including by whom, frequency and intensity. (Ask only if relevant to EBP/Practice)
4. Engagement and outreach techniques and strategies?
5. Are the services limited or unlimited? Typical length? Graduation requirements?
6. Please describe any modifications made to the EBP/ best practice, and the rationale for these changes.
7. From your perspective, how do these EBP/ best practice services benefit clients? Do you have any outcome data to support this?
Please describe the training provided to staff to implement this EBP/ best practice.
1. Who conducted the training (credentials)? How many? Did all staff participate?
2. What was the content? Source material (manual, toolkit)? Teaching aids?
3. What types of training activities/methods are used with staff? Demonstrations? Modeling? Role play? Coaching?
4. If there is on-going/refresher trainings- please describe type and frequency.
5. Is there on-going consultation/technical support to implement the EBP/ best practice?
6. How are new staff members trained up?
Please describe the on-going supervision/support mechanisms for the EBP/ best practice.
1. Frequency, length, content and format of supervision? Access to problem solving? Feedback sessions?*
2. How does supervision help with the work?
3. Does the program keep records/document supervision?*
Please describe EBP/ best practice monitoring and fidelity processes.
1. Who is responsible for monitoring/fidelity?
2. Do all clients in program receive EBP/best practice? What is client retention rate?
3. What types of data are collected: fidelity scales, worksheets, client level outcome data, consumer experience? How often?
4. How is the data used? (e.g., supervision, staff competency, etc.) Who is it shared with? How often?
5. Current fidelity/status of implementation (if available).
6. Please describe any adjustments made to the program based on fidelity data?
What barriers have you encountered in implementing this EBP/ best practice? At the agency level? At the clinician or staff level? State or local regulations?
Have there been any facilitators/conditions within your agency that have helped with implementing this EBP/ best practice?

Shape15

SAMHSA Training (Addresses EQ 2)

Instructions: The purpose of these questions is gain a better understanding of general training and training supported by SAMHSA.

What training has been provided staff to support their position in the project – that is not specific to an EBP (addressed above)?
1. Have project staff received any non-SAMHSA technical assistance (TA) since your project began? If yes, please describe type of training, when it was received and by whom it was funded.
2. Have staff received SAMHSA technical assistance (TA)?
  1. If yes, what type of training or TA was received and when? (e.g., webinar, site visit, regional training, grantee meeting, etc.)
  2. How helpful was the training or TA?
3. Who initiated the TA request and why?
Has training or technical assistance impacted project implementation?

Shape16

Permanent Supportive Housing EBP Questions (Addresses EQ 4)

Instructions: The purpose of these questions is to understand the degree to which grantees are implementing the Permanent Supportive Housing (PSH) EBP.

Note to site visitors: Only grantees implementing PSH are asked this set of questions; see housing screener from PD phone interview. Ask this question block of PD only or Agency/Partner if responsible for implementing EBP. Housing staff will be asked this question block separately. You will score this section after the site visit using the PSH Fidelity Scale sheet.

To what extent do clients have choice among types of housing?
1. Do they have a choice of unit within the housing model? Do they have a choice of neighborhood? How many units do they see?
2. Can they wait for unit of choice without losing their place on eligibility list or the voucher?
3. Can they control the composition of their household?
Are social and clinical service providers located off-site? On-site? If off-site, do they deliver any services on site?
1. Are service providers responsible to for housing management functions?
2. Does housing management have any authority or formal role in social services?
3. Can services be delivered in locations of clients’ choice?
4. How does the program respond if a client loses his/her housing? What actions are taken? (e.g., are they discharged? Still eligible to receive services?)
On average, what proportion of their income do clients pay for housing? What percentage of housing units meets HUD’s Housing Quality Standards? How much do these vary by client?
What proportion of clients live in buildings where units are occupied by people meeting special needs eligibility criteria? What proportion of these units are occupied by people meeting special needs criteria?
1. Do clients live in clustered site housing? (e.g., one large building with 8 or more clients all meeting disability related criteria)
2. Is the housing in the private market housing? Owned by public housing authorities? Owned by non-profit agencies? Behavioral health service agency?
3. How many neighborhoods are clients living in?
Do clients have full legal rights to tenancy?
1. Are there requirements for tenancy? (e.g., sobriety/abstinence? Symptom stability? Treatment participation? Meet with staff?) Are any of these in the lease/occupancy agreement?
2. Is tenancy/voucher contingent on program/treatment compliance?
3. Are there time limits for tenancy? (other than those defined in a standard lease occupancy/voucher agreement)
Are there readiness requirements for clients? How selective are referral sources?
1. Are clients with stability problems given a priority? (e.g., chronic homeless, criminal record, active substance use, active mental health symptoms, refusal to participate in treatment, etc.)
2. Are there requirements to access the housing? (e.g., sobriety/abstinence, medication compliance, stability of psychiatric symptoms, etc.)
3. What is the waiting period for program enrollment? Are clients required to complete some other type of program before accessing housing? (e.g., outpatient, residential, transitional, etc.) What type?
How much control do clients have over their housing? (e.g., can they control staff entry?)
Can clients choose the type of services they want? At program entry?
1. Is this from an array of services – or a standard package?
2. Can the services be changed/adapted at the client’s request? Can clients refuse services?
3. How much consumer control/input is in the design and provision of services?
4. Can intensity of services be altered?
What is the staff/client ratio? For case managers? Prescribing MD?
1. Are there policies on intensity of client/staff contact?
2. Are services team-based?
Are there opportunities for client input into program operations and policy? (e.g., formal grievance policy, community meetings, planning committees, governing body, etc.)
1. Are persons with experiences with substance abuse or mental health employed in regular staff positions? Employed as Peer Specialists?

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this project is 0930-XXXX. Public reporting burden for this collection of information is estimated to average 2 hours per respondent, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

Case Managers, Treatment, and Housing Staff/Provider Discussion Guide

National Evaluation of SAMHSA’s Homeless Programs

The key informants for this session are treatment providers/staff, case managers, and housing providers/staff. The protocol is not intended to be administered in its entirety. Site Visitors should select the appropriate modules of questions depending on the types of staff participating in the interview.

Shape17

Note for Site Visitors:

The purpose of this session is threefold: (1) understand the role of housing in the project, the types of housing provided to clients, and the relationship between housing and service provision; (2) understand the treatment components of the intervention, including, implementation of services and evidence-based practices; and (3) understand the case management and other wraparound service components for the intervention. The discussion guide focuses on implementation, alignment of services with client needs, barriers and facilitators, and lessons learned related to housing, treatment, and case management/wraparound services.

Site visitors should review the treatment, EBPs, case management/wraparound, and housing related questions from the PD Telephone Interview and prefill tables/questions as indicated in the guide. Site visitors should have the Client Flow Chart as well to reference or check components to ensure that it is complete and accurate.

If any of the respondents were not present during the previous session, complete consents before the interview commences and

Review consent process
Note audio taping
Have respondents complete sign-in sheets
Collect consents and sign-in sheets before the interview commences
Any additions and/or changes to the Client Flow Chart should be coordinated with the Economic Advisor and Lead Site Visitor
Consents, sign-in sheets and staff experience charts should be sent to the RTI Data Manager following the interview along with submission of the site visit notes and audio recording.

Introductory Script and Informed Consent:

Shape18

[Site visitors should pass around sign-in sheet]

I want to remind you that everything you say during this conversation will remain private; it’s very important for this study that you feel free to be candid with me. Of course you can refuse to answer any of my questions for any reason. This discussion will take approximately 2 hours, though we can certainly take a break during that time if you’d like.

[Site visitors should note audio taping and provide time for each respondent to read informed consent, complete, and ask questions as needed.]

Do you have any questions before we begin?

Shape19

Overview of Treatment, Case Management, & Housing Provider(s): History of Collaboration, Role in Project & Impact (Addresses EQ 1, 2)

Instructions: The purpose of these questions is to better understand the role of the treatment, case management, and housing providers/agencies (if not the grantee) in the grant, their history of collaboration and the grant’s impact on leveraging existing resources or systems.

Note to site visitors: Ask these questions on role in grant and collaboration only if treatment, case management, or housing is provided by an external agency other than the grantee.

How would you describe your agency in terms of its mission and its position or role in the local housing/treatment system?
1. How does this project fit into that mission?
[Housing Provider Only] How is your agency involved with the city/county/state 10-year plan to end homelessness and the homeless continuum of care?
1. What is your agency’s role in implementing the SSH/CABHI/GBHI project?
2. How does your agency help the project achieve its goals and objectives?
Prior to the grant did your agency collaborate with the grantee or other partner agencies or stakeholders? If yes, which partners and how?
What mechanisms does your agency use in collaborating with project partners (e.g., committees, meetings, client case meetings, other info sharing, etc.)?
1. How often does your agency use each type of collaboration?
How often is planning and assessment updated for [treatment, case management, and housing] services and supports?
What kinds of internal meetings does the project hold? What kinds of external meetings? How often and how long are these meetings? Probe for: Staffing meetings? Case conferences (among staff, clients not present)? Advisory board meetings? Project management meetings? Stakeholder meetings? Meetings with clients as part of treatment?
How do you coordinate with other service providers? Describe communication, documentation, and other processes.
1. With housing providers to maintain housing? (Such as subsidy administrators, landlords, property managers, etc.)
Does the grant support, supplement, or facilitate existing resources or systems? If yes, how?
Has the grant affected how your agency accesses or leverages existing resources or systems? If yes, how?
Did the grant affect your agency’s collaboration with the grantee or other grantee partners? If so, how?
1. Have new linkages been facilitated between your agency and partners? If so, please describe them and the effect on services or outcomes.
In what ways does the project address/fill specific gaps in housing and housing support services in the community?
Has the grant influenced the way your agency provides treatment/housing or supports?

Shape20

Client Flow (Addresses EQ 1)

Instructions: The purpose of these questions is to gain a solid overview of the program through a walkthrough of the Client Flow Chart.

How do you identify project clients?
1. Who does this? (i.e., agency, staff)
2. When and where is it done?
3. Do partners or other groups refer clients to your project? If yes, which partners and for what services?
4. Since project funding, has the process for how you identify clients changed?
How do you recruit project clients?
1. Who does this? (i.e., agency, staff)
2. When and where is it done?
3. Since project funding, has the process for how you recruit clients changed?
How do you screen/assess project clients?
1. Who does this? (i.e., agency, staff)
2. When and where is it done?
3. What processes or instruments are used?
4. Since project funding, has the process for how you screen or assess clients changed?
How do you engage project clients?
1. Who does this? (i.e., agency, staff)
2. When and where is it done?
3. What processes, tools or incentives do you use? Are staff trained in engaging clients?
4. Since project funding, has the process for how you engage clients changed?
For the typical client, how much time elapses between:
1. Identification and recruitment?
2. Recruitment and screening/assessment?
3. Screening/assessment and program acceptance?
4. Program acceptance and administration of GPRA (if not part of initial screening or assessment)?
5. Program acceptance and securing housing?
For the treatment services provided by the project, please specify:
1. What treatment services are provided by the project? (List all treatment services and ask follow up questions for each. Site visitors use separate Treatment Services Table for reporting)
  1. Who (agency/staff) provides the service (e.g., grantee agency, referral)?
  2. Who pays for the service (e.g., grant, in-kind, client)?
  3. When and where (setting & individual/group) is the treatment service provided?
  4. Is the treatment service curriculum/manual based? (Note to site visitors – more specific questions on treatment services in EBP section.)
2. What is the approximate client-to-counselor caseload for treatment providers for this project?
3. Since project funding, has the process for how you provide treatment services changed?
For the case management/wraparound services provided by the project please specify:
1. What specific services are provided? (List all case management/wraparound services and ask follow up questions for each. Site visitors – use the separate Case Management/Wraparound Services Table for reporting)
  1. Who (agency/staff) provides the service (e.g., grantee agency, referral)?
  2. Who pays for the service (e.g., grant, in-kind, client)?
  3. When and where (setting & individual/group) is the case management/wraparound service provided?
  4. Is the case management/wraparound service curriculum/manual based?
  5. Is the service provided as part of case management?
2. What goals and objectives does the project have for case management/wraparound services with respect to client outcomes?
3. Do case managers co-manage clients? Describe how many other case managers, who they are, all or a subset of clients, why/when/how you coordinate.
4. Is case management/wraparound provided differently with SSH/GBHI/CABHI clients than for other clients? If yes, how (e.g., approach, services, etc.)?
5. Since project funding, has the process for how you provide case management changed?
[If not already covered] How are mainstream benefits (e.g., SSI/SSDI, TANF, SNAP, etc.) accessed?
1. Who provides benefits assistance and how?
2. What type of benefits are clients connected with?
3. Are any specific models used, such as SOAR?
4. Has the process for accessing mainstream benefits changed as a result of this grant? If so, how?
For housing, please specify where it is accessed and who provides it. [Review where housing fits on flow chart to understand the focus of the program vis-a-vis housing]
1. Where are clients housed before they come to your program?
  1. Identify the types (e.g., shelter, transitional, etc.) and the approximate percentage in each type.
  2. Identify who provides it. How is it accessed?
2. Where are clients housed during the program?
  1. Identify the types (e.g., shelter, transitional, etc.) and the approximate percentage in each type.
  2. Identify who provides it. How is it accessed?
3. Where are clients housed after the program?
  1. Identify the types (e.g., shelter, transitional, etc.) and the approximate percentage in each type.
  2. Identify who provides it. How is it accessed?
4. If PSH provided, how many units are available for clients? (Note to site visitors – more detailed questions about PSH in fidelity section.)
5. How and when are project clients discharged from housing? Are housing services provided after discharge?
6. Since project funding, has the process for accessing housing changed?
For housing supports, how would you describe your project’s overall approach to client housing stability? Specifically, how does the project integrate, housing, treatment and wraparound services?
1. What specific housing support services is your agency providing project clients?
  1. How and why were those specific services selected?
2. Did your agency provide these services to this population prior to the grant? Has the grant impact the way these services are delivered
3. What goals and objectives does the project have for housing support services and client outcomes?
  1. So far, how successful has the project been in meeting these goals and objectives?
Why did the project choose to provide the [treatment/case management/wraparound] services?
In the project case flow, are there particular bottlenecks or points where clients drop out or terminate services or housing?
How long can project clients receive services from your program? Specify for each service modality.
What is the attrition rate? Does the program have any special procedures to address attrition?
How and when are project clients discharged? Are services provided after discharge?

Shape21

EBPs/Best Practices (Case management, wraparound, and treatment) (Addresses EQ 1)

Instructions: The purpose of these questions is to gain a better understanding of the components/elements of the primary service EBPs/Best Practices being implemented by the site. The respondent should describe the components/elements of each EBP as well as any modifications.

Note to site visitors: In advance of site visit, please identify primary EBPs and review fidelity measures (if available). Be sure to use the appropriate fidelity scale components as reference for questions. You will score the worksheets after the site visit using the appropriate fidelity scale sheet(s).

What is the main EBP/Best Practice(s) implemented by the program?
What are the staffing requirements for the EBP/Best Practice (number and type)?
1. What are the staff qualifications/requirements?
2. Practitioner to client ratio?
What are the core components of the EBP/Best Practice?
1. Specific eligibility criteria? Have there been an issues reaching this target population? Has the criteria shifted/changed since the project started?
2. Modalities of clinical services provided, including by whom, frequency, session format, and intensity- content (Ask only if relevant to EBP/Best Practice)
3. Modalities of non-clinical services provided: including by whom, frequency and intensity. (Ask only if relevant to EBP/Best Practice)
4. Engagement and outreach techniques and strategies?
5. Are the services limited or unlimited? Typical length? Graduation requirements?
6. Please describe any modifications made to the EBP/Best Practice, and the rationale for these changes.
7. From your perspective, how do these EBP/Best Practice services benefit clients? Do you have any outcome data to support this?
Please describe the training provided to staff to implement this EBP/Best Practice.
1. Who conducted the training (credentials)? How many? Did all staff participate?
2. What was the content? Source material (manual, toolkit)? Teaching aids?
3. What types of training activities/methods are used with staff? Demonstrations? Modeling? Role play? Coaching?
4. If there is on-going/refresher trainings- please describe type and frequency.
5. Is there on-going consultation/technical support to implement the EBP/Best Practice?
6. How are new staff members trained up?
Please describe the on-going supervision/support mechanisms for the EBP/Best Practice.
1. Frequency, length, content and format of supervision? Access to problem solving? Feedback sessions?
2. How does supervision help with the work?
3. Does the program keep records/document supervision?
Please describe EBP/Best Practice monitoring and fidelity processes.
1. Who is responsible for monitoring/fidelity?
2. How do you know which clients are receiving the EBP?
3. What types of data are collected: fidelity scales, worksheets, client level outcome data, consumer experience? How often?
4. How is the data used? (e.g., supervision, staff competency, etc.) Who is it shared with? How often?
5. Current fidelity/status of implementation (if available).
6. Please describe any adjustments made to the program based on fidelity data?
What barriers have you encountered in implementing this EBP/Best Practice? At the agency level? At the clinician or staff level? State or local regulations?
Have there been any facilitators/conditions within your agency that have helped with implementing this EBP/Best Practice?

Shape22

Permanent Supportive Housing EBP Questions (Addresses EQ 1)

Instructions: The purpose of these questions is to understand the degree to which grantees are implementing the Permanent Supportive Housing (PSH) EBP.

Note to site visitors: Only grantees implementing PSH are asked this set of questions; see housing screener from PD phone interview. Ask this question block of staff responsible for implementing EBP. You will score this section after the site visit using the PSH Fidelity Scale sheet.

To what extent do clients have choice among types of housing?
1. Do they have a choice of unit within the housing model? Do they have a choice of neighborhood? How many units do they see?
2. Can they wait for unit of choice without losing their place on eligibility list or the voucher?
3. Can they control the composition of their household?
Are social and clinical service providers located off-site? On-site? If off-site, do they deliver any services on site?
1. Are service providers responsible to for housing management functions?
2. Does housing management have any authority or formal role in social services?
3. Can services be delivered in locations of clients’ choice?
4. How does the program respond if a client loses his/her housing? What actions are taken? (e.g., are they discharged? Still eligible to receive services?)
On average, what proportion of their income do clients pay for housing? What percentage of housing units meets HUD’s Housing Quality Standards? How much do these vary by client?
What proportion of clients live in buildings where units are occupied by people meeting special needs eligibility criteria? What proportion of these units are occupied by people meeting special needs criteria?
1. Do clients live in clustered site housing? (e.g., one large building with 8 or more clients all meeting disability related criteria)
2. Is the housing in the private market housing? Owned by public housing authorities? Owned by non-profit agencies? Behavioral health service agency?
3. How many neighborhoods are clients living in?
Do clients have full legal rights to tenancy?
1. Are there requirements for tenancy? (e.g., sobriety/abstinence? Symptom stability? Treatment participation? Meet with staff?) Are any of these in the lease/occupancy agreement?
2. Is tenancy/voucher contingent on program/treatment compliance?
3. Are there time limits for tenancy? (other than those defined in a standard lease occupancy/voucher agreement)
Are there readiness requirements for clients? How selective are referral sources?
1. Are clients with stability problems given a priority? (e.g., chronic homeless, criminal record, active substance use, active mental health symptoms, refusal to participate in treatment, etc.)
2. Are there requirements to access the housing? (e.g., sobriety/abstinence, medication compliance, stability of psychiatric symptoms, etc.)
3. What is the waiting period for program enrollment? Are clients required to complete some other type of program before accessing housing? (e.g., outpatient, residential, transitional, etc.) What type?
How much control do clients have over their housing? (e.g., can they control staff entry?)
Can clients choose the type of services they want? At program entry?
1. Is this from an array of services – or a standard package?
2. Can the services be changed/adapted at the client’s request? Can clients refuse services?
3. How much consumer control/input is in the design and provision of services?
4. Can intensity of services be altered?
What is the staff/client ratio? For case managers? Prescribing MD?
1. Are there policies on intensity of client/staff contact?
2. Are services team-based?
Are there opportunities for client input into program operations and policy? (e.g., formal grievance policy, community meetings, planning committees, governing body, etc.)
1. Are persons with experiences with substance abuse or mental health employed in regular staff positions? Employed as Peer Specialists?

Shape23

Alignment of Services with Client Needs (Addresses EQ 1)

Instructions: The purpose of these questions is to assess the degree to which projects are aligning services with client needs and to gain a better understanding of client choice and how services are tailored to meet client needs and strengths.

Are project clients given a choice with regard to the type of services (e.g., treatment, case management, housing services and supports) they received from your agency?
In what ways does your program address culture? Gender? Trauma?
What is the role of clients and peers in providing project services (e.g., treatment, case management, housing support services) in your agency?
Do clients in this program receive treatment, case management, and/or housing services and supports from consumer run agencies?
How are client needs and strengths identified and used with regard to treatment, case management, and housing services and supports?
How does your agency tailor treatment, case management, housing services and supports to project client needs and strengths?

Housing Support Only:

What role do housing support staff play in shaping the treatment/recovery plans?
What role, if any, do housing support staff or housing staff play in case conferencing?
Do you use client clinical and/or case management assessments to tailor housing support services?
How many clients do you have on your caseload at any one time?
Describe the type and frequency of contact with clients once they are housed?

For how long do you provide housing support services to clients? Does this vary by type of housing?

Shape24

Client Outcomes (Addresses EQ 3)

Instructions: The purpose of these questions is to assess how client outcomes are tracked.

How does the program track client outcomes related to treatment, case management and housing services?
What do you think are the main client outcomes that should be tracked for this project?
If you are involved in data collection, do you feel the GPRA questions are useful measures for your client outcomes?
1. Are there limitations with GPRA/NOMS with regard to the description of your clients and their outcomes?
2. What should we take into account when we interpret GRPA outcomes for this site?
What has been your experience collecting the Supplemental Client Interview Data for the cross-site evaluation?
1. Are these data useful addition in your local data collection? Are these data linked to other outcome data?

Shape25

Barriers, Facilitators, & Innovations (Addresses EQ 2)

Instructions: The purpose of these questions is to better understand the barriers and facilitators to project implementation and service delivery, as well as any changes or innovations the grant led to among grantee, partners and the larger treatment system.

Has the project encountered any barriers or facilitators related to:(Probe in the following areas and indicate how each was addressed)
1. Serving the intended numbers of project clients?
2. Retaining project clients?
3. Forming linkages or collaborating with project partners?
4. Implementing the intended client flow process?
5. Providing treatment and case management services to project clients?
6. Accessing and retaining housing?
Are there contextual issues, such as rules/regulations, policies, service system changes, or changes in housing markets, that have impacted your project or clients?
Have project activities led to changes in how this target population accesses or is provided housing and housing supports in your community?
Have the grant resources or activities changed the way your agency or your partners deliver services (e.g., new processes, technologies, tracking approaches- innovations, policies, types of services offered, screening/assessment, etc.)?
1. Has it impacted how these are integrated with treatment and/or housing? That is, has the program affected your local community system?
Have grant activities or resources created new approaches/innovations among your agency or your partners? (e.g., new processes, technologies, or tracking approaches)
What would be the impact to the target population if this program ended?

Housing & Housing Supports Only:

Have project activities led to changes in how this target population accesses or is provided housing and housing supports in your community?
Has the grant influenced your agency’s provision or access to housing or housing supports?

Shape26

Lessons Learned (Addresses EQ 2)

Instructions: The purpose of these questions is to gather information on lessons learned regarding project organization, implementation and outcomes; and to determine to what degree they are relatable across SSH/GBHI/CABHI grantees.

Do you have any lessons learned or recommendations to offer based on your experience with the program or target population? Would you do anything differently, if you were starting again, related to:
1. Client case flow or use of resources?
2. EBPs selected and how to best adapt/adopt them?
3. Aligning treatment, case management, housing, and housing support services with client needs?
  1. How best to implement case management, treatment and housing services with this population?
4. Client outcomes?
5. System outcomes?
6. Accessing housing and providing support services to aid the target population in maintaining housing?
7. Achieving targets for providing housing and housing supports to project clients?

Shape27

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this project is 0930-XXXX. Public reporting burden for this collection of information is estimated to average 1.5 hours per respondent, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

Key Stakeholders, Associated Service Providers and Local Funders Discussion Guide

National Evaluation of SAMHSA’s Homeless Programs

The key informants for this session are associated service providers (substance abuse, mental health, housing, case management and other wraparound services or any other service providers who provide services to SSH/GBHI/CABHI clients) and key stakeholders including local funders (local Substance Abuse or Mental Health Authority, local social service agency, local Homeless Taskforce/10-Year Homelessness Committee and local Housing Authority), community boards, criminal justice, family services, consumer/peer services, etc. The Project Director and other staff at the discretion of the PD should be included.

Shape28

Note for Site Visitors:

The purpose of this section is to learn about the GBHI/CABHI/SSH program from the perspective of the associated providers, key stakeholders and local funders. We are interested in learning about the agencies/providers involved in the Project, the ways in which they are involved, as well as their perspective on how the Project has been implemented, its impact/contribution to the community and efforts made toward sustainability. Some grantees may not have external providers or stakeholders; however, they are working with other departments within their agency (their internal partners) for treatment, wraparound services and or housing. Review these questions when conducting other interviews with external/internal partners for service delivery. (Note: If PD indicates there are no external stakeholders/partners linkages, please double check—for instance if they are delivering services in another provider agency/housing setting then that agency is a partner/stakeholder, whether formal or informal).

Due to the diversity of the programs, some may include internal stakeholders/ providers (e.g., from other departments in agency) along with external providers and other types of stakeholders. Depending on agency structure and the scope of outside stakeholders this interview should be given twice with a separate group for internal stakeholders/providers and a separate group for external stakeholders/providers. The format of this would generally be a group—it could be a morning breakfast meeting or a lunch. The Project Director and other staff he/she would like to attend from the program should be present. The Project Director may be concerned initially that the site visitors want to speak independently with the stakeholders and that this could undermine the relationship with the grantee; if this is the case, make clear the types of questions that would be asked and that the Grantee is the host. The site visitors should be mindful when asking questions and ensure it remains a discussion rather than a structured interview unless this is only with one partner and the PD.

If a group meeting, this should be administered as a focus group.

Shape29

If any of the respondents were not present during the Opening Session-PD Interview, complete consents before the interview commences and

Review consent process
Note audio taping
Have respondents complete sign-in sheets
Collect consents and sign-in sheets before the interview commences
Any additions and/or changes to the Client Flow Chart should be coordinated with the Economic Advisor and Lead Site Visitor
Consents, sign-in sheets and staff experience charts should be sent to the RTI data manager following the interview along with submission of the site visit notes and audio recording.

Introductory Script and Informed Consent:

[Site visitors should pass around sign-in sheet]

I want to remind you that everything you say during this conversation will remain private; it’s very important for this study that you feel free to be candid with me. Of course you can refuse to answer any of my questions for any reason. This discussion will take approximately 1.5 hours, though we can certainly take a break during that time if you’d like.

[Site visitors should note audio taping and provide time for each respondent to read informed consent, complete, and ask questions as needed.]

Do you have any questions before we begin?

Shape30

Overview of Associated Providers Involved with the Project (Addresses EQ 2)

Instructions: The purpose of these questions is to better understand the role of community stakeholders and providers in the local treatment system.

Please describe your agency and its experiences related to each of the following [Ask of each participating agency or if in same agency, each department].
1. What does your agency/department do? (i.e., types of services provided beyond those in the SSH/CABHI/GBHI project)
2. Populations served by your agency/department?
3. Experience with project target population?
4. Geographic area served by agency/department?
5. Experience with SAMHSA grants/programs?
Prior to this grant, what was the history of collaboration between grantee and providers/stakeholders/local funders?
1. Existing service arrangements/linkages?
2. Do you have similar types of relationships with agencies?

Shape31

Relationship between Associated Providers/Key Stakeholders/Local Funders & the Project (Addresses EQ 1, 2)

Instructions: The purpose of these questions is to better understand the role of community stakeholders and providers in the project.

Relationship between grantee and associated providers/key stakeholders/local funders within the context of the project.
1. Please describe your organization’s role in facilitating SSH/CABHI/GBHI project services?
2. What is the nature of this relationship? (e.g., informal, formalized, MOU, contract, etc.)
3. Has your organization’s role changed since the program was first implemented?
In what ways does your organization collaborate with the GBHI/SSH/CABHI program?
1. What mechanisms does the project use for collaboration? (e.g., community consortium, committees, meetings, client case meetings, other info sharing, etc.) and how often are the various types of collaboration used
2. Have there been any challenges to collaboration? (e.g., difficulties scheduling regular meetings, funding stakeholder time, etc.) How have you overcome them?
Is there a Stakeholder Committee/Community Consortium for this project?
1. What agencies are represented on it?
2. What is its role in the program? How often does it meet?
3. Is this new? Pre-existing?
4. What was the grantee’s involvement in these committees/consortiums prior to and following grant funding?
Are all relevant providers/stakeholders involved in the project?
1. Is there anyone/agency/provider who should be working with the project but isn’t?
2. Why should the individual/agency/provider be involved?

Shape32

Associated Providers/Key Stakeholders/Local Funders Perspective on Services & Client Outcomes (Addresses EQ 1, 2)

Instructions: The purpose of these questions is to understand the perspective of community stakeholders and providers regarding services and client outcomes.

Perspective of SSH/CABHI/GBHI project services.
1. What is your understanding of the project goals? Are these goals being met?
  1. Is the project serving the intended population and providing the intended services?
  2. To what degree does the project integrate services and housing?
    1. Is housing a primary goal of the project?
  3. If the project did not exist, would these services be available elsewhere?
2. What are the project’s strengths? In what ways could it be improved?
Client outcomes.
1. From your perspective, how has the project impacted clients? [Can be based on direct observance or through reports from individuals other than SSH/CABHI/GBHI project staff/management]
2. What findings has the program shared with you, if any?
3. Are outcomes used to tailor or improve services, and if so, how?

Shape33

Systems Change (Addresses EQ 4)

Instructions: The purpose of these questions is to better understand barriers and facilitators to implementation and service delivery.

From your perspective, have there been any changes to the environment/service system that has impacted the project? If yes, what was the change and what has its impact been? [Note any areas identified in the PD interview and ask detail]
1. Changes in regulations/policies?
2. Changes in service system?
3. Changes in housing market?
4. Changes in state/local funding streams?
5. Changes in collaborative process and agencies?
6. Innovations?
7. Other changes?
Has the program impacted the community treatment/service system beyond the project itself?
1. Changes to outreach, referral, screening, treatment access in local social service/treatment system?
2. Changes in state/local policies or processes?
3. Other changes?

Shape34

Barriers, Facilitators & Innovations (Addresses EQ 2)

Instructions: The purpose of these questions is to better understand the barriers and facilitators to project implementation and service delivery, as well as any changes or innovations the grant led to among grantee, partners, and the larger treatment system.

Has the project encountered any barriers implementation or service delivery?
Have there been factors that supported implementation or service delivery?
Has the grant led to any changes/innovations in the treatment system? For example, are there new processes, technologies, or tracking approaches?

Shape35

Project Sustainability Activities (Addresses EQ 6)

Instructions: The purpose of these questions is to better understand the role of stakeholders and providers in project sustainability.

Have you been involved with any sustainability planning for the project? If yes, in what ways?
1. Attended meetings specifically focused on sustainability? Frequency?
2. Reviewed materials or evaluation data?
3. Identifying sources of funding for program services? Financing strategies?
4. Serving as program “champions” or seeking out champions for fundraising?
5. What is your opinion on the progress of sustainability planning?
Which program elements do you think are the most important to sustain?
How does the project sustainability plan fit in the community service system?
1. Are there planning efforts related to the local HUD Consolidated Plan or Continuum of Care?
What effect do you think not sustaining this program will have on clients? The community? The local service systems? Your agency? (e.g., won’t be able to provide educational service to clients because that was provided by a partner agency)
What are the most important needs of the <insert target population>?
1. Do you have any policy recommendations for this population?

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this project is 0930-XXXX. Public reporting burden for this collection of information is estimated to average 1 hour per respondent, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

Evaluator Discussion Guide

National Evaluation of SAMHSA’s Homeless Programs

The key informant for this session is the evaluator (internal or external).

Shape36

Note for Site Visitors:

The purpose of this session is to understand the evaluation and quality assurance activities, how the evaluation fits in with sustainability, EBP fidelity, and lessons learned.

Site visitors should prefill measures and other items from the PD Telephone Interview and review and verify any information relative to the evaluation gathered during the Opening Session/Project Director Interview if the evaluator was not present.

If any of the respondents were not present during the previous session, complete consents before the interview commences and

Review consent process
Note audio taping
Have respondents complete sign-in sheets
Collect consents and sign-in sheets before the interview commences
Any additions and/or changes to the Client Flow Chart should be coordinated with the economic site visitor and lead site visitor
Consents, sign-in sheets and staff experience charts should be sent to the RTI data manager following the interview along with submission of the site visit notes and audio recording.

Introductory Script:

[Site visitors should pass around sign-in sheet]

Shape37

I want to remind you that everything you say during this conversation will remain private; it’s very important for this study that you feel free to be candid with me. Of course you can refuse to answer any of my questions for any reason. This discussion will take approximately 1 hour though we can certainly take a break during that time if you’d like.

[Site visitors should note audio taping and provide time for each respondent to read informed consent, complete, and ask questions as needed.]

Do you have any questions before we begin?

Shape38

Evaluation Overview & Integration Into Project (Addresses EQ 2)

Instructions: The purpose of these questions is to better understand the responsibilities of the local evaluator, the design of the evaluation and any current findings.

What is the design of your local evaluation?
1. What is the main focus?
How is the evaluation integrated with the program?
1. Participation in project planning?
2. Management?
3. Clinical meetings?
4. Sustainability planning?
5. Quality assurance?
6. Feedback to clients?

Shape39

Process Evaluation (Addresses EQ 1, 3, 4)

Instructions: The purpose of these questions is to better understand the process data being collected as part of the local evaluation.

What is the focus of the process evaluation?
1. What methods of data collection do you use for the process evaluation? (e.g., focus groups, observations, document review, etc.)
  1. How, by whom, on what schedule?
How is client participation in treatment measured or tracked?
1. Is client attendance at treatment sessions tracked?
2. Is there a tracking mechanism for services referred and services received? If yes, please describe.
3. What is your assessment of client participation in decision making [low, medium, high]?
Do you track housing tenure and type of housing for program clients?
1. If yes, describe how it is tracked (e.g., forms electronically, paper charts, etc.).
2. How is this information used in the evaluation?
3. Is this information used in quality assurance processes?
Is data collected about the project’s sustainability efforts?
1. If yes, what is collected, how is it collected, by whom and on what schedule?
How does the program use data to improve intervention and services or client participation/attendance?

Shape40

GPRA/NOMs Data & Outcome Evaluation (Addresses EQ 3)

Instructions: The purpose of these questions is gain a better understanding of how GPRA/NOMs data is used to inform the local evaluation and what other data collection tools are being used.

How are GPRA/NOMs data collected? [Ask only if evaluator was not present at the Opening Session-PD Interview or not addressed above]
1. Who collects baseline data? When and where?
2. Who collects 6-month data ? When and where?
3. Who collects discharge data ? When and where?
For your program, how are GPRA/NOMs data used locally? What is useful about the GPRA/NOMs data? What are the limitations of the GPRA/NOMs data?
1. Is there anything we should take into account when analyzing the baseline and 6-month follow-up GPRA/NOMs data from your program?
Review with evaluator the GPRA/NOMs Intake, 6-month follow-up, and discharge numbers and rates.
1. How do these rates compare with expectations?
2. If rates do not meet expectations, what steps are being taken to address problems meeting intake and 6-month follow-up rates?
3. If rates do meet expectations for intake and 6-month follow-up rates what is the program doing to achieve this?
What has been your experience collecting the Supplemental Client Interview Data for the cross-site evaluation?
1. Are these data useful addition in your local data collection? Are these data linked to other outcome data?
Are the GPRA/NOMs data linked with other outcome data at the client or consumer level? Are outcome and process data linked or otherwise used together? Please describe.

Beyond GPRA/NOMs data, are other outcome data being collected for the local evaluation?

Additional Measure	Type of Measure 1-Client baseline assessment 2-Self-report client changes measure 3-Client satisfaction 4-Services 5-Partner 6-Staff 7-Cultural Competence 8-Other, specify	When is it administered 1-Baseline 2-6-Month 3-12-Month 4-Discharge 5-Annual 6-Quarterly 7-Other, specify	Who administers/collects this information

Why did you select these measures?
1. How do you plan to use these data?
What are the most meaningful measures for your program or target population (whether you are using this measure or not)? Are there measures other than those listed above that would provide useful or meaningful data?

Shape41

Evaluation Analysis & Reporting (Addresses EQ 3, 4)

Instructions: The purpose of these questions is to better understand the role of the evaluator in the analysis and reporting of outcomes.

What are the evaluator’s role/responsibilities for reporting findings?
1. Regularity of data runs/analysis?
2. Regularity of reporting findings?
3. Role in written reports? (e.g., quarterly reports, annual report, papers, presentations)
4. Anything not mentioned?
With whom does the evaluator meet to discuss evaluation processes and findings? How frequently? [Prompt to understand if the evaluator is integrated into the program, program staff, stakeholders, sustainability activities]
What are your main findings so far?
1. Have your findings resulted in any changes in the program or your evaluation design?
2. Are any of your local evaluation findings different from your GPRA/NOMs findings? Please describe.
3. How have you used local evaluation findings to advocate for the program?

Shape42

Fidelity Assessment (Addresses EQ 1)

Instructions: The purpose of these questions is to better understand the role of the evaluator in EBP fidelity assessment(s). If the evaluator is not involved in the fidelity assessment, this section may be skipped.

Review primary EBPs with evaluator and clarify if fidelity assessment is being used.

Primary EBPs	Using fidelity assessment? Y/N	If yes, what is the fidelity measure(s)? (Note if standardized or created by evaluator, name of the measure, etc.)	How are you using this measure(s)?	Who is completing the measure(s)? (e.g., staff, clients, evaluation, etc.)	When is the measure completed? (e.g., monthly, every six months, etc.)

Is the program using a manualized format of the EBP?
1. If so, have any adaptations been made to the manual?
What is the current status of program implementation/fidelity?

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this project is 0930-XXXX. Public reporting burden for this collection of information is estimated to average 1.5 hours per respondent, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

Client Focus Group Discussion Guide

National Evaluation of SAMHSA’s Homeless Programs

The key informants for this session are program clients (from different stages of the program).

Shape43

Note for Site Visitors:

The purpose of this session is to learn about the SSH/GBHI/CABHI program from the client perspective. The focus group will begin with an introduction that describes the purpose of the focus group, the types of information participants will be asked, how the information will be kept confidential and that participants have the right to not answer any question asked or end their participation at any time. After the introduction is read, and after verbally reviewing the consent form, ask for questions and if in agreement have participants sign the client focus group informed consent.

The questions for the group begin with basic information on the clients participating in the group, such as length of involvement with the program, history of homelessness, and prior participation in services similar to those provided by the SSH/GBHI/CABHI project. The remaining questions focus on the types of services clients received including housing, their satisfaction with these services compared to previous experiences and other services available in the community that are similar.

When on site, find out the structure of the focus group from the Project Director in terms of whether clients should eat before or during the group, whether there are other activities scheduled afterward, etc. Grantee and project staff should not observe or participate in the focus group. Focus group participants should put only their first name and last initial (not their full last name) on the sign-in sheet and consent forms. Once clients gather, introduce site visitors and purpose, hand out consents and pens, give introduction to informed consent (see script below), as well as:

Review consent process, read consent and ask clients to sign with only first name and last initial
Note audio taping (also included in the consent)
Have respondents complete sign-in sheets with only first name and last initial
Collect consents and sign-in sheets
Consents and sign-in sheets will be sent to RTI Data Manager following the interview along with submission of the site visit notes and audio recording.

Shape44

Informed Consent & Sign-in Sheets: Send consent to PD prior to site visit; bring copies to site visit and hand out when clients begin arriving to focus group. Follow script below for introduction and informed consent. Also, read consent, field questions and aid in clients finding signature lines regarding audio taping and general consent. Clients should only sign with first name and last initial—not last name. If sign with last name, black out name before sending to RTI. Collect consents prior to beginning interview and turning on audio tape.

Introduction:

Hi, I am___________ and this is ___________. Thank you for coming to this group discussion about the <GBHI/SSH/CABHI Program>. We are part of a research team from <Organization> that is evaluating programs like <GBHI/SSH/CABHI Program> around the country that are funded by the Substance Abuse and Mental Health Services Administration, part of the federal government.

Privacy Statement: Our purpose in talking with you today is to better understand how you feel about the services you receive through the <GBHI/SSH/CABHI Program>, what you like and what you don’t like, and things you think could be done better. This information will be used to help to understand the types of help programs (like this one) are providing to clients and improve services for future programs.

Before we begin, we want you to know that your participation in this group interview is voluntary and confidential. You do not have to participate in this group discussion if you do not want to. You may choose not to answer any question that is asked.

Statements made in this group are private. Nothing you say in this group will ever be reported in any way that would allow you to be identified. We ask that you use first names only.

_____________ will be taking notes during the session to document everyone’s comments. These notes will not be shared with any program staff. When we report the comments made in this focus group, we will not use your name if we use a quote. Instead, we will just say, “a client described how the program could be improved by ….” Or “clients described their experiences as follows ….” We will also be taping this group to help us remember what you have said and to make sure our notes are correct. Because we are taping, please make sure you never use your last name—first names only so we can protect your privacy. At the end of the study we will erase the tapes.

Again, the purpose of today’s focus group is to find out about your feelings about the <GBHI/SSH/CABHI Program>. There are no right or wrong answers. We want to hear what you think in your own words.

This discussion will last about [an hour to an hour and a half].

Shape45

Do you have any questions?

We have some questions we want to ask you. You will be doing most of the talking. Before we start, I want to mention a few more things. There is no need to raise hands during the discussion. Speak right up, but please respect others when they are talking. When today’s discussion is over, please respect the privacy of your fellow group members. Please do not repeat anything they have said outside this meeting. If you need to go to the bathroom or take a break, please feel free to do so. If you would like to leave the group at any time, or you have an appointment to attend, please feel free to leave.

Now I think that we are almost ready to begin. Before we start, let’s go around the room and share your first name and how long you’ve been involved with the <GBHI/SSH/CABHI Program>.

Shape46

Descriptive Client Information (Addresses EQ 1, 2)

Instructions: The purpose of these questions is to gain a better understanding of client experiences with homelessness and service engagement.

Length of participation in CABHI/GBHI/SSH program?
Prior experience with program like this one? Please describe any past experience you’ve had with mental health services? Substance abuse treatment? Housing services? COD services?

Shape47

Services Clients Receive Through the Program (Addresses EQ 1)

Instructions: The purpose of these questions is to better understand the services clients receive, the degree of choice, rules associated with program involvement, and to identify existing service needs.

How did you hear about the program?
1. Recruitment/referral source – person, agency, location?
2. What types of services were you told you’d receive?
What were you told about the mission or goals of the program?
What types of services/help have you received from the program? Probe for types of treatment, case management and wraparound services.
1. Who provides these services?
2. How frequently do you see/meet with staff?
Can you choose the types of services you want? Do you need to follow a specific program? Can you refuse services?
If you do not participate in treatment- does it affect your housing or the services you may get from the program? Will you be kicked out of your housing if you do not receive services?
Are there particular rules you have to follow to retain program services or housing? (e.g., sobriety, medication compliance, etc.)
How integrated are program services? Are services co-located?
How long can you receive program services?
Were there services you didn’t receive that you were told you would?
1. Why do you think that happened?
2. What was the effect of your not receiving those promised services?
Are there services you needed but did not receive?
Do you have to pay for any of the program services – either out of pocket, through Medicaid benefits or other types of benefits?
How has the program helped you access mainstream benefits? (e.g., SSI/DI, Medicaid, Medicare, DSS, etc.)

Shape48

Housing for Clients (Addresses EQ 1)

Instructions: The purpose of these questions is to gain a better understanding of how the program provides housing, the types of housing available and the rules associated with housing.

In what ways has the program helped you with housing?
1. Locating housing? Completing paperwork?
2. Do you have choice as to the type of housing you get and where it’s located? How did you choose your place?
3. Do you have to pay for the housing? If so, about how much (what percent) of your income? Is it difficult to pay for the housing?
What is the process for obtaining housing?
1. Were there specific requirements you had to meet before you could get housing?
2. Did you have to complete any specific program or services to prove you could be in your own housing?
What type(s) of housing are you in?
1. Is it your own room or apartment?
2. Is it in a building with other people that may have similar problems as yours or is it integrated in the neighborhood?
3. Do you have your own lease?
  1. Who is the landlord?
  2. Do you have rights as a tenant? Are there instances in which you may lose your housing? If so, what are they?
    1. Lease violation?
    2. Destruction of property?
    3. Treatment non-compliance?
    4. Use of alcohol? Drugs?
    5. Can staff enter your apartment at any time?
4. How long can you stay in the housing?
  1. Is the housing permanent or are you expected to move at some point? If so, when?
5. Are there rules you have to follow to keep your housing?
  1. Are you required to stay in treatment to keep your housing?
  2. Are there other requirements?
  3. What do you think of these rules?
What happens with housing when you finish your treatment program?

Shape49

Client Satisfaction with Program Services & Recommendations for Change (Addresses EQ 1, 4)

Instructions: The purpose of these questions is to assess client satisfaction with the program and staff.

How do you feel about the program/treatment services you’ve received?
1. What do you like best about the services?
2. What do you not like about the services?
3. How could the program be improved?
How satisfied are you with your housing? What would you change about your housing if you could? Why?
Have you encountered any difficulties in obtaining or keeping your housing?
How do you feel about the program staff?
1. Are staff respectful to you? Your culture?
2. Do program staff understand your needs and problems?
3. In what ways have program staff been helpful to you? What could they do differently?
Have you seen improvements in your life as a result of your participation in the program? (e.g., help with sobriety, improved emotional/well-being, social relationships, etc.)
1. What has been most helpful about the program? Most useful to you?
2. What is different about your experience with this program compared with other treatment or housing programs you’ve participated in?
Have you encountered any problems/barriers participating in program services? (e.g., transportation, childcare, schedule of services, etc.)
Do you have any suggestions for ways to improve the program or remove barriers for getting housed or staying housed?
What other services are available for clients in the community?
1. Have you been referred/ linked to other services/programs by the GBHI/CABHI/SSH program?
2. If this program did not exist, where else could you find services like the ones you receive from this program?
From your perspective, what should be done at the federal-level around housing, substance abuse, and/or mental health policies? What types of programs/services do you think are the most helpful to <insert program target population>?
Is there anything else you would like us to know about your involvement in the program that I have not asked?

Public Burden Statement: An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB control number for this project is 0930-XXXX. Public reporting burden for this collection of information is estimated to average 2 hours per respondent, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to SAMHSA Reports Clearance Officer, 1 Choke Cherry Road, Room 7-1044, Rockville, Maryland, 20857.

Name of program:

Date:

National Evaluation of SAMHSA’s Homeless Programs

Cost Questionnaire^¹

(Addresses EQ 1, 2, 5)

This questionnaire is designed to collect resource use and cost information for the grantee over two periods: program start-up immediately following the grant award and ongoing costs during the [Fiscal year during which the site visit falls – e.g., FY 2013, 2014, 2015). This questionnaire should be completed by the program director or other designated staff with the assistance of a financial officer and other staff as needed.

Shape50

If you have any questions about the questionnaire, please contact:

Alex Cowell

RTI International

[email protected]

(919) 541-8754

Each section has instructions on how to complete it. To help complete the questionnaire, we have pre-filled it using data from the grant application. Please confirm the pre-filled data is correct and reflects actual expenditures.

Ongoing Costs 3

1. Fiscal year 3

2. Funding 3

3. Total Labor Costs 4

4. Contracted services 7

5. Buildings and facilities 8

6. Depreciation 9

7. Supplies and materials 10

8. Miscellaneous resources and costs 11

9. Administrative overhead 12

Labor Allocation 14

Personnel 14

Direct Services 16

Administrative Support Activities 16

Ongoing Costs

1. Fiscal year

Please complete the following questions for [Fiscal year during which the site visit occurred]. If actual expenditures are not available for the [Fiscal Year] please complete the form using the most current budget information.

Fiscal year:

	to
(month/year)		(month/year)

Are you using a budget to complete the following questions: Yes No

Shape51

Note! Please answer all questions for this grantee for the fiscal year reported above.

If yes, we will contact you to update the report budget items when your fiscal year is complete.

2. Funding

Please report the sources of funding for the grantee.

Source of funding	$ amount of funds
[GRANT PROGRAM]

Other source 1
Other source 2
Other source 3
Other source 4
Other source 5
Other source 6
Other source 7
Total funding

Shape52

Note! The remainder of the questionnaire frequently asks that you report costs at three levels: (1) for the grantee as a whole and for those costs broken into (2) the [Grant Program] versus (3) all other costs.

If you are able to report [Grant Program] costs separately, please also be sure to report costs for the grantee as a whole.

If you are only able to report costs for the grantee as a whole, please simply write N/A where indicated and report only costs for the grantee as a whole.

Reminder! Sections 3 – 10 of the questionnaire frequently ask that you report at three levels: (1) for the grantee as a whole and for the grantee broken into (2) the [Grant Program] versus (3) all other resources.

If you are able to report [Grant Program] costs separately, please also be sure to report costs for the grantee as a whole.

If you are only able to report costs for the grantee as a whole, please simply write N/A where indicated and report only costs for the grantee as a whole.

3. Total Labor Costs

3.1 Please provide the cost of each of the following category of personnel in FISCAL YEAR: (1) paid employees, (2) contracted employees, and (3) any other personnel costs. It is unlikely the grantee will have “other personnel costs.”

Type of personnel	Whole grantee $ personnel cost for FISCAL YEAR	[Grant Program] $ personnel cost for FISCAL YEAR.	[Grant Program]$ personnel cost for FISCAL YEAR. Please write N/A if not available
Paid employees. What was the total labor expense, including all fringe and payroll tax?
Contracted employees. Please include only the cost of paid people, not subcontracts to other organizations.
Other personnel costs (please specify)
Total personnel costs

3.2 What was the fringe rate for full-time employees of the grantee in FISCAL YEAR?

_______________%

3.3. Please indicate which of the following benefits are included in the fringe rate for full-time employees.

Covered benefit in full-time fringe rate	Yes (please place an X)
Health insurance
Pension/retirement
Disability
Vacation
Sick leave
Other (Specify: )

3.4 What were the payroll taxes of full-time employees of the grantee in FISCAL YEAR?

_____________%

3.6 Does the fringe rate for full-time employees also apply to part-time employees? Please place an X by which is correct.

Yes	If yes, please go to Section 4
No

3.7 What was the fringe rate for part-time employees of the grantee in FISCAL YEAR?

_____________%

3.8 Please indicate which of the following benefits are included in the fringe rate for part-time employees.

Covered benefit in full-time fringe rate	Yes (please place an X)
Health insurance
Pension/retirement
Disability
Vacation
Sick leave
Other (Specify: )

3.9 What were the payroll taxes of part-time employees of the grantee in FISCAL YEAR?

___________%

4. Contracted services

If the grantee had a contract with a company/corporation to provide a service to support [Grant Program], then enter that information in the answer to the question below. Please do not include here information for contracts with individual people. That information should be included the Labor Allocation section.

Shape54

Example. If the grantee had a contract in FISCAL YEAR with a contracted cleaning service, then include the cost to the program for these services here in Section 5.

However, if you have a contract with someone to perform psychological evaluations at the program, then include the cost of her/his services in the Labor Allocation section.

4.1 What was the total cost of contracted services in FISCAL YEAR for the grantee as a whole? (Contracted services may include legal services provided to the grantee, repair and maintenance, security, housekeeping, pest control.) Please do not include any staff contracted to provide services to patients; these staff will be included in the Labor Allocation.

Contracted Service	Contracted services cost for grantee as a whole for FISCAL YEAR	Contracted service cost for [Grantee Program] for FISCAL YEAR
Medical
Pharmacy
Laboratory
Legal
Accounting
Security
Computer
Advertising
Repair and Maintenance
Pest Control
Housekeeping
Other:

Shape55

Reminder! Sections 3 – 10 of the questionnaire frequently ask that you report at three levels: (1) for the grantee as a whole and for the grantee broken into (2) the [Grant Program] grant versus (3) all other resources.

If you are able to report [Grant Program] costs separately, please also be sure to report costs for the grantee as a whole.

If you are only able to report costs for the grantee as a whole, please simply write N/A where indicated and report only costs for the grantee as a whole.

5. Buildings and facilities

This section collects information on the value of the building space used by the grantee during FISCAL YEAR.

5.1. What were the grantee total expenditures (e.g., rent, lease, or mortgage payments) for the space used by the grantee during FISCAL YEAR? If the building space was jointly used with another entity, please prorate the amount to reflect the portion of space costs incurred by the grantee only.

Building space cost	Whole grantee building space $ for FISCAL YEAR	[GRANT PROGRAM] building space $ for FISCAL YEAR. Please write N/A if not available	Non-[GRANT PROGRAM] building space $ for FISCAL YEAR. Please write N/A if not available

5.2. Do the expenditures for the space used by the grantee accurately reflect the current market value of the space? Please place an X by which is correct

Yes	(go to 7.1)
No (space is free or below market rate)

5.3. How large is the space in all the buildings used by the grantee during FISCAL YEAR? If building space was jointly used with another grantee, please prorate the amount of space to reflect the portion of the total space used by the grantee only.

Space	Whole grantee space (square feet) for FISCAL YEAR	[GRANT PROGRAM] space (square feet) for FISCAL YEAR. Please write N/A if not available	Non-[GRANT PROGRAM] space (square feet) for FISCAL YEAR. Please write N/A if not available

5.4. What would you estimate total expenditures on space would have been in FISCAL YEAR if fair market value had been paid for the space? If you are unable to estimate the fair market value for the space, we may contact a local real estate agent/company who may be able to help estimate the market value.

Estimated market value	Whole grantee building space estimated market value for FISCAL YEAR	[GRANT PROGRAM] building space estimated market value for FISCAL YEAR. Please write N/A if not available	Non-[GRANT PROGRAM] building space estimated market value for FISCAL YEAR. Please write N/A if not available

6. Depreciation

6.1. Please list the total cost of depreciation for the grantee in FISCAL YEAR. Include depreciation for vehicles, furniture, equipment, security systems, and computers.

Depreciation	Whole grantee $ depreciation for FISCAL YEAR	[GRANT PROGRAM] $ depreciation for FISCAL YEAR. Please write N/A if not available	Non-[GRANT PROGRAM] $ depreciation for FISCAL YEAR. Please write N/A if not available

7. Supplies and materials

7.1. What was the total cost of for supplies and materials used by the grantee in FISCAL YEAR? Please include the value of donated or subsidized (in-kind) supplies and materials.

Shape56

Note! In sections 8 and 9 the key data are the total values (e.g. total supplies and materials) and not the individual line items (e.g. office supplies). Please do not feel obliged to enter all the line item values in sections 8 and 9; they are only there to help you provide the total.

Supplies and materials costs. Itemization is only for your convenience. ‘Total supplies and materials’ are the only data that will be used.	Whole grantee $ supplies & materials	[GRANT PROGRAM] $ supplies & materials for FISCAL YEAR. Please write N/A if not available	Non-[GRANT PROGRAM] $ supplies & materials for FISCAL YEAR. Please write N/A if not available
Food
Transportation
Office Supplies
Housekeeping
Other supplies
Total supplies and materials

8. Miscellaneous resources and costs

8.1. What was the cost of other miscellaneous items used by the grantee in FISCAL YEAR? Please include the value of donated or subsidized (in-kind) resources not previously covered in this questionnaire.

Misc. resources and costs. Itemization is only for your convenience. ‘Total miscellaneous resources’ are the only data that will be used.	Whole grantee $ misc. resources and costs	[GRANT PROGRAM] $ misc. resources and costs for FISCAL YEAR. Please write N/A if not available	Non-[GRANT PROGRAM] $ misc. resources and costs for FISCAL YEAR. Please write N/A if not available
Utilities (e.g., electricity, gas, oil, water and sewer, garbage)
Insurance (e.g., liability, malpractice, director and officers)
Non-Payroll Taxes (e.g., federal, state, local)
Communications (e.g., telephone, postage, printing & duplicating, advertising, publications)
Dues, memberships, and fees
Staff training
Staff traveling
Any other costs not yet accounted for in this questionnaire
Total miscellaneous resources and costs

9. Administrative overhead

This section collects information on an administrative overhead rate that may have been applied to the grantees’ grants, contracts, and other sources of funds, such as federal grants. Usually, overhead rates are used to pay for administrative services that support activities not paid for directly. Examples include marketing, outreach, business office, and billing.

10.1. Is there an overhead rate or administrative charge that is incurred by the grantee?

Yes

 If no, thank you for your participation.

10.2. Have you included this overhead rate/administrative charge in the cost information you have already provided in this questionnaire (under Sections 3 through 9)?

Yes

 If yes, thank you for your participation.

10.3. What is the overhead rate (or administrative charge)?

Overhead Rate: ________% or Administrative Charge: $__________

Administrative overhead	Whole grantee administrative overhead rate (%) or charge ($) for FISCAL YEAR	[GRANT PROGRAM] administrative overhead rate (%) or charge ($) for FISCAL YEAR. Please write N/A if not available	Non-[GRANT PROGRAM] administrative overhead rate (%) or charge ($) for FISCAL YEAR. Please write N/A if not available

10.4. To which costs does this overhead rate (or administrative charge) apply?

Type of cost	Yes (please place an X)	No (please place an X)
Labor
Total
Other (please specify)

Labor Allocation

Personnel

1. How many staff, including in-kind and contracted staff, work on your [Grant Program] ____

2. List of employees and salary. Please list below all of your agencies’ staff that work on your [Grant Program]. Please indicate their job title, percent FTE at the grantee as a whole, and percent FTE devoted to [GRANT PROGRAM] (which will be the same as the percent at the grantee if the employee works exclusively on your [Grant Program]). Please list your entire agency’s staff who work on your [Grant Program], include in-kind (i.e., not paid by Grant program funds) and contracted staff.

Job Title/Role on Program	Staff Type	In-kind or Contracted	Estimated hourly wage of employee	Part-Time	Full-Time	% FTE on [GRANT PROGRAM]

3. For each key staff type listed in Question 2 record the total hours worked in a typical week.

Staff Type	Number of Staff	Total Hours in a typical Week
Staff type 1
Staff type 2
Staff type 3
Staff type 4
Staff type 5
Staff type 6

The totals hours listed above are now proportioned across the key activities each staff type performs in a typical week. Staff activities are divided into two broad categories: Direct Services and Administrative Support Activities.

Direct Services

Direct Services include all services that are provided face-to-face with a patient and/or client and typically include services such as counseling and case management.

4. For each staff type listed in Question 3, how many hours in a typical week are spent on the following:

Services	Hours in a typical week
Assessment
Outreach
Non-clinical Case Management
Clinical Case Management
Co-Occurring Treatment
Housing Support Activities
Inpatient Mental Health Treatment
Inpatient Substance Abuse Treatment
Medical Treatment
Testing
Outpatient Mental Health Group Counseling
Outpatient Mental Health Individual Counseling
Outpatient Substance Abuse Group Counseling
Outpatient Substance Abuse Individual Counseling
Pharmacotherapy
SBI
Trauma Services
Recovery Support Services

Administrative Support Activities

Administrative Support Activities include all activities which do not directly involve a patient and/or client such as preparing for a counseling session, staff meetings and completing paper work.

5. For each staff type listed in Question 3, how many hours in a typical week are spent on the following:

Activities	Hours in a typical week
Program Administration
Required data collection and reporting (GPRA, NOMS)
Staff Training
Collaboration\Partner Development and Outreach
Other:

Partner Funding

National Evaluation of SAMHSA’s Homeless Programs

Services & Housing Funding Discussion Guide

The following questions review funding sources for the types of services and housing that may be provided for clients in your [Grant Program]. We have pre-filled some of the responses based on a review of documents from and previous discussions with your program.

1. What are the funding sources for the following types of services?

Assessment

Outreach

Non-clinical Case Management

Clinical Case Management

Co-Occurring Treatment

Housing Support Activities

Inpatient Mental Health Treatment

Inpatient Substance Abuse Treatment

Medical Treatment

Testing

Outpatient Mental Health Group Counseling

Outpatient Mental Health Individual Counseling

Outpatient Substance Abuse Group Counseling

Outpatient Substance Abuse Individual Counseling

Pharmacotherapy

Screening, Brief Intervention (SBI)

Trauma Services

Recovery Support Services

Probe for the following funding sources: SAPT Block Grant, Medicaid, State Welfare (General Assistance), Private Agency Funds , Private Grants, State Grants, SAMHSA Grants Other Federal Grants, HUD Services Contract/Grants, VA Service Contracts, City/County Grants, GBHI, SSH, CABHI, PATH funding.

Interviewer: All responses should be matched to one of funding categories listed. The responses should be recorded in the Leveraging data sheet. For each identified combination of funding source and activity/service, ask the respondent what proportion of clients that used the combination in the past 3 months.

1a. For [Service A] what is the percentage of clients that typically receive/utilize the service?

Prompt: 0-25%, 26-50%, 51 – 75%, or 76 – 100%

1aa. For [Service A] what proportion of clients that received the service through [Funding Source A] in the past three months?

Prompt: 0-25%, 26-50%, 51 – 75%, or 76 – 100%

2. What are the funding sources for each of the following types of housing?

Emergency shelter

Shelter plus Care

Transitional housing

Sober transitional housing

Permanent housing

Sober permanent housing

Probe for the following funding sources:

HUD Section 8, HUD Disability, HUD Women and Children, HUD Transitional, Private Funding, Agency Ownership, Tax Investment Credits, Local State/County/City Housing Authority Funding, Other Federal funding Other City/County

Interviewer: All responses should be matched to one of the funding categories listed. The responses should be recorded in the Leveraging data sheet. For each identified combination of funding source and housing type, ask the respondent what proportion of clients used the combination in the past three months.

2a. For [Housing Type A] what is the percentage of clients that typically receive/utilize that housing type?

Prompt: 0-25%, 26-50%, 51 – 75%, or 76 – 100%

2aa. For [Housing Type A] what is the percentage of clients that typically receive that housing type [Funding Source A]?

Prompt: 0-25%, 26-50%, 51 – 75%, or 76 – 100%

Shape57

The Cost Questionnaire is complete. Thank you for participating!

1 This questionnaire is adapted from the Substance Abuse Services Cost Analysis Program (SASCAP™; trademark RTI International) cost and labor module. The original version of the SASCAP™ questionnaire was funded in part under a contract with the Substance Abuse and Mental Health Services Administration (SAMHSA) (contract no. 270-97-7002).

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Stephen Orme
File Modified	0000-00-00
File Created	2021-01-23

Form Discussion Guide_C Discussion Guide_C Discussion Guide_Cost Questionnaire

Evaluation of Programs to Provide Services to Persons Who Are Homeless with Mental and /or Substance Use Disorders

Attachment 02_SV Discussion Guide_Cost Questionnaire

Project Director Interview

OMB: 0930-0339

Table of Contents

Ongoing Costs

1. Fiscal year

2. Funding

3. Total Labor Costs

4. Contracted services

5. Buildings and facilities

6. Depreciation

7. Supplies and materials

8. Miscellaneous resources and costs

9. Administrative overhead

Labor Allocation

Personnel

Direct Services

Administrative Support Activities

Partner Funding