Attachment E – Interview Guide for Grantee and Sub-Grantee Site Visits
OMB Control No: ____-____
Expiration Date: __/__/____
Interview Guide for Grantee and Sub-grantee Site Visits for the SUPPORT Act Grants Evaluation
This document covers the key topics to be covered in individual interviews with program managers, staff, and partners at grantees and subgrantees implementing the Substance Use-Disorder Prevention that Promotes Opioid Recovery and Treatment for Patients and Communities (SUPPORT) Act grant programs. Each topic indicates which staff will respond (i.e., program managers, program staff, and partners). The purpose of these interviews is to understand implementation of this new grant program to serve individuals and communities with SUD. Interviews will take place at each of the four grantees and eight subgrantees selected for the implementation study. These interviews will take place once and will be supplemented with a Data Collection Planning Interview and a Final Reflection Interview. A table at the end of the document summarizes interview topics by type of respondent (e.g., managers, staff, and partners).
Introductory statement for staff: The evaluation of the Department of Labor’s (DOL) SUPPORT Act grant program is being conducted by Abt Associates and MDRC under contract to DOL’s Chief Evaluation Office, in partnership with the Employment and Training Administration. The SUPPORT Act provides grant resources to local workforce development boards to address the economic and workforce impacts associated with the high rates of opioid use disorder (OUD) or substance use disorder (SUD) in Florida, Maryland, Ohio, and Wisconsin. The evaluation aims to provide new and critical information on promising practices and implementation challenges in providing services that address both employment and treatment needs for those with OUDs/SUDs. The goal of the evaluation is to document best practices, challenges, and lessons for both policymakers and program administrators. In conducting site visits to each of the programs, we are talking to project directors and staff, as well as partner organizations. We are here to learn about the service delivery of your program model and understand how it operates. Our aim is to learn from your experiences, not audit or rate your programs. This interview will take up to 90 minutes.
Privacy Statement: Before beginning the interview, I (we) want to thank you for agreeing to participate in the evaluation of the SUPPORT Act grant program. I (we) know that you are busy and we will try to be as focused as possible. We have many questions and are going to talk to many different people, so please do not feel as though we expect you to be able to answer every question. And, we understand that your participation in this discussion is voluntary and you may choose to not answer some questions.
Information collected will be kept private to the extent permitted by law. The views you express will be kept private, and nothing we publish in this evaluation will identify you by name. Though we take notes during these interviews, information is never repeated with the name of the respondent. When we write our reports and discuss our findings, information from all interviews is compiled and presented so that no one person can be identified. We also ask that you refrain from sharing anything we discuss today with others to help us ensure your privacy and the privacy of others we are interviewing. Do you have any questions before we begin?
The Paperwork Reduction Act Statement: This collection of information is voluntary and will be used to understand programs that integrate employment and substance use disorder services. Public reporting burden for this collection of information is estimated to average 90 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: XXXX-XXXX, Exp: XX/XX/XXXX. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to Hannah Betesh (Abt Associates); [email protected].
Respondent Information
Name, title, organization/affiliation, length of involvement with the organization and with the program
Educational background and prior work experience
Overall role/job responsibilities
Program Context and Environment—Program Managers and Partners
Organization background—Program Managers and Partners
[Site visitors: Pre-populate with information collected through grant applications, materials sent ahead of time (such as an org chart), knowledge development interviews, organizations’ websites, and online survey. Confirm and/or obtain any missing information.]
Other programs and services offered by organization/agency, including prior experience offering key grant services
Organization’s major sources of funding
Organization’s total number of paid staff, including number of staff and percent FTE, as well as which of these staff are funded through the grant.
Types of individuals served or targeted, including prior experience serving grant target populations
Annual number of participants served overall
Local substance use disorder conditions—Program Managers and Partners
Trends in of substance use disorder, including types of substances, in areas served by program (e.g., arrests, hospitalizations, overdose, etc.) [Site visitors: pre-populate from secondary sources, such as SAMSHSA, CDC prior to visit]
Treatment and recovery service landscape, including availability and accessibility of services
Effects of COVID-19 on substance use, treatment and recovery services, such as use of telemedicine, changes in capacity, enrollment fluctuations in residential treatment, and vaccine mandates and their impacts on staffing and service access.
Prior grants and initiatives in area to address high rates of SUD/OUD in area
History of inter-agency efforts to address high rates of SUD/OUD in area
Local economic conditions—Program Managers and Partners
Factors affecting the local labor market:
Major industries and employers
Recent economic trends (e.g., occupations experiencing earnings and job growth)
Unemployment levels, job availability, wage levels
Other factors that affect economy (e.g., natural disaster, companies coming or going)
Factors affecting target population(s)’ success in finding and keeping jobs
Availability of jobs at their skill level and desired wages
Availability of jobs suitable for people in recovery
Availability of employers providing support to people in recovery/“Recovery Friendly Workplaces”
Policy context, such as fair chance hiring and occupational licensing restrictions
Effect of the COVID-19 pandemic on workforce system service delivery
Experience and challenges with virtual service delivery
Experience and challenges with other service delivery changes (vaccine mandates, indoor mask mandates, other changes to be determined).
Extent to which changes introduced to adapt to COVID-19 are being/ will be maintained.
Effects of COVID-19 on local labor market:
Effects on labor supply stemming from vaccine mandates, school closures, etc.
Effects of stimulus payments, Child Tax Credit, and Pandemic Unemployment Assistance on workers and labor supply
Effects of business closures or capacity restrictions due to surges and new variants
Other effects of the pandemic on the local labor market
Program Goals and Structure—Program Managers and Partners
Grant background and goals—Program Managers
[Site visitors: Pre-populate with information collected through grant applications, knowledge development interviews, organizations’ websites, and web-based survey. Confirm and/or obtain any missing information.]
Reasons for applying for the grant and interest in developing strategies to integrate SUD treatment and employment services
Process for selecting subgrantees
Program service area
Type of service area (portion of a city/county, single county, multiple counties/region)
Size of service area (in terms of geography and population)
Population density (urban, suburban, rural, mixed)
Demographics of the service area (education levels, race and ethnicity, income levels, poverty status, immigration)
Changes in demographics over time
Goals of grant, at application and changes over time (including due to COVID-19)
Resources and capacity—Program Managers and Partners
Annual budget and funding sources, including resources beyond grant (e.g., Medicaid, SAMHSA grants, resources from other workforce system programs)
Other resources required to operate the program such as facilities, technology, and treatment/recovery services
Stability of funding
Sustainability and prospects for future funding
Management and staffing—Program Managers
Grantee and subgrantee roles and coordination
Management structure
How program is situated/managed within the larger organization or institution
Who has authority over the program
Number of grant-funded staff and positions [Site visitors: pre-populate with information collected through the web-based survey]
Titles
Required experience and qualifications
Primary responsibilities
Number of FTEs by position
Location (at AJC or at partner organizations)
Staffing needs
Qualifications
Use of staff with lived experience
Staff training needed specifically for the grant
Staff turnover and shortages (generally and due to COVID-19)
Partnerships and organizational linkages--Program Managers and Partners
[Site visitors: pre-populate with information collected about partnerships through the web-based survey]
Partnership history and evolution
How and why partnership was developed
Length of the partnership
For pre-existing relationships, how focus of grant changed the partnership, if at all
Changes in partnership over grant period
How focus of grant changed the partnership, if at all
Relationships with other organizations like sub-grantee [ask partners only]
Partnership coordination:
Staff responsible for coordination
Schedule and format for coordination meetings
Process and frequency of data exchange between partners
Partnership challenges and how they have been addressed
Challenges related to differences in mission and culture between organizations/systems
Challenges related to employer needs and context
Challenges related to maintaining partner engagement over time
Challenges related to grant performance measures
Benefits of partnerships
Recruitment and enrollment—Program Managers and Staff
Outreach and recruitment—Program Managers and Staff
Target population, including focus on Type I vs Type II participants, successes and challenges around recruiting target population
Outreach and recruitment strategies (e.g., website, social media, word-of-mouth, fliers, referrals, community events, other agencies/programs)
Effectiveness of strategies
Most common way participants are referred to the program
Differences in strategies for different target populations
Variation in participant characteristics based on referral source – justice involvement, employment history, readiness for work.
Eligibility determination and enrollment—Program Managers and Staff
How individuals apply to the program (including any changes to planned procedures due to COVID-19)
Application steps and information collected (in what order and by whom)
Eligibility requirements, including differences for different grant populations
Use of assessments, including what assessments and how they are used
Length of application process
Program enrollment
Initial services after enrollment
Program services—Program Managers and Partners
Description of services provided—Program Managers, Staff, and Partners
[Site visitors: pre-populate with information collected about services provided through the web-based survey]
Career services
Employment and Treatment Plans
Job readiness skill development (i.e., “soft” skills)
Identifying barriers
Job coaching, matching, placement
Provision of payments and fees for employment and training related applications, tests, certifications
Occupational training services
Peer Recovery Specialist (PRS) training
State context for Peer Recovery Specialist training
Ability to bill Medicaid for Peer Recovery Specialists services
Career pathway for Peer Recovery Specialists
Availability of Peer Recovery Specialist training programs on Eligible Training Provider List
Training details
Eligibility requirements (length of sobriety, background checks, minimum education)
Classroom and practicum hours required
Barriers to completion of classroom training requirements and how they were addressed
Barriers to completion of practicum requirements and how they were addressed
Strategies for integration of peers in clinical placements/with clinical staff
Application, exam, and credentialing process
Training challenges and successes
Training for incumbent workers on serving people with SUD
Industries/occupations of incumbent workers
Engagement of employer in development of incumbent worker training
Training content, length, and format
Differences between this training and what might otherwise be offered in the workforce system
Career pathways in industry/occupation and how addressed in training
Training challenges and successes
Other occupational training
Industries/occupations targeted
How and why industries/credentials were selected
Training content, length, and format
Differences between this training and what might otherwise be offered in the workforce system
Career pathways in targeted occupations
Role of employers in developing occupational training curriculum
Training challenges and successes
Work-based training services, including subsidized employment or on-the job training OJTs), paid internships, and unpaid work experience
Industries/occupations targeted
Work-based training schedule, length, and location
Differences between work-based training under this grant and what might otherwise be offered in the workforce system
Role of employers in developing work-based training opportunities
Work-based training challenges and successes
Employment services
Job search and placement assistance
Job development services
Job retention services
Employer engagement and training
Targeted training for Human Resources, mentors, and others
Engagement strategies for involving employers in the grant
How activities and engagement vary depending on employers’ level of familiarity with and openness to hiring people with SUD
Case management and supportive services
Frequency of case management meetings
Case management approach
Use of goal-setting models
Use of trauma-informed models
Tailoring of case management models for people in recovery from SUD
Types of supportive services available and how these differ under the grant
Most common supportive service needs (and how these differ from typical workforce system population needs)
Referrals to other organizations and services, including community-based and government programs, for supportive services
Most common referral and extent of coordination with agency referred to
Local policies and initiatives to align services, reduce enrollment burden on participants, or facilitate referrals
Treatment and recovery services for SUD/OUD and mental health
Development and use of individual treatment plan, including which partners are responsible for the plan
Medication for opioid use disorder (MOUD)
Outpatient treatment
Therapy/counseling
Monitoring and support for adherence to treatment
Typical length and intensity of each treatment and recovery service component
Follow up services provided to participants once they have left the program
Mode of follow-up services
Frequency of follow-up contact
Follow up data collected on participants
Service delivery structure—Program Managers, Staff, and Partners
For each service listed above:
Criteria and process for determining which services participants receive, including participant choice, grant/organizational requirements, and staff discretion
Required services/services provided to all participants
Services provided on an as-needed basis
Service sequencing and duration
Service provider and location
Staff involved in service delivery (including co-location of staff)
Organizational partners in delivering service
Participant characteristics and participation in services—Staff
Differences in participant characteristics between grant target populations
Trends or changes in demographics over grant period
Differences between participants and other workforce system customers
Participation in services
Utilization of different services and why
Persistence in services and how long people participate
Patterns in utilization or persistence by target population or other characteristics
Challenges with meeting training program requirements, such as attendance.
Barriers to participants’ success in the program; extent to which SUDs are an issue
Strategies to mitigate barriers
Participant strengths and facilitators of success in the program
Engagement and reengagement strategies
Program performance and outcomes—Program Managers
Successes and challenges meeting grant outcomes and reasons why
Enrollment in job training and services
Receipt of supportive services, treatment, and recovery services
Completion of job training and services
Entry into employment related to training and services
Employment retention
Perceptions on appropriateness of designated performance indicators for grant and why
Suggested performance indicators
Operational Challenges and Successes—Program Managers, Staff and Partners
Challenges the program encountered and how they were overcome, including challenges related to combining treatment and employment services and forming relevant partnerships
Promising approaches developed by program, including approaches to combining treatment/employment services and forming relevant partnerships
Areas for improvement; plans for changes and modifications
[Grantees] Plans to scale up, replicate in other locations across the state
The
following table provides a summary of interview topics and potential
respondents for each.
Summary of Interview Topics and Respondent Type
Topic |
Respondent Type |
||
Program Managers |
Program Staff |
Partners |
|
Respondent Information
|
X |
X |
X |
Project context and environment
|
X |
|
X |
Program goals and structure
organizational linkages |
X
|
|
X |
Recruitment and enrollment
|
X
|
|
X |
Program services
|
X |
X |
X |
Participant characteristics and participation in services
|
|
X |
|
Program performance and outcomes
|
X |
|
|
Operational Challenges and Successes
|
X |
X |
X |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2023-08-28 |