0. Respondent Background Questions (All interviews) 1
1. Service Delivery and Community Context 1
B. Opportunities for Justice-involved Individuals 2
C. Policies Regarding Release and Supervision of Justice-involved Individuals 2
2. Overview of the Grantee Organization 3
B. REO Program Budget and Funding 5
C. Program Staffing and Administrative Structure 6
6. Recruitment, Eligibility, and Enrollment 9
A. Target Population/Participant Characteristics 9
C. Eligibility Determination 10
7. Case Management and Service Planning 12
A. Case Management Services 12
B. Occupational Skills Training 14
D. Work-Based Learning Opportunities 16
10. Ancillary Services, Exit, and Follow-Up 19
A. Work Readiness and Life Skills 19
E. Incentives to Participants 22
F. Exit and Follow up Services 22
11. Alternative Programs and Services 23
A. Summary of Programs Providing Similar Services to REO 23
B. Description of the Services Provided by Alternative Programs 24
C. Coordination between REO and Alternative Programs 24
13. Overall Successes, Challenges, and Lessons Learned 26
Let me start by saying thank you for taking the time to meet with me today.
My name is _________________ and I work for Mathematica/Social Policy Research Associates. As you may have heard, I’m part of a research team that is evaluating programs around the country being funded by the U.S. Department of Labor to provide education, training, and employment services under the Reentry Employment Opportunities (REO) grant. The purpose of the evaluation is to better understand how REO funded services are provided and to identify potential promising practices.
My goal today is to learn about your [2017, 2018 and 2019] REO funded project(s) known here as the [fill in name of program]. Specifically, I’d like to learn more about the types of services this program is providing to participants, the ways in which partnerships work, and the larger community context in which participants live and work. I’m also here to learn about what has worked well in implementing this grant, what challenges you have faced, and how you’ve addressed those challenges. The research team will use the information collected today to help develop our understanding of what services participants in the evaluation receive and to help build our understanding of REO programs beyond what we can learn from administrative and survey data. We also hope to identify promising practices and insights into operating a program such as this one that we can share with practitioners and policymakers alike.
The study team will use this information to develop a report for DOL. While we may use quotes and draw information from this conversation, we will pool this information together with similar conversations we are having with programs and partner staff around the country and will not identify anyone here by name. Any quotes we use will be anonymous.
Do you have any questions before I get started? Do we have your consent to proceed with the discussion?
We have about [one hour/one and a half hours] for our discussion. Do I have your permission to record this conversation for my notes?
Let’s get started with few background questions. Throughout, when I ask about the REO-funded program, I’m asking about the program funded by the [2017, 2018 and 2019] Reentry Employment Opportunities grant, known here as [fill in name of program].
What is your role in this REO-funded program?
How long have you been with your organization? With this REO-funded program?
How long have you been in your current position?
Do you have other roles in your organization? If so, what are they? What portion of your time is spent on REO?
Now, I’d like to ask you a few questions about the local economy and the reentry context. [site visitors: record the current unemployment rate in the area ______ prior to the visit and note the timeframe for the grant(s) in question relative to COVID-19.]
Let’s talk first about the local economy.
Who are the major employers and industries in the area?
Is it difficult for local employers to find qualified employees? If so, why?
What type of skills (specific occupational skills, soft skills, etc.) are most valued by local employers?
How did the local economy change due to the COVID-19 pandemic?
What other changes (i.e., non-pandemic or recession-related), if any, have there been in the local economy since you got your REO grant(s) that have affected employment?
What would you say are the most pressing needs facing justice-involved individuals in your community: finding employment, housing, mental health/substance abuse treatment, or something else?
What do you feel are the biggest barriers to finding stable employment in this community for justice-involved individuals?
What are employer’s concerns around hiring justice-involved individuals?
Please describe the major employers of justice-involved individuals in your area.
Who are these employers?
Which industries or occupations, if any, are more open to hiring justice-involved individuals?
Are there any exemplary companies or hiring programs by employers that stand out in your community? What makes them distinctive?
Are there state or local policies, such as occupational licensing requirements or fair chance ordinances, that affect employment opportunities for justice-involved individuals? If so, please describe them.
Tell me more about how your local WIOA program serves justice-involved individuals.
What percentage of the participants in the WIOA program are justice-involved? Does the local WIOA program target this group specifically?
How are justice-system involved individuals different, if at all, from other WIOA program participants?
Has the local WIOA service provider ever had programs specifically for justice-involved individuals? If so, please describe.
What community characteristics have most supported the implementation of programs serving justice-involved individuals (i.e., the nature of employment, partners, support from community leaders, etc.)?
What community characteristics have been most challenging when it comes to implementing services for justice-involved individuals? Later, we will discuss other services in your community available to individuals with justice system involvement.
I would now like to ask you about the justice-involved population and the release and supervision processes in your area.
What are the names of the justice system partners that refer participants to the REO program?
What types of partners are they (e.g., courts, prisons, parole/probation offices?
Where are they located?
Are there other partners that do not refer participants to REO? If so, who are they and why don’t they?
Please tell me the following about the typical release process for the justice-involved population served by the REO program.
Approximately what percentage of justice-involved individuals are released directly into the community as compared to a half-way or transitional facility? Can only the former or both groups be served by REO?
What requirements do individuals on probation or parole have to meet around job search and placement in your area?
Have there been any significant changes around release and (parole/probation) supervision during the grant period? If so, please describe.
What changes to release practices were made due to COVID-19 during the grant period?
Is there a large demand for programs like REO among the local justice-involved population? If so, why? If not, why not?
Is there a large demand for programs like REO among justice system partners such as the courts, prisons, jails, or probation and parole? If so, please describe. If not, why not?
How well do you think the REO eligibility criteria align with the local reentry population? [Site visitors: see section 6.C.1.a. for definitions of these criteria]
Please tell me about your organization and its history, including its goals, services, and service locations.
Why did you apply for the REO grant? [For intermediaries] Why did you apply as an intermediary?
How did you determine the areas of training and other services to provide?
How does the REO program fit into the mission/goals of the organization?
Have your organization’s goals changed since beginning REO? If so, how?
What other programs have you operated or do you currently operate for justice-involved individuals? For each one, please indicate the target population, funding source and dates it was active.
How has your experience operating these other programs influenced what you are doing with your REO-funded program?
I’d now like to ask you some questions about the interactions between intermediary and subgrantee organizations in the operation of your REO grant. [Site visitors: if working with a CBO grantee, omit the following questions]
[For intermediaries] Do you have local affiliate organizations that are not part of the REO grant? If so, how many?
What were the key criteria in selecting subgrantees for REO?
How much did prior experience with this grant or working with similar programs or populations matter?
What selection process, if any, did subgrantees undergo?
How are REO responsibilities divided across subgrantees? To what extent were subgrantees involved with any planning?
How did you decide on enrollment goals?
[For intermediaries] How did you divide funding across subgrantees?
What was the extent of any guidance given by intermediaries about program services, populations to target, and trainings/sectors of focus?
[For intermediaries] To what extent do subgrantees implement a uniform service model?
How do subgrantees differ? (Services provided? Target population? Training offerings?)
How do you coordinate with subgrantees to ensure consistency in implementation?
What type of guidance and support do intermediaries provide to subgrantees? To what extent and how frequently do each of the following types of support occur?
Technical assistance calls/visits?
Grant monitoring calls/visits?
Guidance documents?
In-person convenings for subgrantees to share and learn from each other?
Other?
How are updates or information from DOL communicated to subgrantees?
How do data and reporting responsibilities work across subgrantees?
Is there a shared data system? If so, please describe.
[For intermediaries] How do you assess data quality and address related issues across subgrantees?
[For intermediaries] How is performance measured and shared across the subgrantees?
Do subgrantees all have the same or different performance goals/targets? If different, what are the differences?
Does subgrantee performance result in adjustments to their enrollment goals or funding levels? If so, how?
[For intermediaries] What has worked best so far in coordinating work across your subgrantees?
[For intermediaries] What have been the biggest challenges so far in coordinating work across subgrantees?
Which subgrantees have struggled most in implementing the program? Why?
What approaches have worked best in addressing subgrantee challenges?
Next, I’d like to talk about the grant award and the program budget.
Have you requested any modifications to the grant since it was awarded? If so, what were they and were your requests approved?
Which individuals and organizations make up the core team that is responsible for overseeing the grant?
[For intermediaries?] Have there been any changes in the number of subgrantees? If so, which subgrantees have been added or removed and why?
Have there been any changes in the enrollment targets? If so, what adjustments have been made and why?
What was your total, annual organization budget across all funding sources for the year(s) your REO grant(s) was awarded?
How does this amount compare to other years? Explain any changes.
Do you receive other funding for your REO-funded program in addition to the DOL grant?
If so, who provides these funds? What is the level of funding from each?
Are these funds earmarked for particular types of participants, services, activities, or materials?
What additional funding does your organization receive for other programs and services that REO participants may access or use?
Who provides these funds? How much are they? Are these funds earmarked for particular services, activities, or materials?
Let’s talk about the staffing of this REO grant.
Which staff are involved in day-to-day operations? How are frontline staff supervised?
[Site visitors: Please work with the coordinator to fill out the below table ahead of the visit and confirm this information on site] Please confirm the list of staff member who work on this program, including their name, role (e.g., program coordinator, case manager), full time equivalency at the organization and on the REO-funded program, their time at the organization and whether they are funded through the REO grant or work on the program through other funding.
Name |
Role |
FTE at Org |
FTE on REO |
Time at Org |
Funded by REO grant? |
Jessica Jones |
Case Manager |
.8 |
.5 |
2 years |
Y |
|
|
|
|
|
|
Did you fill these roles with existing staff or did you hire them for REO?
What are the key requirements for filling each position? Did you face any challenges in trying to recruit and hire qualified staff for these positions? If so, please describe.
Have you faced any turnover for these positions? If so, please describe.
How often and in what positions?
How long did they take to fill?
What types of support, such as grantee conference calls or convenings, have you received from DOL [or, for subgrantees, from the intermediary] on how to design or implement the program funded by your most recent REO grant?
Was this guidance adequate for your needs?
Would additional guidance have been helpful? If so, on what topics?
[If organization has received prior REO grants] How, if at all, did any guidance on other REO grant(s) inform implementation of this program?
What types of training were provided to staff—frontline and managers—in preparation for launching your REO grant(s)?
Who provided this training?
How long were the trainings, and what topics did they cover?
Is this training also provided to any new staff when they are hired?
Are these types of trainings also provided for staff in the organization who do not work on REO, or are they specific to this program?
[If organization has received prior REO grants] How were these trainings different from those provided to staff under your other REO grant(s)?
I would now like to ask you about the partnerships that make the REO program possible.
Please describe each critical program partner organization. [site visitors: consider partners such as American Job Centers, Community Colleges, and criminal justice system agencies that provide key services such as training, employment services or significant recruitment referrals, and not lightly used partners, such as supportive services, that case managers refer to infrequently. If the respondent does not mention American Job Centers, ask explicitly about whether they partner with the workforce system and the reasons why or why not. For each partner, please ask the following questions.]
What is the name of the partner organization?
How long has this partnership been in place?
What type of organization is it (workforce, social services, education etc.)?
[For Partners] How is your organization funded?
[For Partners] How are your REO-related services funded? Are they funded through the REO grant or other funding? If other funding, what funding?
Does the partner have a formal agreement with the REO program (such as an MOU or a contract)? If so, please describe.
Broadly speaking, what types of services does the partner provide for REO participations (training services, employment services, recruitment, housing, legal services, etc.)?
Where are partner services provided (at the partner’s location or at the REO-funded program office)?
How much experience does the partner have in working with this population?
How has this partnership evolved over time? Did your partnerships change due to the COVID-19 pandemic? For example, did you add new partners?
To what extent do you coordinate around REO?
[For CBO/subgrantee] What partner organizations do you coordinate with most/least?
For CBO/subgrantee] How often do you communicate about REO? Which REO staff communicate with partner organizations?
What program elements or services do you coordinate about most?
How are referral decisions to/from partner organizations decided?
[For CBO/subgrantee] Do you think you have the right mix of partners? Why or why not? If no, what additional types of partners might strengthen your program?
What has been your process for building and growing these partnerships?
How has this/these partnership[s] most benefitted your participants?
What has proven most challenging around these partnerships and why? How have you overcome challenges?
What challenges, if any, has COVID-19 presented to these partnerships?
[For CBO/subgrantee] Which partnerships do you think are most likely to last after the program funding ends? [For partners] Is this partnership likely to last after the program funding ends?
Now I’d like to hear about partnerships with employers.
To what extent were employers involved in the planning stages of the REO grant?
How did their perspective shape the design or implementation of the grant?
Are employers involved in ongoing program planning decisions?
If so, how many employers are involved?
Which employers? What industries are they in?
Is there variation in the extent of their involvement?
How do they provide their input (regular meetings, as issues come up)?
Are employers involved in providing employment-focused services and/or work-based learning through REO?
If so, what roles do they play (such as offering internships, speaking at trainings, participating in mock interviews, etc.)?
In what other ways, if any, are they involved?
[For CBO/subgrantee] What practices have been successful in engaging employers?
[For CBO/subgrantee] What efforts have you made to encourage employers to hire justice-involved individuals?
Have you assessed the needs of local employers? If so, how?
Do you participate in the Federal Bonding program? If so, what works well and is challenging about that program? If not, why not?
How, if at all, has COVID-19 affected any employer outreach efforts?
[For CBO/subgrantee] Do you have job development staff reaching out to employers? If so, please describe their responsibilities and some examples of the ways they work with employers.
[For CBO/subgrantee] Do you target certain types of employers? If yes, which types and why?
Do you target particular industries?
Do you target particular firm sizes?
Other targeted characteristics?
Next, I’d like to ask about your recruitment and enrollment process.
[Site visitors: For section 6, please obtain participant statistics ahead of time from program records and confirm as much information as possible ]
How would you describe your target population for REO?
What is their typical age, race, and gender?
What kind of work history, job skills and educational background do they typically have prior to enrolling?
What types of criminal backgrounds do they typically have?
Did your target population change at all due to the COVID-19 pandemic? If so, how?
[For young adult serving programs only] Describe the characteristics of your young adult participants.
Do they tend to be at the lower or upper end of the young adult age range?
Approximately what percentage of the young adults participants, if any, do not have prior justice system involvement? [Note to site visitors: per FOA, up to 10 percent of participants can be enrolled as high school dropouts without justice system involvement].
How do young adults (18-24) differ from older or younger populations?
Have enrolled participants’ characteristics, such as their demographics, justice involvement, and educational background, varied from what you expected at the start of the grant? If so, how?
Were you able to start enrolling participants within three months of your grant award? If not, why not?
Were you able to make your enrollment goal/do you anticipate being able to make it?
If so, what has most contributed to doing so?
If not, why not?
How, if at all, did COVID-19 affected your ability to meet your enrollment target?
What kinds of recruitment activities do you conduct (in facility, community-based, etc.)?
What are the primary referral sources for your program? About what proportion of participants are from referrals versus walk-ins?
How are probation, parole, jail, prison and other justice system partner staff involved in the recruitment and enrollment process, if at all?
What access do REO recruitment staff have to prisons, jails, and other facilities from which potential REO participants may be exiting?
Has it been challenging to find interested individuals or is there more interest than available slots? How has your organization responded to any challenges or surpluses?
What works well about your recruitment methods, including any partnerships?
What could work better about your recruitment methods, including any partnerships?
How, if at all, did COVID-19 affect recruitment efforts for REO?
Does your program include any eligibility criteria beyond those required by DOL? If so:
What are the additional criteria? When were these different criteria implemented and why? [Note to site visitor: DOL criteria for young adults include being 18-24, currently or previously involved with the justice system (up to 10% of participants may be those who have dropped out of high school without criminal justice involvement), low-income or residing in a low-income area, never convicted of a sex crime other than prostitution. DOL criteria for adults include being 25 or older, never convicted of a sex crime other than prostitution, low-income or residing in a low-income area, previously incarcerated or released from prison or jail within 180 days of enrollment or currently under supervision.]
If you work with justice-involved individuals in other capacities, how are these criteria similar or different than what you use for those programs or services?
How do you identify who is eligible for the program?
Which staff are involved in identifying who might be eligible?
Do you screen potential participants for work suitability, motivation or other factors prior to program enrollment?
Do you use a criminogenic risk assessment to determine eligibility (rather than for case planning)? If so, which tool and how do use use it?
Do you use other assessments to determine eligibility? If so, which ones?
Have you had any difficulty finding individuals who meet the program’s eligibility criteria? If yes, what are the key challenges?
How often do you screen someone who is ultimately determined ineligible? What steps do REO program staff take in these instances?
[Site visitors: for impact study respondents, please confirm as much information as possible based on prior information gathered on these programs.]
Beyond eligibility determination, please describe the structure of your program’s intake and enrollment process.
Do you conduct group or individual intake sessions?
If you have group intake sessions, where and when do they occur?
When do you notify a participant of acceptance into the program?
How long after enrollment do participants begin receiving services?
Do you enroll participants in cohorts or on a rolling basis? If you enroll in cohorts, how large are the cohorts?
Do you offer any incentives to encourage enrollment or to encourage participation after enrollment?
Next I’d like to ask you about your case management services.
What is your overall approach to case management services in the REO program?
What are the goals of case management?
How is REO case management different, if at all, from other programs that your organization may operate?
How are case management services delivered?
At what point(s) in the program do participants begin and end receiving case management services?
How many times per month are case managers required to communicate with participants on their caseload? How long are these required contacts?
Where do case management meetings take place?
What other ways do case managers contact participants (e.g., via phone, text, email or social media)?
What challenges have you had making meetings happen regularly?
What do case managers and participants typically meet about?
What is the average caseload for each case manager? Is this too high/too low/about right? Please explain.
What are the most common service referrals (e.g., housing assistance) that case managers make? Who are your common referral partners?
What proportion of participants are referred for other services, and what proportion actually receive them?
Is this different for formal/contracted partners as compared to soft referrals? If so, how?
How do REO case managers track participants’ receipt of partner services?
How does the case management process inform services provided by REO partners?
How, if at all, do you coordinate with case managers for other program partners (e.g., college programs, probation/parole, etc.)?
How, if at all, do you coordinate with case managers for other non-partner programs (e.g., schools, TANF, foster care, etc.)?
Does your REO program use a cognitive behavioral therapy (CBT) program model, such as Thinking for Change, Motivational Interviewing, etc.? If yes:
What programs do you use? How and why did you decide to use them?
How long is the program you chose (e.g., total hours of services and calendar length)?
What proportion of participants receive CBT? What proportion complete the program? What are common reasons for not completing?
When and how do you integrate CBT into program services?
Which staff are responsible for delivering them?
What training have your staff received in using this model?
[For intermediary organizations] What kind of support and oversight does your organization provide to help staff implement this model with fidelity?
How do you think these programs benefit REO participants?
What drawbacks, if any, do you see in these programs?
What assessments, including those used during recruitment and intake [site visitors: see 6.C.2.d and collect any copies of customized tools], do you use to plan the types of services to offer participants?
What tools do you use and why did you choose these particular assessments?
At what point in the program are participants first assessed? How often, if at all, are participants re-assessed? For which tools?
Do you develop an individual employment/development plan, or IEP/IDP, for each participant? Can you describe the process for developing one and what is included in it?
Do you have access to other partner organizations’ service plans? Parole/probation reports and plans? Court dispositions/dates? How are these tools used in developing service plans?
[For young adult serving organizations] Do you coordinate with other systems that serve young adults (schools, foster care, etc.) when developing participants’ service plans?
Can you provide a me a copy of your planning tool?
What works well about these tools? In what ways could these tools work better?
Did any of these processes change due to the COVID-19 pandemic? If so, how?
Now, I have several questions for you about the educational services the program provides.
Which of the following education services do you offer? [site visitors: please obtain a menu or course book if available.]
Classes/programming for a high school diploma?
GED prep and testing?
ESOL/ESL/ELL?
College entry test prep?
Assistance with figuring out college financial aid or applying to college?
Individualized tutoring services?
Other? Please describe.
For each of the above educational services:
Who is the provider of that service (a partner, and which partner or REO program staff)? Why was this option chosen?
Where is it provided (at REO offices or at a partner location)? Why was this option chosen?
How do you determine if individuals need this service?
About what percentage of REO participants are provided this service?
Describe the timing of this service.
When do participants begin relative to the start of the program?
How often do they attend?
For how long does it typically last?
Are participants required to complete it before participating in vocational training or any other components of the program?
What academic credentials do participants achieve for completing it?
How do you monitor participant progress toward their educational goals?
Do you share academic progress information with other partners? If so, how?
How are these services different than what you provided prior to REO? Do they differ for individuals you serve that are not enrolled in the REO program?
What have you found most helpful in implementing these REO services?
What, if anything, has been challenging about providing educational services to REO participants?
How, if at all, did these services change under COVID-19?
Next, I’d like to ask about the occupational skills training participants receive.
What occupational skills training courses do you offer to REO participants? [site visitors: please obtain a listing of courses if available.]
How many courses are there?
In what fields?
Are these single courses or part of sequence of courses?
How did you determine the areas of training that you provide under your REO grant?
Are these options connected to local employer demands? If so, how? If not, why not?
Were employer or training partners involved in the course selection process? If so, which partners and how did they contribute? If not, why not?
Are training options tailored for the specific population of participants in the REO program (e.g., learning issues or their backgrounds)? If so, how? If not, why not?
For each type of occupational skills training:
Who provides the course (a partner, and which partner, or REO program staff)? Why was this option chosen?
Where is this training provided (at REO offices or a partner location)? Why was this option chosen?
What is the setting for this training? How is it typically delivered (i.e., in a classroom, at a worksite, a lab, other)?
How often does each course meet? How long is each session?
How long does each course/sequence of courses last?
What curriculum is used for each course or sequence of courses?
What credentials, if any, do participants receive for completing the course? Do participants have to complete an assessment to receive it?
About what percentage of REO participants begin each course? What proportion complete? What are common reasons for not completing?
Are these courses available to non-REO participants or REO participants only?
[If more than one option is available] How do participants decide which occupational skills training to enroll in?
What type of career exploration support is available?
Are each of the courses available to any REO participant who wants it? If not, why not (e.g., capacity, qualifications, etc.)?
Are participants able to select a preference? If not, why not?
How do you determine if participants are qualified to begin [their selected] training?
How are participants’ backgrounds (prior work experience, educational background, specifics of justice-involvement, interests, etc.) taken into account?
What additional steps might they have to take to get ready (e.g., enroll in college, complete educational requirements, complete prior course sequences)?
When do participants begin training courses relative to the start of the program?
How do you monitor participant progress toward their educational goals?
Are participants paid for their time spent in occupational skills training? If so, is this a stipend provided by the program (through program funds) or by an employer?
Are there any types of occupational skills training that you would like to offer but are unable to at this time? If so, what are they and why are you unable to offer them?
How, if at all, did these services change under COVID-19?
I am now going to ask more broadly about the career pathways your program has focused on.
How do you define career pathways at your organization?
How, if at all, were career pathways incorporated into the programming that your organization offered prior to REO?
Explain the sequence or steps of career pathways training for REO participants and what are the most common entry points.
How do different education, occupational training, and/or work-based learning opportunities build on each other?
What credentials, certificates, or degrees can participants earn?
In what ways can participants build on their career pathways training after completing REO?
What types of careers do REO participants pursue based on their training?
Were these career pathways trainings part of your organization before REO?
If so, how were they adapted for REO, if at all?
What, if anything, has been challenging about adapting or offering career pathways for REO participants?
I now have some questions about the work-based learning (WBL) opportunities available.
Which of the following WBL opportunities do REO participants have available to them?
Job shadowing or visits to job sites?
Paid internships?
Unpaid internships?
On-the-Job Training (OJT)?
Registered Apprenticeships?
Pre-apprenticeships or other forms of apprenticeships?
Other?
For each opportunity, can you describe:
The length of the opportunity (days, weeks, months)?
The total hours and hours per week?
Whether it is paid or unpaid?
What types of tasks participants have in these positions?
Whether it is available to REO participants only or other participants?
How did you select the types of WBL opportunities to offer?
[If offering a registered apprenticeship] Why did you decide to offer a Registered Apprenticeship option for REO? What value do you think this opportunity, in particular, offers participants? Employers?
How, if at all, are these WBL opportunities different from what was available at your organization prior to REO?
Did you have to adapt opportunities in any way for REO?
Did you have to adapt them (for REO or previously) due to participants’ justice-involvement? If so, how?
What industries and employers are involved in providing these WBL opportunities?
How did you select these employer partners?
What role did or do these employers play in planning or implementing WBL opportunities?
Are only certain participants eligible for these WBL opportunities?
If so, how do you match REO participants to a WBL opportunities?
What factors (educational background, work experience, criminal justice involvement, interests, number of positions, etc.) do you take into account in placing participants in WBL opportunities?
What proportion of participants enroll in WBL? What proportion complete it? What are common reasons for not completing?
What, if anything, has been challenging about adapting or offering work-based learning opportunities to REO participants?
Are there more participants interested in WBL than available opportunities? If so, about how many?
How, if at all, did these services change under COVID-19?
Overall, have these work-based learning experiences been positive ones?
How well have participants performed in these WBL opportunities?
How have employers responded to having REO participants on-site?
How have participants benefited?
Are employers interested in hiring or working with REO participants again? Why or why not?
Would you recommend WBL for this type of program and population? Why or why not?
I would now like to ask you about services REO provides that help participants find work.
Let’s talk about each of the following types of assistance.
Do you provide job search assistance, where participants look for opportunities, mostly on their own, with some help from staff?
What tools or resources are used for job search assistance?
Who provides this service (REO program staff, AJC staff or both)?
Is this service for REO participants only or other individuals at your organization?
Do you provide job placement assistance, where you provide a more guided approach to job searches?
Do you provide this support around specific industries or specific employers? If so, which ones?
Why types of jobs are targeted (temporary, permanent, transitional)?
How are participants matched with jobs? What criteria are used?
Who provides this service (REO program staff, AJC staff or both)?
What strategies have been most successful in placing participants in jobs?
Is this service for REO participants only or other participants?
How do you determine whether a participant receives more guided job placement assistance versus less intensive job search assistance?
Do you provide job development services where you build a relationship with a particular set of employers and work with them around placing participants?
If so, which employers do you work with?
How did these relationships develop?
Why were these employers selected?
What types of jobs do these employers offer?
For what reasons do you think these employers are open to hiring justice-involved individuals?
Is this service for REO participants only or other participants?
How do you use information about the job market to guide job seekers? Do you advise participants to pursue industries and jobs that have a high number of open positions, or growth opportunities?
Do participants have access to employment placement staff or placement services even after being placed in their first post-program job? If so, how long do they have access to these staff and services?
In what ways, if any, have you adapted these placement services to be relevant to justice-involved individuals?
How are these services different from what was provided prior to REO?
How, if at all, have these services changed under COVID-19?
What challenges do you encounter in trying to place justice-involved individuals?
What are common barriers to placing individuals?
What strategies do you use to overcome employers’ concerns?
Next, I want to ask you about several other types of services that support participants.
Do you offer work readiness and life skills courses or workshops to REO participants? If so, which of the following do you provide? [site visitor: please obtain a copy of any class or workshop schedules.]
Classes on work readiness (resume development, interview preparation, or workplace behavior – dress, communication, attendance, etc.)?
Classes on healthy relationships, anger management, or positive parenting?
Counseling for those experiencing sexual abuse or violence?
Financial literacy training or counseling?
Conflict resolution training?
Other?
For each of these services:
When do participants first get access to these services relative to their start?
Who provides these services and where are they offered?
How many total hours are these classes/workshops? Over what period of time?
How frequently are these courses or workshops provided?
What percentage of participants take them? What proportion complete? What are common reasons for not completing?
Are there life skills services you would like to offer that you currently do not? If so, which ones and why?
Do you offer formal mentoring services (i.e., with a trained mentor)? If so,
What proportion of your participants receive mentoring services?
At what point in the process do participants receive mentoring services?
Is mentoring provided one-on-one or in a group setting? Why is this model used?
What structured activities do mentors/mentees participate in? Are there required activities that mentors/mentees must attend?
How frequently do mentors/mentees meet? In what setting? What are the goals of these meetings?
How long is the mentor/mentee relationship expected to last?
How are mentors and mentees matched? What factors are considered when matching mentors and mentees? What types of participants receive a mentor?
Do you have enough mentors for the number of interested participants? If not, why not?
What has worked well in providing mentoring services?
What have been the biggest challenges in providing mentoring?
How, if at all, have you had to adapt these services under COVID-19?
Please indicate whether each of the following types of legal services are provided as part of the REO program.
Record mitigation and expungement services?
[For young adult programs] Record sealing?
Navigating or resolving child support?
Dealing with restraining orders or help with domestic violence issues?
Child custody issues?
Restoration of driving privileges and licenses?
Court representation?
Help paying for court fees or fines?
Other?
For each service listed above, please indicate:
Is this service provided by the REO program or a partner organization? If a partner, which partner and what type of arrangement is used (e.g., soft referral, MOU, subcontract)?
Where is the service delivered?
Is the service provided one-on-one or in a group setting?
When is this service available?
Do all participants receive assistance with legal services? If not, what proportion need them? How does the program decide who will receive these services?
Are some legal needs easier for the program to meet than others? Which ones?
Are there other legal services you would like to offer? If yes, please explain.
What are the most frequent legal service needs that REO participants report having?
How do these differ, if at all, from the needs of your clients in other programs or before you had REO funding?
What has been working well in providing legal services?
What has been the biggest challenge in providing legal services?
Which of the following supportive services are provided through REOgrant funds?
Housing?
Transportation?
Child care?
Substance abuse treatment services?
Mental/Physical health services?
Uniforms, work clothes or work equipment?
Needs-related payments (e.g., utilities, child care)?
Other?
For each available service, please describe the following.
Is this funded through REO or leveraged funds?
Is this service provided by the REO program or a partner organization? If a partner, which partner and what type of arrangement is used (e.g., soft referral, MOU, subcontract)?
What are the most frequent supportive service needs of REO participants?
How do these differ, if at all, from the needs of your clients in other programs or before REO?
What proportion of participants actually receive each type of support service?
What has worked well in providing supportive services?
What has been the biggest challenge in providing supportive services?
Does your program provide financial or other incentives to participants to encourage program completion? If so, please describe.
Are incentives used around particular program elements?
How large are the incentives and in what form (cash, gift card, etc.)?
When and how do participants receive the incentive?
What proportion of participants have received them so far?
What has worked well in providing incentives?
What has been the biggest challenge in providing these incentives?
On average, how long are participants expected to stay in the program?
Is this based on a set length of time (e.g., nine months) or on achieving specific outcomes such as obtaining employment?
On average, how long do participants actually stay in the program?
What are common reasons that participants do not stay in the full length?
When and how do you formally exit participants from the program?
Does exit occur when participants achieve a particular outcome (such as obtaining employment)?
Does exit occur when participants go a particular length of time without receiving services? If so, what is that timeframe?
Please describe the following about your follow-up process.
At what point do you begin to follow-up with participants?
What approaches do you use (in person, phone, text, email)? Which approaches work best?
Do you ever reach out to employers or other parties directly rather than going through participants? If so, what is the protocol for doing so?
How often does follow up contact occur?
What data (pay stubs, school transcripts) do you collect during follow-up?
Which staff members conduct follow-up? If not case managers, how do you transition contact from case management staff to staff who provide follow up services?
Do you provide services during the follow-up period?
If so, what services? How do you determine who gets what services?
How do you balance or prioritize the need to serve current participants versus those in follow up?
Is there a point past which program graduates can no longer access services?
Overall, what are the biggest challenges to doing follow-up/maintaining contact? What strategies have you employed to overcome these challenges?
[Site visitor: This section is only asked at impact study sites. Please confirm as much information as possible from study planning conversations with impact study sites and referral lists created for control group members. Also please find out as much as possible about alternative providers you are not meeting with from the REO program staff, but note that they may not know the answer to some questions.]
Are there programs or organizations that provide services similar to the REO program in your area? If so, what programs or organizations provide similar services?
For each program or organization listed above, please answer the following questions about the program operations.
How long has it been operating?
What is the main funding for its services?
How many staff work on the program?
Where is it located? What geographic area does it serve? To what extent, if at all, does that area overlap with REO?
What does the population look like in this program or at this organization?
Does the program or the organization serve only justice-system involved individuals or a combination of different individuals?
Does the program serve a similar demographic as REO (e.g., age, gender, type of justice system involvement, risk level, etc.)?
In what ways does the target population differ from REO’s?
Describe the key services of each alternative program listed above.
What types of case management services does it provide? Does it use any evidence-based case management models?
What types of education and occupational skills training does it provide?
Does it engage employers? If so, how?
Does the program offer career pathways programming, registered apprenticeships, or other forms of work-based learning? If so, please describe.
Does it offer work readiness or employment services? If so, please describe.
Does the program offer mentoring, legal or life skills-related services? If so, please describe.
Does the program or organization provide supportive or follow up services? If so, please describe.
What types of partnerships does the program have with other organizations in the community?
In what ways are the programs at this organization distinct from REO? In what ways are they the same?
Do you communicate or coordinate with the alternative providers/REO program?
If so, how often and what do you communicate about?
If not, why not?
What types of additional services are needed for helping justice involved individuals in your community?
Next, I want to ask you about the program outcomes that the REO program tracks.
Please describe the most important outcomes for participants to achieve.
Which are most important to the REO program? Why?
Which are most important to the participants? Why?
If different, how does your program address this difference?
Please describe how your program participants were affected by the COVID-19 pandemic.
Please describe the following about participants’ abilities to find jobs after they complete the program.
Approximately what proportion of participants find work within the first three months after completing the program?
How long does it take them to find work on average?
In what industries or occupations do they typically find their first jobs?
What is the typical wage and hours worked for a participant who finds work?
What proportion find jobs that are related to the training they received through the REO program?
What are common reasons participants are unable to find a job?
What portion of those who find work are able to stay employed during the follow up period?
What are common reasons participants might struggle to stay employed?
What services, supports, or other factors play a role in helping them stay employed?
Please answer the following about participant recidivism.
Approximately what percentage of participants are arrested, convicted or incarcerated during program participation? For what reasons (parole violations vs. new crimes?
Approximately what percentage of participants are arrested, convicted or incarcerated during the follow up period? For what reasons (parole violations vs. new crimes?
From your perspective, what are the common reasons for participant recidivism?
How do REO staff learn when a participant has been arrested, convicted, or incarcerated?
To what extent do your program goals and desired outcomes overlap with DOL’s specified outcomes? To what extent do they differ?
Have you been able to achieve the benchmark goals negotiated with DOL?
What has been most challenging in trying to achieve DOL’s outcomes? Why? Did COVID-19 influence your programs ability to achieve DOL’s outcomes?
What strategies has your program put in place to help achieve DOL’s outcomes?
Are you using your own management information system, or MIS, to track individual-level data on REO participants (in addition to the DOL Access database)?
If so, what is the system?
At what point are REO participants entered into your system?
At what point are participants entered into the DOL Access Database?
If you have your own system, how is information transferred across systems?
Have you experienced any challenges in transferring data between systems?
Do partner staff have access to any data systems?
If so, which partners? What type of access do they have (read-only versus write)?
If not, how do you track and monitor the services provided by partner organizations?
Do you have processes in place to validate the accuracy and quality of the data entered by staff? If so, please describe.
What challenges have you had, if any with data entry or in reporting on the grant?
How do you monitor grant performance?
What criteria or measures do you use to determine performance levels?
What data are used to inform these measures?
How have the data been used to improve your REO program?
In thinking about the overall grant and the period from planning to the present, I’d like to get your thoughts on what has worked well, what have been the biggest challenges, and what we can learn from your experience.
How well do you believe the REO-funded program serves the needs of participants?
Which elements work best?
Which elements could work better?
What are the biggest ways the program helps justice-involved individuals?
What are the program’s biggest accomplishments?
With regard to participants?
With regard to building and growing the program and partnerships?
What are some of the program’s biggest challenges?
With regard to participants?
With regard to building and growing the program and partnerships?
What has been done to address these challenges?
What else do you plan to do to address these challenges further?
What other resources are needed to help participants meet their goals?
Do you think that REO offers the right services to help participants achieve their goals? If not, what other services would you like to add?
Are there any critical services you would like to provide but cannot fund under this grant?
What aspects of your REO program do you think your organization might try to sustain beyond the end of the grant?
Have you thought about how you might fund any efforts to sustain REO programming? If so, what plans do you have?
If your implementation of REO were to be replicated elsewhere in your state or in the country, what are the key elements that would need to be included for the replication to be successful? Which elements would you change?
What are some ways the program has improved since it first began?
What would you do differently if you were starting the program now?
What lessons would you share with other REO grantees? With other practitioners serving justice-involved individuals? With policymakers and funders?
Is there anything else we haven’t discussed that we should be aware of?
|
Draft REO Implementation Study Protocol – Master Questions List |
|
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Report |
| Author | Anne Paprocki |
| File Modified | 0000-00-00 |
| File Created | 2021-07-18 |