Interview Master Protocol

^{Attachment 2: Semi-Structured Interviews Master Protocol}

Linking to Employment Activities Pre-Release (LEAP) Evaluation

Site Visit Protocols

August 25, 2016

This page has been left blank for double-sided copying.

Master protocol for semi-structured interviews

This protocol is the source of questions for semistructured, in-depth interviews with five types of administrators and managers on-site: (1) Local Workforce Investment Board (LWIB) representatives and grant managers, (2) LEAP project managers, (3) jail administrators, (4) jail-based American Job Centers (AJC) managers, and (5) community-based AJC managers. Not every question will be asked of each respondent; tailored protocols with appropriate subsets of questions will be developed for each respondent type and for each of two rounds of interviews (years 1 and 2).

Instructions to interviewers: Bullets below each question represent probes for important details; depending on how well you are doing on time, try to make sure respondents touch on each of these issues. Text in [ ] corresponds to sections of write-up template.

Part I. Introduction (5 minutes)

My name is ___________________, and this is my colleague _________________ and we are from Mathematica Policy Research/Social Policy Research Associates. We are part of an independent research team contracted by the U.S. Department of Labor (DOL) to study the implementation of the [Program Name] program. Thank you very much for agreeing to participate in this conversation. Your participation is very important to the study.

To help us better understand how the program is working, we would like to ask you some questions about your experiences delivering the program. Our team will use your responses in conjunction with other data sources to identify themes to describe the overall experience of implementing the program. Your name will never be mentioned in a report, though we might use quotes from your interview to illustrate findings without using your name. The audio recording of the discussion is just to help us remember what you say in your own words and will not be shared with anyone outside our research team. [Placeholder –insert into text once we have received CoC approval: To further protect your privacy, the study team has obtained a Certificate of Confidentiality from the National Institutes of Health, which allows researchers to refuse to hand over identifying information in response to legal demands.] We want to reiterate that being part of this discussion is voluntary, and you may choose not to answer a question if you wish.

We have about [one hour/one and a half hours] for our discussion. Do you have any questions before we begin recording?

Ok ,we’re going to turn on the audio recorder now.

Part II. Participant Background [A1]

Let’s start by discussing your background and responsibilities.

1. What is your job title and what are your primary responsibilities for the LEAP grant?

2. What proportion of your time do you spend on LEAP? If not 100%, what other responsibilities do you have?

3. How long have you worked at your organization? In your current position?

4. What prior experience do you have relevant to your current duties on LEAP?

Part III. Local Context [A3]

I’d like to ask you a few questions about the local economy and your target population.

5. How would you describe the health of the local economy and the state of unemployment in this area?

• Who are the major employers and industries?

6. What do you feel are the biggest barriers for former inmates in your community in avoiding recidivism?

• Mental health or substance abuse problems (make sure to probe for this)

• Limited work experience

• Lack of employer willingness to hire former inmates

• Lack of education

• Lack of stable housing

• Lack of transportation

• Lack of family support

• Lack of health care and poor physical health

• High child support payments

• Negative peer and neighborhood influences

7. What are the most important employment skills that this population is lacking?

• Basic literacy and numeracy

• Vocational or technical skills

• Interviewing and interpersonal skills

• Credentials, including GED

• Job search skills

• Job retention skills

Part IV. Grantee Background [A2, A4]

I’d like to talk about how your organization came to be involved in LEAP.

8. How does the LEAP program align with the mission of your organization?

9. How do you hope to benefit from participating in this program? What are your goals for the program?

10. Is there broad support within your organization for this program?

11. Is there one particular individual in your organization who serves as a champion and driving force?

12. What are the advantages and limitations of having the LWIB lead this effort?

• Which other partners have been critical to roll-out of the program?

13. What prior experience does your organization have in serving justice-involved individuals before or after release?

• How long have you worked directly with this population?

• Have you implemented similar programs in the past?

• Do you have prior experience working with the jail or the Sheriff’s office?

14. What aspect of this particular program do you think will be most important for ensuring successful outcomes for program participants?

• Which is most critical: pre-release services, post-release services, or the link between pre- and post-? Why?

Part V. Grant and Program Planning [A2]

I’d like to talk about the application and start-up phase for the grant.

15. How was the decision made to apply for the LEAP grant?

• Who had the initial idea, and who was involved in the decision making?

• Were there any reservations about applying for the grant? If yes, what were they?

16. How did you decide which jail facility would accommodate the jail-based AJC?

17. Can you describe the process you used for rolling out the program after the grant was awarded?

• Who was involved in planning or design of the program, including partners?

• How long was the planning process?

18. Were there any challenges during planning that affected implementation? If yes, what were they, and how were they addressed?

19. Have you requested any modifications to the grant since it was awarded, such as an extension of the end date? Were your requests approved?

20. Did DOL have any concerns about how you wanted to use grant funds? If yes, did you have to make any changes to your implementation plans as a result?

21. Which individuals and organizations make up the core team that is responsible for setting policy for the program going forward?

Part VI. Partnerships [B1, B3, B4]

I’d like to talk about the partnerships that have been formed to implement the program.

22. Who are the main partner organizations that are involved in the program, and what is their role?

• Partners in corrections/law enforcement

• Partners for housing

• Partners for transportation assistance

• Partners for public assistance (TANF, SNAP, etc.)

• Partners for mental health or substance abuse services

• Partners for other health care services

• Partners for veterans services

• Partners for adult and continuing education

• Other community or faith-based partners (ask to specify)

23. Does your organization have previous experience working with any of these partners? In what capacity?

• How would you describe your relationship with each of these partners?

24. What prior experience do these partners have working with the justice-involved population? What experience do they have with employment programs?

25. What factors do you believe have facilitated each collaboration, and what makes your partnership effective?

• Have you faced any challenges working with any of these partners? How did you overcome them?

26. To what extent are these partners involved in shared decision making for the program?

27. Do you feel that there is a shared vision for the LEAP program among your organization and your partners?

Part VII. Integration of LEAP within the Jail Facilities [B7]

28. What is the value of a jail-based AJC from the perspective of the jail facility?

• Did the jail have any reservations about the program? If yes, what were they? How were those concerns addressed?

29. What role has the district attorney, the public defender’s office, or the local probation department played in the planning or implementation of the program? What is their perspective on the need for this type of program?

30. How well aligned are the priorities of the LWIB and the corrections community with regard to the LEAP program?

31. To what extent was the perspective of staff within the jail facility where LEAP is delivered represented in the program planning process?

• Which jail representatives were involved in planning?

• Is the jail involved in ongoing policy and program decisions about the program?

32. What operational or logistical issues did the jail facility have to address to integrate the jail-AJC into the facility?

• Transporting participants to program location

• Access to Internet for staff and participants

• Security access for AJC staff and other partners

• Changes in duties or protocols for jail staff

33. Have there been any security incidents related to the program since it began? If yes, can you describe what happened? Have measures been taken to prevent further problems?

34. Have there been any other challenges integrating the LEAP program into the jail facility? If yes, what were they and how were they addressed?

Part VIII. Partner Arrangements [B2, B5]

35. What type of contract, subgrant, or memorandum of understanding does the LWIB have in place with each of the LEAP partners?

• Are there specific performance targets associated with any of these contracts? If yes, what are the targets?

36. What is the LWIB’s approach to coordinating the program among the various partner agencies?

• How often do LEAP staff communicate with partners? Do they have regular meetings? What is discussed in these meetings?

• How are guidelines or changes in policy communicated to partners?

37. Are any of the partners co-located with LEAP program staff?

38. How is partner implementation of the program monitored?

• To what extent do LEAP managers directly supervise partner staff?

• How does the program ensure that partner staff are qualified to serve LEAP participants? What qualifications are prioritized?

39. What data on participants do the LEAP administration share with partners? What data do partners share with LEAP? What methods are used for data sharing?

Part IX. Relationship with Employers [B6]

Now I’d like to hear about your relationship with employers in the community.

40. To what extent were employer interests represented in the planning stage for the program?

• How did their perspective shape the design or implementation of the program?

41. Are you able to include the employer perspective in ongoing policy and programmatic decisions?

• How many employers are involved? In what industries? Is there variation in the extent of their involvement?

42. Are employers involved in the provision of services through LEAP? For example, offering internships or training, participating in mock interviews, etc.

• In what other ways, if any, are they involved?

43. Have you faced any challenges getting employers to participate in the program?

44. What efforts have you made to encourage employers to hire former inmates?

• Have you assessed the needs of local employers? If so, how?

• What common concerns do you hear from employers regarding hiring former inmates?

• Do you have job development staff reaching out to employers?

• Do you target certain types of employers? If yes, which types and why?

45. How many employers do you actively work with to hire former inmates?

• What are the common industries? How large or small are the employers? Where are they located, geographically, compared to the communities where your participants return?

Part X. Administrative Structure and Staffing [A1, D5]

Let’s talk about the structure and management of the LEAP program.

46. What is the overall structure of oversight and staffing for the program?

• What are the primary staff roles and responsibilities?

• Managers

• Frontline pre-release staff

• Frontline post-release staff

• Have these roles been filled by existing staff or have you hired new staff?

• What qualifications do you look for when hiring for these positions?

47. Have you faced any turnover for these positions?

• How often and in what positions? Are these positions offering pre-release or post-release services?

• What is your process for filling vacant positions?

• How long did they take to fill?

48. How are the day-to-day operations of the program managed?

• How are frontline staff supervised?

• How frequently do program staff meet to discuss progress or changes?

49. What approach does the program take to management?

• What types of decisions are frontline staff encouraged to make, and what do they need to discuss with a manager or higher-up?

50. What types of guidance have you received from federal or state authorities on how to design or implement the program?

• Was this guidance adequate for your needs?

• Would additional guidance have been helpful? If so, on what topics?

51. What types of initial training were provided to staff—frontline and managers—in preparation for LEAP? How long were the trainings, and what topics did they cover?

• Is this training provided to new staff when they are hired?

• Who provides the training?

52. What types of ongoing training do you provide for LEAP staff? How long are the trainings, and what topics do they cover?

• Who provides the trainings?

• How often do staff attend trainings?

Part XI. Operation of the Jail-AJC [D6]

I’d like to ask a few questions about how the jail-AJC was set up and operationalized.

53. Who was responsible for implementing the plan for the jail-AJC?

54. How did you determine what space within the jail would be used?

• Was the space already being used for another purpose?

• What is the square footage of the space?

• Is the space currently being shared with any other programs?

55. How is the space organized?

• How many rooms are there?

• Are there computers available for inmates to use?

56. Is Internet access available for inmates and staff?

• What content are inmates allowed to access?

• Are there restrictions for staff?

• Have there been challenges getting permission for Internet access?

57. Is there a mobile unit to take the LEAP program to other parts of the jail if needed?

• If yes, how often is it used?

58. Did the jail have to make any special accommodations for the program or for staff?

• Did staff have to go through any special security training with the jail?

• Have there been any issues with staff accessing the facility or being able to provide services? If yes, what were they and how have they been addressed?

Part XII. Participant Recruitment and Enrollment [D1, D6]

Let’s talk about how potential participants enter the program.

59. How would you describe the population you are targeting for the program?

60. When did you start enrolling participants, and do you have a planned end date for enrollment?

61. What is your target number for enrollment into the program?

• How close are you to meeting your targets?

62. Do you conduct outreach to inform potentially eligible inmates about the program?

• What strategies have you used for recruitment? Which have been the most successful?

• How do inmates express interest—is there an application process?

• Are inmates typically interested in the program, or do they have concerns?

63. What criteria do staff use to determine eligibility for the program?

• Which staff are involved in identifying who might be eligible?

• Do you use specific assessments to determine eligibility? If so, which ones?

• Do you use a criminogenic risk score? If so, how is it structured, or what criteria is it based on? Can we receive a copy of how the score is calculated?

64. Have there been challenges garnering interest in the program or is there more interest than available slots in the program?

• If there are challenges, have you made changes to the program to increase interest?

• If more interest, do you use a waiting list to determine enrollment?

• Are there certain types of participants that you believe are more motivated to participate, or who are more responsive to services?

• What type of individual do you think is best served by the program?

65. Do you enroll participants in cohorts or on a rolling basis?

• If you enroll in cohorts, how large are the cohorts, and how often do cohorts start receiving services?

66. Can you walk me through the typical intake process?

• Where does intake and enrollment occur?

• Do you offer a group orientation to the program?

• How long after enrollment do participants begin receiving services?

67. Do you offer any incentives to encourage enrollment or to encourage participation after they are enrolled?

Part XIII. Participant Access to the Jail-AJC [D6]

68. How are inmates transported to the jail-AJC for services?

• Are they brought individually or in small groups?

• Is there a limit on how many can be in the jail-AJC at one time?

69. Are there any restrictions on when participants can receive services?

• How long can they spend in the jail-AJC at one time?

70. Can participants take materials to read back with them to their cell?

71. Have there been any challenges with the logistics of participant access to the jail-AJC?

72. Is there anything that could be improved about this process?

Part XIV. Pre-release: Assessment and Service Planning [D2]

Let’s talk about how you determine which services are appropriate for each participant.

73. Do you develop an individual development plan, or IDP, for each participant? Can you describe the process for developing one?

• Which staff are involved in developing the IDP?

• How do participants provide their input into the IDP?

• Does the IDP include wraparound services, such as housing assistance, transportation assistance, mental health treatment, substance abuse treatment, access to public benefits?

• How is the IDP communicated to staff who provide services to participants after release?

• Is the IDP shared with any partners?

74. Do you use interest or skill assessments to determine the types of services that should be offered to a participant? Can you tell me the name of the tools you use, and whether you have prior experience with these tools?

75. Do you use assessments on an ongoing basis during the program, or do you update the IDP? If so, which assessments? How often do you update the IDP?

Part XV. Pre-release: Case Management [D3]

I’d like to talk about how case management is provided.

76. How many case managers provide services to LEAP participants?

77. What is a typical LEAP caseload for case managers?

78. Are these staff working solely with LEAP participants, or do they have other caseloads?

• If they have other caseloads, what proportion of their time do they spend with LEAP cases?

79. Do case managers spend the full day inside the jail facility, or do they divide their time between the jail-AJC and the community-AJC?

• If they divide their time,

• How much time do they spend in either place?

• Do they have both pre-release and post-release LEAP caseloads?

80. What do case managers usually discuss when working with participants?

• Does it differ for pre- and post-release?

81. How often do case managers typically meet one-on-one with LEAP participants?

82. In general, do case managers spend more time with pre-release or post-release cases? Why might one require more time than the other?

Part XVI. Pre-release: Employment and Support Services [D3, D4, D6]

I would like to hear about the types of services that participants receive.

83. Can you describe the different types of services that are offered in the LEAP program pre-release?
    
    For each service, ask about (1) content, and (2) length and frequency (or intensity in terms of total hours), (3) individual or group format, and (4) what proportion of participants receive the service.

• Career inventories

• Labor market information

• Resume preparation

• Interviewing skills

• Job search assistance

• Assistance getting IDs

• Registration with employment database

• Job readiness/soft skills training

• Behavioral supports or therapies

• Financial literacy

• Life skills (for example, time management)

• Remediation and GED preparation

• Vocational training

84. Do any of the services provided result in a credential or certificate?

• If yes, what credentials? Are they industry recognized?

85. What is the typical sequence of services for participants while incarcerated?

86. How long do you expect participants to be active in the program before they are released?

87. Are there other non-LEAP services within the facility that participants can receive?
    
    For each service, ask (1) how they access them, (2) who provides them, (3) how referrals are made, and (4) what portion of participants receive them.

• Mental health

• Substance abuse treatment or groups

• Cognitive therapy

• Educational services

• Parenting or relationship counseling

• Employment services

• Others?

88. Have there been any challenges providing LEAP services to participants pre-release?

• Has the program experienced any problems operating from within a jail facility? If yes, what has been done to address those problems?

Part XVII. Transition to Post-Release Services [E1, E2]

I would like to hear about how participants are transitioned to the community-AJC services.

89. How have you structured the “hand-off” of participants at the time of their release to the community-AJC?

• Are there different case managers for pre- and post-release?

• If yes, do participants meet with staff in the community-AJC before or after they are released? If after, is an appointment made for them? How quickly do they meet with staff after they are released?

• How do these pre-release and post-release staff communicate with each other about participants? How regularly do these staff meet about their LEAP cases?

• If the same staff provide pre- and post-release services, do staff schedule a meeting date during the pre-release process for the participant to meet with them after the participant’s release? How soon does this meeting happen?

90. What percentage of participants show up for their first appointment at the community-AJC post-release?

• What are some common reasons for participants not showing up?

• What process do you follow if a participant does not show? How good is your contact information to follow up and reschedule?

91. What strategies have you used to ensure that participants come to the community-AJC after they are released to participate in services? Have they been successful?

Part XVIII. Post-Release: Employment and Support Services [E3, G]

I would like to hear about the types of services participants receive in the community-AJC.

92. Can you describe the different types of job-related services that are offered in the LEAP program post-release?
    
    For each service, ask about (1) content, and (2) length and frequency (or intensity in terms of total hours), (3) individual or group format, (4) what proportion of participants receive the service, (5) who provides the service.

• Career inventories

• Labor market information

• Resource rooms with computers, newspapers, and other job search resources

• Resume preparation

• Interviewing skills

• Job search assistance

• Assistance getting IDs

• Registration with employment database

• Job readiness/soft skills training

• Financial literacy

• Behavioral support or therapy

• Remediation and GED preparation

• Vocational training—types of training, length of training programs, credentials offered, typical cost, funding source

• Job placement—types of employers, types of jobs, typical wages

• On-the-job training—types of employers, types of jobs, length of placements, amount subsidized, funding source

• Work supports (clothes and tools)

• Retention services—frequency of contact, remote or in-person contact, other retention supports

93. What is the typical sequence of services for participants after they visit the community AJC for the first time?

94. Are these services customized to be relevant to the justice-involved population? If yes, how?

95. To what extent are LEAP participants co-enrolled in other AJC programs?

• Wagner-Peyser

• Workforce Investment and Opportunity Act (WIOA)

• Trade Adjustment Assistance (TAA)

• Veterans Administration

• National Emergency Grants

• TANF

• Vocational rehabilitation

• Migrant seasonal farm workers

• Others?

96. How do you connect participants with other supports or wraparound services that they might need upon release?

• Housing assistance

• Public assistance

• Transportation assistance

• Mental health treatment

• Substance abuse treatment

• Subsidized health care or insurance

• Parenting or relationship counseling

• Child support assistance

• Other supports

97. How do LEAP case managers track participant receipt of these other services?

• What proportion of participants are referred for other services, and what proportion actually receive them?

• How do LEAP case managers communicate or coordinate with case managers at partner organizations?

98. To what extent are you satisfied that the services being delivered by the program are the right “mix” in terms of intensity, the topics covered, and the time allocated to them?

99. What other services do you think would be useful to participants?

100. Are participants receiving the services that are tailored to their individual needs?

101. Have there been any challenges with providing services to participants post-release? If yes, what kinds and how have they been addressed?

Part XIX. Data Collection [F1-F3]

Let’s talk about your data collection mechanisms.

102. Are you using a management information system, or MIS, to track individual-level data on participants? What types of information do you track?

• Participant demographics

• Assessment information

• Pre- and post-release service receipt

• Engagement in post-release services

• Referrals to other service providers

• Employment outcomes

• Recidivism outcomes

103. Do partner staff have access to this system?

• If not, how do you track and monitor the services provided by partner organizations?

104. Do you have processes in place to validate the accuracy and quality of the data entered by staff? If so, can you describe them?

105. Have there been any challenges with using the data system, either in data entry or in reporting on the program?

106. How do you monitor program performance?

• What criteria or measures do you use to determine performance levels?

• What data are used to inform these measures?

• How have the data been used to improve the program?

Part XX. Participant Outcomes [F4, F5]

107. Are participants generally able to find jobs after they are released?

• Approximately what proportion of participants find work within the first 3 months after they are released?

• How long does it take to find work on average?

• In what industries or occupations do they typically find jobs?

• What is the typical wage and hours worked for a participant who finds work?

108. Are those who find work able to stay employed?

• If yes, what services, supports, or other factors play a role in helping them stay employed?

• If not, what are common reasons?

109. Have any participants returned to incarceration after being released?

• If yes, do you have a sense of what proportion of participants return?

• What are the common reasons for participants to return to incarceration?

• How does the program know when a participant has returned to incarceration?

• How does returning to incarceration affect their enrollment in the LEAP program? Can they continue to receive services in the jail-AJC?

Part XXI. Program Changes [H1]

Let’s talk about how the program has changed in the past year.

110. Have changes been made to how the program is delivered based on statistics about participant participation and outcomes?

• What changes, and why were they made?

• How did the changes affect staff and participants?

111. Have significant changes been made for other reasons?

• What changes, and why were they made?

• How did the changes affect staff and participants?

Part XXII. Success, Challenges, and Lessons Learned [H2]

In thinking about the overall program and the period from planning to the present, I’d like to get your thoughts on what has worked well, what have been the biggest challenges, and what we can learn from your experience.

112. What have been the biggest successes in starting up and carrying out the program?

• Successes in terms of grant partnerships?

• Successes in terms of operating the jail-AJC?

• Successes in terms of working with participants pre-release?

• Successes in terms of linking participants to services post-release?

• Successes in terms of working with participants post-release?

113. What have been the biggest challenges in starting up and carrying out the program?

• What was done to address these challenges?

• Have these challenges been addressed successfully?

114. What other challenges exist that will affect whether program participants are able to achieve positive employment and recidivism outcomes?

• For example, a lack of housing, treatment, or other services in the community

115. What do you think could be improved about the program to better serve this population?

116. What lessons have you learned about serving this population that you think would be helpful for other jurisdictions that might want to implement a similar program?

Part XXIII. Sustainability [C]

I’d like to talk about sustainability of the program.

117. What interest is there at the local or state level for continuing the LEAP program beyond the grant period?

• [If there is interest] do you plan to extend the program? For how long?

118. What factors will determine whether the program will be sustained beyond the end of the grant?

119. What potential sources of funding have you identified that could be used to support the program?

120. What obstacles do you foresee to successfully extending the program beyond the grant period?

121. If this program were to be replicated elsewhere in your state or in the country, what are the key program elements that would need to be included for the program to be successful?

[If time remains, ask:] Is there anything else about the program or your experience you would like to share that didn’t come up already in the discussion?

Thank you for taking the time to share your thoughts and ideas with us today. This discussion has been very helpful in learning more about the [Program Name] program and staff experiences. We appreciate your time.