Discussion guide for program staff at potential sites (Attachment C)

A ttachment C – Discussion Guide for Program Staff at Potential Sites

Introductory Script

The Building Evidence on Employment Strategies for Low-Income Families Project (BEES) is sponsored by the Office of Planning, Research, and Evaluation (OPRE) under contract with MDRC and its subcontractors, Abt Associates and MEF Associates. BEES aims to bolster the knowledge base of employment-focused interventions for TANF recipients and other disadvantaged groups, including those facing substance abuse and mental health challenges, by rigorously testing interventions that are designed to promote employment and economic security.

We are beginning a field assessment phase of the project to understand what types of employment interventions are connected with State and local agencies such as Temporary Assistance for Needy Families (TANF), Supplemental Nutrition Assistance Program (SNAP), workforce development, behavioral health, and criminal justice, and what programs or organizations are currently implementing or planning to implement these types of interventions. We are also interested in learning more about what State and local administrators and staff think are the most effective methods to move TANF clients and other disadvantaged groups, including those facing substance/opioid use disorder and mental health challenges, to employment. Your participation will help ACF understand the most important employment interventions that should be tested around the country.

We have two main purposes for gathering your feedback. First, we are interested in hearing about the priority areas of programming in your [state, county, city], as well as what you think effective employment services look like. Second, since we are aiming to identify interventions that are of high interest to the field, we are hoping you will be able to identify existing programs that might be appropriate sites for a random assignment evaluation.

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. This information collection has been reviewed and approved by the Office of Management and Budget with OMB #: 0970-XXXX which expires on XX/XX/XXXX.

This interview takes about two and a half hours, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. Your participation is voluntary. You may choose not to answer any questions you do not wish to answer, or end the interview at any time. We have systems in place to protect your identity and keep your responses private. In writing up our findings from these interviews, we never attribute anything to a specific respondent. The research staff participating in this study have signed notices that they will keep all information gleaned through these interviews private to the extent permitted by law.

Individual (s) Interviewed: (names and titles)

Sponsoring Organization:

Date of Communication:

[Modules and questions will be selected for each discussion, as appropriate.]

Module A: Background

1. What is the function of the agency/organization? What programs are housed within the agency/organization? How is the agency/organization funded?

Probe: When was the agency/organization established? When was the program providing employment services established?

2. Please describe your current role within the agency/organization? What programs/services are you responsible for administering/providing?

3. Please tell me any other background information that we should know about this agency/organization, and your involvement in it.

Module B: Purpose of Employment Services

1. When you think about employment services for disadvantaged populations generally, are there groups or types of clients that you feel may benefit most from these services? Why this/these groups?

Probe: TANF recipients, people with substance/opioid use disorder, people with mental or behavioral health challenges, people who have been formerly incarcerated, etc.

2. Obtaining employment is typically the overall objective for these services. That is, the point of providing these services is to help individuals obtain employment. However, some programs have different philosophies that guide the services that they provide. What employment services do you think are most effective? And how would the approach differ, if at all, depending on who they serving?

Probes: direct job placement, transitional or subsidized employment, on-the-job training or occupational skills training, etc.

Now I’d like to ask you some broad questions about your overall view of employment services.

3. Thinking about the employment organizations and programs you are aware of, can you tell me your thoughts on what types of interventions may be most effective for TANF clients and other disadvantaged groups, including those facing mental health and addiction challenges?

Probe: Why (or not) do you believe that these interventions may be effective? Is there a specific structure or model to these interventions? If so, what components does the model entail? Are you aware of any evaluations of services?

Module C: Participants in Employment Services

Now that we’ve discussed your state/locality’s overall approach to providing employment services, I’d like to ask you some specific questions about the populations, to whom you provide employment services.

1. Who are eligible to receive your program’s employment services?

Probe: TANF clients? SNAP recipients? Noncustodial parents? Individuals who are formerly incarcerated? Homeless individuals? Individuals with substance/opioid use disorder? Mental or behavioral health issues? Why do some groups receive these services but not others?

2. Does your program target specific types of clients for employment services?

Probe: If so, why? And who? Do they have specific barriers to employment?

3. Do the employment services provided vary by the population served? How are their specific challenges/barriers to employment addressed?

Probe: Are there different services based on the population? Can you give me some examples of the types of these services? Are clients with specific challenges/barriers to employment served “in house” or are they referred to other organizations? If so, which ones? What areas do they specialize in?

4. In your opinion (or if you have data to support this), which types of clients tend to benefit most from employment services?

Probe [for TANF]: Are there characteristics of TANF clients that help them to be better served by employment services? Why this/these groups?

Module D: Employment Activities and Operation

Next, I’d like to ask you some more detailed questions about the employment services that your program provides.

1. Can you walk me through the process of getting employment services?

Probes: When does a client get these services? What kind of assessment of the individual’s employment history and skills is done before providing services, if any? Please outline numbers of participants per year at each step of your enrollment and service delivery. What is considered completion of the program? How long is your program in months?

2. Can you please describe the types of employment services that your program provides?

Probes: One-on-one services, such as counseling, resume prep or mock interviewing? Group-based services, like job clubs? Self-guided services like access to computers or websites? Job developers? If they have job developers, how do they go about making connections with employers? Occupational skills training? Transitional or subsidized employment?

3. Can you please describe the specific order or flow of the services delivered?

Probe: Are clients provided with a menu of services or do they have to follow a specific protocol? Please describe any curricula used in delivering employment services.

4. Can you please describe the types of referrals that your program makes and for whom these referrals are made?

Probes: Services for people with substance/opioid use disorder? Mental health services? Behavioral health services? Reentry services?

5. What is the theory of change for the services provided? That is, how do expect these services to help clients gain employment?

6. If you receive government contracts/funding for employment services, can you describe how this process works? How are agencies/organizations selected? How do they specific employment programs receive their funding allocation?

7. Who makes decisions about what employment services an individual receives? What is the process used to make these decisions?

8. Thinking about the employment services that your state/local agency provides, are there other related services that you would like to provide to clients or think that clients need but aren’t provided? If so, what are these services? Why are they not provided? Why do you think they would be helpful to clients?

Probes: Services for people with substance/opioid use disorder? Mental health service? Behavioral health services? Housing? Services and supports to address incarceration/returning to the community after incarceration?

9. Please describe the challenges you face in implementing employment services?

Probes: What issues do you think may impact efforts to improve these programs? Are there any emerging issues, trends, or problems you see in the near future? Ask specifically how the opioid epidemic is affecting their program? Are they experiencing an increase in clients with opioid dependency? If so, how have they tailored their services to address their needs?

10. What services do you think are most effective at helping clients to obtain employment?

11. If you could change anything about the employment services that you provide or how you provide them, what would you change and why?

Module E: Staffing

I’d like to ask you some questions about your role as someone who provides employment services.

1. What training did you receive prior to providing employment services?

Probes: Do you receive on-going training? How are you kept informed of changes in technology or other innovations that would affect how you provide employment services?

1. What are the caseloads for the workers who provide employment services?

3. How do you adjust your service delivery based on the needs of your clients?

Probe: ask specifically about the populations that BEES is targeting, e.g. opioid dependency, etc.

Module F: Evaluation Readiness/Interest

As I mentioned at the beginning of this interview, we aim to conduct a rigorous test of the effectiveness of employment interventions for TANF clients and other disadvantaged groups, including those facing mental health and addiction challenges. We are currently talking to other program administrators and staff to learn about the services that they provide. We are also gathering information that will help us select sites to participate in the study.

1. Please tell me about any experiences that you have had previously with evaluations of your interventions/program models?

2. Sample size is an important aspect of an evaluation. How many clients do you serve and over what period of time is that?

Probes: Past month? Past six months?

3. Where else do people in need of employment services go in your community?

Probe: Which services are most similar to yours and why? Do you ever refer people to these programs and, if so, under what circumstances?

4. What data do you systematically collect about your clients, their service receipt, and their outcomes?

Probes: demographic information, employment and earnings history, public assistance receipt, substance/opioid use disorder challenges, mental health issues, arrests, convictions, etc.

5. How interested are you in participating in a national evaluation of employment interventions for TANF recipients and other disadvantaged groups, including those facing mental health and addiction challenges?

6. Rigorous evaluations sometimes require adjusting the program. How easy or difficult do you think it would be tailor your intervention/program model to the BEES target populations?

7. Are there any particular interventions or program models that you think are interesting, but for which there is currently insufficient information about whether they work well to support a decision to implement them?

8. Do you recommend anyone else with whom we should speak who has additional information that might be helpful?

9. Is there anything else that you would like to tell me about your intervention/program model?

7