Discussion guide for state and local administrators (Attachment B)

A ttachment B – Discussion Guide for State and Local Administrators

Introductory Script

The Building Evidence on Employment Strategies for Low-Income Families Project (BEES) is sponsored by the Office of Planning, Research, and Evaluation (OPRE) under contract with MDRC and its subcontractors, Abt Associates and MEF Associates. BEES aims to bolster the knowledge base of employment-focused interventions for TANF recipients and other disadvantaged groups, including those facing substance abuse and mental health challenges, by rigorously testing interventions that are designed to promote employment and economic security.

We are beginning a field assessment phase of the project to understand what types of employment interventions are connected with State and local agencies such as Temporary Assistance for Needy Families (TANF), Supplemental Nutrition Assistance Program (SNAP), workforce development, behavioral health, and criminal justice, and what programs or organizations are currently implementing or planning to implement these types of interventions. We are also interested in learning more about what State and local administrators and staff think are the most effective methods to move TANF clients and other disadvantaged groups, including those facing substance/opioid use disorder and mental health challenges, to employment. Your participation will help ACF understand the most important employment interventions that should be tested around the country.

We have two main purposes for gathering your feedback. First, we are interested in hearing about the priority areas of programming in your [state, county, city], as well as what you think effective employment services look like. Second, since we are aiming to identify interventions that are of high interest to the field, we are hoping you will be able to identify existing programs that might be appropriate sites for a random assignment evaluation.

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. This information collection has been reviewed and approved by the Office of Management and Budget with OMB #: 0970-XXXX which expires on XX/XX/XXXX.

This interview takes about two hours, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. Your participation is voluntary. You may choose not to answer any questions you do not wish to answer, or end the interview at any time. We have systems in place to protect your identity and keep your responses private. In writing up our findings from these interviews, we never attribute anything to a specific respondent. The research staff participating in this study have signed notices that they will keep all information gleaned through these interviews private to the extent permitted by law.

Individual (s) Interviewed: (names and titles)

Sponsoring Organization:

Date of Communication:

[Modules and questions will be selected for each discussion, as appropriate.]

Module A: Background

1. What is the function of the agency? What programs are housed within the agency? How is the agency funded?

2. Please describe your current role within [TANF, SNAP, workforce development, substance abuse, behavioral health, mental health, criminal justice, other]

3. Please tell me any other background information that we should know about this agency, and your involvement in it.

Module B: Purpose of Employment Services

1. When you think about employment services for disadvantaged populations generally, are there groups or types of clients that you feel may benefit most from these services? Why this/these groups?

Probe: TANF recipients, people with substance/opioid use disorder, people with mental or behavioral health challenges, people who have been formerly incarcerated, etc.

2. Obtaining employment is typically the overall objective for these services. That is, the point of providing these services is to help individuals obtain employment. However, some programs have different philosophies that guide the services that they provide. What employment services do you think are most effective? And how would the approach differ, if at all, depending on who they serving?

Probes: direct job placement, transitional or subsidized employment, on-the-job training or occupational skills training, etc.

3. How do funding decisions at the state or local level affect employment services?

Probes: How do they affect the types of services?

Now I’d like to ask you some broad questions about your overall view of employment services.

4. Thinking about the employment organizations and programs you are aware of, can you tell me your thoughts on what types of interventions may be most effective for TANF clients and other disadvantaged groups, including those facing mental health and addiction challenges?

Probe: Why (or not) do you believe that these interventions may be effective? Are you aware of any evaluations of services?

Module C: Participants in Employment Services

Now that we’ve discussed your state/locality’s overall approach to providing employment services, I’d like to ask you some specific questions about the populations, to whom you provide employment services.

1. Who are eligible to receive publicly-funded employment services in your state/locality?

Probe: TANF clients? SNAP recipients? Noncustodial parents? Individuals who are formerly incarcerated? Homeless individuals? Individuals with substance/opioid use disorder? Mental or behavioral health issues? Why do some groups receive these services but not others?

2. [FOR TANF Administrators/Staff] Does your state/locality target specific types of TANF clients for employment services?

Probe: If so, why? And who? Do they have specific barriers to employment?

3. Do the employment services provided vary by the population served? How are their specific challenges/barriers to employment addressed?

Probe: Are there different services based on the population? Can you give me some examples of the types of these services? Are clients with specific challenges/barriers to employment served “in house” or are they referred to other organizations? If so, which ones? What areas do they specialize in?

4. In your opinion (or if you have data to support this), which types of clients tend to benefit most from employment services?

Probe [for TANF]: Are there characteristics of TANF clients that help them to be better served by employment services? Why this/these groups?

5. Would it be feasible you to share administrative data with the research team? Specifically, what are the restrictions on sharing your administrative data with outside partners? If you are able to share your administrative data with outside partners, how would the research team go about gaining access to the data?

Module D: Employment Activities and Operation

Next, I’d like to ask you some more detailed questions about the employment services that your state/local agency provides.

1. Can you walk me through the process of getting employment services?

Probes: When does a client get these services? What kind of assessment of the individual’s employment history and skills is done before providing services, if any?

2. Can you please describe the types of employment services that your state/local agency provides?

Probes: One-on-one services, such as counseling, resume prep or mock interviewing? Group-based services, like job clubs? Self-guided services like access to computers or websites? Job developers? If they have job developers, how do they go about making connections with employers? Occupational skills training? Transitional or subsidized employment?

3. If you contract out for employment services, can you describe this process and how individuals are referred to the provider?

4. Who makes decisions about what employment services an individual receives?

5. Thinking about the employment services that your state/local agency provides, are there other related services that you would like to provide to clients or think that clients need but aren’t provided? If so, what are these services? Why are they not provided? Why do you think they would be helpful to clients?

Probes: Services for people with substance/opioid use disorder? Mental health service? Behavioral health services? Housing? Services and supports to address incarceration/returning to the community after incarceration?

6. Please describe the challenges you face in implementing employment services?

Probes: What issues do you think may impact efforts to improve these programs? Are there any emerging issues, trends, or problems you see in the near future? Ask specifically how the opioid epidemic is affecting their employment rates?

7. What services do you think are most effective at helping clients to obtain employment?

8. If you could change anything about the employment services that you provide or how you provide them, what would you change and why?

Module E: Employment providers

I’d like to ask you some questions about who provides employment services.

1. Who provides employment services?

Probes: Specific types of organizations.

2. How are the contracts for these organizations managed?

3. What services other than employment do they provide?

Probes: Supportive services? Services for people with substance/opioid use disorder? Mental health? Behavioral health?

Module F: Evaluation Readiness/Interest

As I mentioned at the beginning of this interview, we aim to conduct a rigorous test of the effectiveness of employment interventions for TANF clients and other disadvantaged groups, including those facing mental health and addiction challenges. We are currently talking to other state and local administrators to learn about the services that they provide. We are also gathering information that will help us select sites to participate in the study.

1. Please tell me about any experiences that you have had previously with evaluations of your employment services?

2. Sample size is an important aspect of an evaluation. How many clients receive employment services in your state/locality and over what period of time is that?

Probes: Past month? Past six months?

3. [FOR TANF administrators] What data do you systematically collect about TANF recipients, their service receipt, and their outcomes?

Probes: work history, length on TANF, wage when employed, etc.

4. How interested are you in participating in an evaluation of employment interventions for TANF recipients and other disadvantaged groups, including those facing mental health and addiction challenges?

5. Are there any particular interventions or program models that you think are interesting, but for which there is currently insufficient information about whether they work well to support a decision to implement them?

6. Do you recommend anyone else with whom we should speak who has additional information that might be helpful?

7