In-depth participant interviews

DRAFT

Attachment F
IN-depth participant interview guide

This page has been left blank for double-sided copying.

Evaluation of Employment Coaching for TANF

and Related Populations

In-depth Interview Topic Guide

Introductory Statement and Consent

Thank you so much for meeting with me today. My name is __________ and I am with a company called [Mathematica Policy Research/Abt Associates]. We are an independent evaluator assisting the U.S. Department of Health and Human Services on a national study to learn more about whether providing people with a coach to assist them with setting and meeting goals helps them get and keep a job. The study will also help us learn whether there are ways to make the [coaching/program] work better. As part of this study, we are talking to people like you who have been paired with a coach to better understand your experiences with and views about the program.

Your participation in this study is voluntary. Our conversation will take about 2.5 hours. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-XXXX and the expiration date is XX/XX/XXXX. During our conversation, anything you say will be private, except if you say something that suggests you are very likely to harm yourself, that you are planning to hurt another person or child, or that someone is likely to harm you. We also won’t use your name or any other identifying information when we report the results of our study. We also won’t share what you say with your program or your coach. You may also choose not to answer any question you do not want to answer. I would like to record our conversation so I don’t miss anything. No one will hear the tape except for researchers and the person who types it up. Is it okay with you if I tape this conversation? If you want me to turn the tape off for any reason or at any time, just say so. [INTERVIEWER: TURN THE TAPE RECORDER ON]

A. Relationship building and participant background

1. Tell me a little bit about yourself.

1. Family composition (number and ages of children)

2. Favorite activities and hobbies

3. Length of engagement with program that administers TANF (if receiving TANF)

B. Client hopes, expectations, and service needs

1. Tell me about what brought you to [name of program]. How did you find out about the program? What did people say about the program?

2. When did you start participating in [name of program]?

3. What were your hopes when you first came to [name of program]?

4. What do you think is the purpose of [name of program]?

5. Tell me about what you have been doing as part of [name of program].

1. Orientation

2. Assessment

3. Goal or activity planning

4. Activities engaged in (job search, work experience, coaching sessions, training/education, etc.)

5. Work support services received (e.g., child care assistance including vouchers or funds; transportation assistance such as gas cards or bus passes; clothes, uniforms, tools or other supplies and equipment; tuition assistance; assistance finding stable housing; assistance with budgeting, credit, banking, or other financial matters; or assistance expunging a criminal record or other legal assistance)

6. Personal support services received (e.g., mental health counseling, domestic violence counseling or support, substance abuse treatment, or services to help with stress reduction or other approaches to improve emotional well-being)

7. What other kinds of help do you get from [name of program]? (e.g., help finding a job, help finding other services you need, help setting goals for your future.) Is the help useful? Why or why not?

C. Relationship with coach and other workers

1. Tell me all of the people you are working with at [name of program].

1. Role and types of help each worker provides

2. Tell me about [name of coach].

1. Whether currently working with coach

1. If relationship has ended, reasons why

2. Length of time working or worked with coach

3. How often contact occurs, how long sessions last, which modes of interaction are used (in-person meetings, phone calls, email, text), and how often each mode is used

4. If meet in-person, where in-person contact typically occurs and how often. Length of typical session.

5. If in-person contact is one-on-one or in group settings

3. What do you talk about when you meet with [name of coach]? Walk me through a typical session with your coach.

1. Content of initial coaching session

2. Content of subsequent coaching sessions (ongoing assessment/reassessment, review of goals from previous session, goal development, tracking and accountability)

3. Amount of time during coaching sessions spent on goal setting

4. Tools, activities, and types of materials used during coaching sessions

   1. Opinions of usefulness

5. Extent to which the participant determines what the participant and coach discuss

4. Do you meet with a case manager in addition to a coach?

1. Whether currently working with a case manager

   1. If relationship has ended, reasons why

2. How long participant has been or was meeting with case manager. How often contact occurs, how long sessions last, and which mode of interactions are used (in-person meetings, phone calls, email, text)

3. If meet in-person, where in-person contact typically occurs and how often. Length of typical session.

4. If in-person contact is one-on-one or in group settings

5. [If have case manager] What do you talk about with your case manager? Walk me through a typical session with your case manager.

1. Content of initial meeting

2. Content of subsequent meetings

3. Amount of time during meetings spent on goal setting

4. Tools, activities, and types of materials used during meetings

1. Opinions of usefulness

5. To what extent the participant determines what the participant and case manager will discuss and the next steps in the conversation

6. Perceived difference between participant’s coach and case manager

7. Whether the guidance the participant receives from caseworker and coach ever conflict (e.g., around what you need to do)

6. Going back to talking about [name of coach]. Do you ever leave your meeting with [name of coach] with things to do before your next appointment?

1. Types of tasks

2. Who determines the tasks

3. What specific tasks did participant find most and least useful

4. Whether participant typically completed tasks. If did not complete certain tasks, why not?

7. Does what you talked about during a session with [name of coach] influence your behavior after you leave? How so?

8. What do you think about the meetings you have with [name of coach]?

1. Likes and dislikes

2. Opinions of usefulness

9. Sometimes people miss meetings they have scheduled because other important things come up, or they are busy and forget about their meeting, or unexpected things happen like their car breaks down or their child care falls through. Have you missed any meetings with [name of coach]?

1. Reasons meetings have been missed

2. Consequences of missing meetings

3. Whether and how started attending meetings again

10. We just talked about missing meetings with [name of coach]. Has there ever been a time when you stopped participating in the program altogether (including not meeting with your case manager, if applicable)? Can you tell me more about that? What was happening in your life at the time? What changed that made you stop participating?

1. Has participant ever lost some or all of their benefits because of not completing program requirements—that is, been sanctioned

D. Service delivery and the goal achievement process

1. Do you currently have a goal or goals that you are working toward with your coach? What is that goal (those goals)?

1. [If participant answers no] Did you have a goal or goals you worked toward with your coach in the past? What was [that goal/those goals]? [If they have no goals they worked toward with their coach, skip to question 7]

2. [If multiple goals] Which goal is the most important to participant? (Focus on this one in next questions)

3. When was the goal developed?

4. How was the goal developed?

1. Role of coach in developing goal

5. How important is the goal to the participant? How motivated is the participant?

2. Did anything get in your way when you were figuring out what your goal would be?

1. Challenges faced

2. Strategies for addressing challenges

3. What happened if the participant didn’t have a goal when they were in the program?

3. What did you do after you identified your goal?

a. Action steps developed to achieve goal

b. Who developed the action steps and how

i. Role of coach in developing action steps

4. [If action steps were planned] Did anything get in your way while planning the steps you were going to take?

1. Challenges faced

2. Strategies for addressing challenges

5. Tell me about your experience with working on your goal

1. Challenges faced getting started and strategies for addressing challenges

2. If action steps were developed, whether they were taken to meet the goal. If action steps were not developed or the steps developed were not taken, how else did the participant work toward the goal?

3. Challenges with completing the action steps or goal (e.g., having difficulty getting started, having difficulty finishing tasks)

4. What did participant do when they faced these challenges?

5. How did what they actually do differ from what was planned?

6. What rewards, incentives, or other forms of motivation did the participant receive from the coach, if any

6. Did you change the goal you were working on at [name of program] or the planned steps (if steps were developed)?

1. Process for reviewing and revising goals and action steps (how often, how communicated)

2. Role of coach in helping to refine and change goals and action steps

7. Before working with your coach, did you have goals? What were they?

8. Please tell me about a goal that you worked on but did not achieve since you started working with your coach.

1. What was the goal?

2. When did you set the goal?

3. Did the coach help set the goal?

4. What prevented you from achieving the goal?

1. Other assistance or services that would have been helpful

5. Did you revise your goal, or did you just decide not to continue to work toward it?

   1. Process for reviewing and revising goals (how often, mode of communication)

   2. Role of coach in helping to refine and change goals

9. Please tell me about the goal that you have achieved that you are most proud of.

1. What was the goal?

2. When did you set the goal?

3. Did the coach help set the goal?

4. If achieved goal, what helped the participant achieve the goal? Was the coach helpful? Were other people or services helpful?

5. If did not achieve goal, what prevented the participant from achieving it?

1. Other assistance or services that would have been helpful

10. Did you set a new goal after you reached the goal you were working on? If so, what was it? Did it build off the last goal you achieved?

11. Does/Did [name of coach] do anything to motivate you when you are/were working with them on your goals? [Note: This question should be asked about the goal-setting process in general, not about the specific goal they have been discussing.]

1. Role of coach in motivating participant to achieve goal(s)

2. Receipt of rewards or incentives (e.g., things like gift cards, gas money, certificates, or praise)

3. Perceived effectiveness/usefulness of rewards, incentives, or other motivation techniques

12. Does/did [name of coach] ask you about how you are/were doing on your goal(s)? What happened if you had not met a goal? What about if you had met a goal? [Note: This question should be asked about the goal-setting process in general, not about the specific goal they have been discussing]

1. Did coach ask whether you had taken the action steps and met goals? What happened when you faced challenges?

13. Did [name of coach] do anything to celebrate your achievements? What did he or she do?

E. Progress toward improving work and life outcomes

1. Tell me about your work experience before you started [name of program]?

1. Employment status when started the program

2. Employment history in two years prior to starting program -- unemployment and employment

2. Are you working now?

3. [If working] Tell me more about the job you have now.

1. Type of job, how long been with employer; views and expectations on how long participant expects to keep this job

1. How got this job; whether and how coaching program helped in finding or maintaining a job; other sources of help

2. Training or education programs participated in to prepare or qualify for job

3. Satisfaction with current job, major positive aspects and negative aspects

4. Whether currently looking for a different job

5. [If looking for a different job] Current or most recent job search experience (for example, what job search activities they are doing, challenges, help received with it, type of job looking for, whether and how coaching program is helping with finding a new job)

4. [If not working] Are you doing anything to look for or prepare for a job?

1. Type of job looking for or preparing for

2. Current or most recent job search experience (for example, what job search activities they are doing, challenges, help received with it, whether and how coaching program is helping with finding a job)

3. Training or education programs participating in to prepare or qualify for job

5. Has [name of coach] helped you get closer to reaching your work goals? To reaching other goals for your life? How so?

6. What are your goals for your next job/career?

a. Aspirations, goals, and plans related to future employment/career

F. Post-program – what might be different as a result of the coaching experience

1. Would you say the coaching program has changed you or the way you do things? How so?

1. Differences in views, knowledge, skills or behavior related to:

1. Goal setting (for example, extent to which participant is goal setting on their own)

2. Employment (for example, job search, workplace behavior, long-term plans/career plans)

3. Feelings about the future

4. Motivation

5. Personal strengths and belief in ability to succeed

6. Stress management

7. Interactions and relationships with child(ren) and other family and friends

2. Assessment of what coaching program experiences and other factors may have contributed to any changes

2. What are the three main benefits you’ve received from the coaching program?

3. What do you wish was different about the coaching program?

4. Please think about other times you have worked with someone in a program who was helping you get a job, like a case manager, case worker, or a job specialist. How is working with [name of coach] different?

G. Wrap up

1. What are your hopes for the future?

2. Is there anything else you’d like to share about your experience with the program or your coach?