Attachment S Semi-structured frontline staff discussion guide_final_v2

ATTACHMENT S

Frontline staff and supervisors Discussion Guide FOR FOLLOW-UP SEMI-STRUCTURED INTERVIEWS

PROGRAM FRONTLINE STAFF AND SUPERVISORS

DISCUSSION GUIDE

[Note to interviewers: This guide is intended for individuals identified as coaches and their supervisors in the offices in the selected programs. Respondents will be familiar with how coaching is delivered to clients. In preparation for the interview, review the notes from the previous interview, ongoing technical assistance calls, and the reports we have written on the program.]

Date of Interview:

Interviewer(s):

Site:

Respondent Name(s):

Title(s):

Respondent Affiliation:

E-Mail:

Phone:

Address:

Fax:

Introductory Statement and Consent

Thank you for participating in the Evaluation of Employment Coaching for TANF and Related Populations, a national study being conducted by Mathematica and its partner Abt Associates and MDRC for the U.S. Department of Health and Human Services.

We very much appreciate all your work on the study. We learned a lot about your program when we talked with you last year. Now that we have analyzed the information we collected from you, other program staff, and your program participants, we would like to ask you some additional questions. [TAILOR LIST DEPENDING ON THE PROGRAM.] These questions are about the coaches’ relationships with the participants, discussions about executive skills/ skills to help complete tasks, stay organized, and control emotions, and the use of incentives. The information we collect will help other coaching programs learn from your experiences and successes. The length of the interview is 60 minutes. Your participation in this study is important.

Your participation in this study is voluntary. During our conversation, anything you say will be kept private and we won’t use your name or any other identifying information when we report the results of our study. No identified information from the interviews will be shared by the researchers with anybody from [PROGRAM NAME or AGENCY]. Your responses will be combined with the responses of other staff and no individual names will be reported. You may also choose not to answer any question you do not want to answer. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0506 and the expiration date is XX/XX/XXXX.

I would like to record our conversation so I don’t miss anything. No one will hear the recording except for researchers. We will destroy the recording at the end of the study. Is it okay with you if I record this conversation? If you want me to stop recording for any reason or at any time, just say so.

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN

Through this information collection, ACF is gathering information to understand how coaching programs are serving individuals. Public reporting burden for this collection of information is estimated to average 60 minutes per respondent, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0506 and the expiration date is XX/XX/XXXX. If you have any comments on this collection of information, please contact Sheena McConnell, Mathematica, 1100 First St. NE, Washington, DC 20002-4221.

[INTERVIEWER: IF RESPONDENT CONSENTS, PRESS THE RECORD BUTTON]

1. Current Position and Responsibilities

I would like to begin by asking a little about your position and your [agency/organization].

Have your roles and responsibilities at [agency/organization] changed since we last spoke? If so, probe for roles related to:

• Program enrollment/intake

• Client assessment

• Coaching

• Case management

• Social work

• Counseling

• Delivering employment services

• Arranging enrollment in other programs [Ask respondent to name other programs]

• Making referrals/navigation of other services

• Other responsibilities

2. Coaches’ relationships with participants

We have heard from participants at coaching programs that the relationship between the coach and the program participant is very important for successful coaching. We would like to explore these relationships and practices that might strengthen them.

1. What do you do in your interactions with participants to build trust?

   1. When you first meet with them

   2. On an ongoing basis

2. Which of the following aspects of coaching do you think are the most important in building a strong, trusting relationship? Please discuss each one.

   1. Allowing the participant to set their own goals and action plans

   2. Not telling the participants what to do

   3. Following up on issues raised in the coaching session

   4. Spending time listening to the participant

   5. Listening without judgement

   6. Sharing your own experiences and challenges

   7. Talking to the participant about topics that are not related to employment

   8. Getting to know them personally

   9. Honesty and authenticity

   10. Engaging outside of the meeting through email and texts

   11. Something else

3. Which of these aspects of the relationship are most important? Why?

4. Now that you have had experience both meeting in person and virtually with participants, is it as easy to develop relationships virtually as it was in person? How does it differ? Does it differ by whether you have already developed a relationship with the participant in person?

5. For FaDSS: Does meeting in a participant’s home impact the strength of the relationship? In what ways?

6. How long does it take to develop a strong relationship with a participant? Does this vary from one participant to another? If so, why do you think that is?

7. Please describe some participants you felt you had strong relationships with and what specifically you thought helped strengthen those relationships.

8. Did you find it easier to have a strong relationships with participants if you have shared life experiences? If so, please describe what kinds of shared experiences are important.

9. What are the main challenges to building a strong relationship? Please probe for:

   1. Insufficient number of times meeting with the participant

   2. The participant not showing up to appointments or being late

   3. Meeting virtually

   4. Changes in coaches

   5. FaDSS and Goal4 It!: talking about the TANF work requirements

   6. Setting boundaries about when you can be in contact with the participant

   7. Other challenges?

10. FaDSS and Goal4 It!: How do you talk with participants about the TANF work requirements? Is this different that how case managers would talk about them?

11. Have you coached a participant who previously had another coach? If so, what did you say about the change in coaches? Were there any particular challenges in coaching someone who previously had another coach?

12. Please describe some participants you found more challenging to work with. Why were they more challenging? How did you address those challenges?

3. Discussion of executive skills

MyGoals: MyGoals differs from many other coaching programs in that coaches discuss a participant’s executive skills with them. I would like to ask some questions about talking with participants about executive skills.

1. When do you first talk about executive skills with participants? Is it when you administer the executive skills questionnaire or some other time? What are some reactions participants have to talking about executive skills?

2. At what other points during coaching do you talk about executive skills?

3. How often do you talk with participants about executive skills?

4. Do you name executive skills? Are there some you name and others you do not? Probe for:

   1. Response inhibition

   2. Working memory

   3. Emotional control

   4. Task initiation

   5. Sustained attention

   6. Planning prioritization

   7. Organization

   8. Time management

   9. Goal-directed persistence

   10. Flexibility

   11. Metacognition

   12. Stress tolerance

   13. Others

5. Which skills do you talk about with participants most often? Least often?

6. Do you think participants understand what you mean by executive skills? How can you tell?

7. How do participants respond when you talk about executive skills?

8. Do you think participants change their behavior because they understand their executive skills strengths and weaknesses? If so, can you give me some examples?

9. In your opinion is discussing executive skills with participants an important component of MyGoals?

10. Are there any challenges in discussing executive skills? If so, how have you addressed them?

11. Do you understand your own strengths and weaknesses in executive skills?

FaDSS, Goal4 It!, and LIFT: In some other coaching programs, the coaches talk to the participants about self-regulation skills. These are sometimes referred to as “executive functioning” skills or “soft” skills. They are the skills used to finish tasks, stay organized, and control emotions.

1. Do you/your staff talk with the participants about these skills? For example, do you/they talk about time management, emotional regulation etc.?

2. Do you/they name the skills? If so, which ones do you/they name? If not, why not?

3. Do you think it is helpful to discuss those skills?

4. Discussion of incentives

LIFT and MyGoals: Your program differs from some other coaching programs in that incentives are provided for engaging in the program and [for MyGoals] achieving employment outcomes. We would like to discuss your opinion about the effectiveness of these incentives.

1. What do you think are the advantages of offering incentives?

2. Do you think participants change their behavior because of the incentives?

   1. In your opinion, are participants more likely to come to sessions because of the incentives?

   2. For MyGoals: in your opinion, do you think the employment incentives motivate the participants to find a job or stay on a job?

3. Are there other benefits of offering incentives?

4. What do you think about the structure of the incentives?

   1. Amount

   2. Timing and what participants need to do to receive incentive payments

5. Are there administrative challenges in providing the incentives? If so, what are they?

6. Are there other disadvantages to providing incentives?

7. In general, do you think offering incentives is a key element of the program? Why or why not?

FaDSS and Goal4 It!: Some programs offer incentives for participating in the program or achieving employment milestones. Your program does not. I’d like to discuss what you think about offering incentives?

1. Do you think it would be helpful for your program to offer incentives?

2. How do you think participants would react to receiving an incentive? Would it change their behavior? Would it mean they were more likely to show up?

3. If so, how much would you think would be a reasonable amount for an incentive? Would you offer them for participating in the program, achieving an employment goal, or achieving another milestone?

4. What do you think are the benefits and challenges of offering incentives?

5. Wrap-Up: Desired Take-Aways from this Study

Thank you so much for sharing this information with us. We sincerely hope that this study can be useful to you. Is there anything else you would like to share with me about the topics we discussed today?

THANK YOU FOR YOUR TIME