Attachment R Semi-structured mgmt discussion guide_final_v2

ATTACHMENT r

Program DIRECTOR/management discussion guide for follow-up SEMI-structured interviews

This page has been left blank for double-sided copying.

PROGRAM DIRECTOR/MANAGEMENT DISCUSSION GUIDE

Note to interviewers: This guide is intended for individuals identified as directors and/or other senior/program managers in the selected coaching interventions. Respondents will be familiar at a high level with the lead agency or program administrative structure, services, mission and goals, as well as the rationale, development, design, implementation history, structure and performance of the coaching intervention. In preparation for the interview, review the notes from the previous interview, ongoing technical assistance calls, and the reports we have written on the program.

Date of Interview:

Interviewer(s):

Site:

Respondent Name(s):

Title(s):

Respondent(s) Affiliation:

E-Mail(s):

Phone(s):

Address(es):

Fax(es):

Introductory Statement and Consent

Thank you for participating in the Evaluation of Employment Coaching for TANF and Related Populations, a national study being conducted by Mathematica and its partners Abt Associates and MDRC for the U.S. Department of Health and Human Services.

We very much appreciate all your work on the study. We learned a lot about your program when we talked with you last year. Now that we have analyzed the information we collected from you and your staff and participants, we would like to ask you some additional questions. These questions are about the coaches’ relationships with the participants, discussions about executive skills/skills to help complete tasks, stay organized, and control emotions, and the use of incentives. The information we collect will help other coaching programs learn from your experiences, challenges, and successes. The length of the interview is 60 minutes. Your participation in this study is important.

Your participation in this study is voluntary. During our conversation, anything you say will be kept private and we won’t use your name or any other identifying information when we report the results of our study. No identified information from the interviews will be shared by the researchers with anybody from [PROGRAM NAME or AGENCY]. Your responses will be combined with the responses of other staff and no individual names will be reported. You may also choose not to answer any question you do not want to answer. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0506 and the expiration date is XX/XX/XX.

I would like to record our conversation so I don’t miss anything. No one will hear the recording except for researchers. Is it okay with you if I record this conversation? We will destroy the recording at the end of the study. If you want me to stop recording for any reason or at any time, just say so.

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN

Through this information collection, ACF is gathering information to understand how coaching programs are serving individuals. Public reporting burden for this collection of information is estimated to average 60 minutes per respondent, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB # is 0970-0506 and the expiration date is XX/XX/XXXX. If you have any comments on this collection of information, please contact Sheena McConnell, Mathematica, 1100 First St. NE, Washington, DC 20002-4221.

[INTERVIEWER: IF RESPONDENT CONSENTS, PRESS THE RECORD BUTTON]

1. Current Position and Responsibilities

I would like to begin by asking a little about your position at your [agency/organization].

Have your roles and responsibilities at [agency/organization] changed since we last spoke with you? If so, probe for roles related to:

• Agency administration/planning/policy and procedure development

• Responsible for administrative oversight and/or policies and procedures for:

Eligibility and intake

Case management

Social work

Counseling

Coaching

Employment services

Referrals to other programs and services

Performance management

Other oversight responsibilities

• Other organizational responsibilities, such as:

Community involvement and outreach

Hiring

Budget planning

Fundraising

Other organizational responsibilities

2. Coaches’ relationships with participants

We have heard from participants at coaching programs, including [NAME OF PROGRAM], that they have developed strong, trusting relationships with their coaches. Some participants note that their relationships with their coaches are like “friendships.” Some point out that they are markedly different from the relationships they typically have with case managers. We would like to explore what makes these relationships different.

1. How do you define a strong relationship between coaches and participants? What are the key characteristics of a strong relationship?

2. When you select and hire staff to be coaches, do you look for specific characteristics or skills or experience that would help them to develop strong relationships with participants? What are those characteristics, skills, or experience?

3. In training coaches, do you explicitly train them on techniques for developing a strong relationship with participants? If so, what are those techniques? How do you train them on the techniques?

4. What do you think is the key to success in coaches developing strong relationships with participants?

   1. Probe for: characteristics of coach, allowing the participant to set their own goals and action plans, following up on issues raised in the coaching session, spending time listening to the participant and withholding judgement, the coach sharing his or her own experiences and challenges, talking to the participant about topics that are not related to employment, engaging outside of the meeting with email and texts, or authenticity and honesty.

   2. Is there something else?

5. How do you match coaches to participants? Do you match them based on their characteristics? For example, do you match based on gender, race/ethnicity, age, or parental status? Why or why not?

   1. If yes, do you think this matching contributes to successful relationship building? Why?

6. Are there particular circumstances in which it is harder for coaches to develop relationships with participants? If so, what are they?

7. Do you think the relationships have been easier or harder to develop during the COVID-19 pandemic? Why?

8. What challenges have your coaches faced in developing relationships? How have you addressed those challenges?

9. How has turnover of coaches affected the ability of participants to develop relationships with a coach? How do you discuss coach turnover with the participants? How do coaches work with clients who have transitioned from another coach? Do you take any other actions to reduce any negative impacts of the turnover of coaches?

10. [For FaDSS and Goal4 It!]: How do the coaches talk with participants about the TANF work requirements? Do the work requirements affect the ability of coaches to form strong trusting relationships with participants? If so, how? Do sanctions affect the ability of coaches to form strong trusting relationships with participants? If so, how?

11. Do you monitor whether the coaches have strong relationships with participants? If so, how?

3. Discussion of executive skills

MyGoals: MyGoals differs from many other coaching programs in that coaches discuss a participant’s executive skills with them. I would like to ask some questions about talking with participants about executive skills.

1. Would you say it is easy or difficult to train coaches about executive skills? Do coaches understand the concepts? Do they understand their own strengths and weaknesses in executive skills?

2. When are coaches first supposed to talk with participants about executive skills? When they administer the executive skills questionnaire or at a different time?

3. When else are they supposed to talk about executive skills with participants?

4. How often are the coaches supposed to talk with participants about executive skills? Do they name them?

5. Do participants understand the different executive skills terms? How do you know?

6. How do participants respond to talking about executive skills?

7. Do you think participants change their behavior once they understand their executive skills strengths and weaknesses? Can you give me some examples?

   1. How are coaches trained to support this behavior change?

8. Do you talk with your coaches about their executive skills? Your executive skills? Why or why not?

9. In your opinion, is discussing executive skills with participants an important component of MyGoals? Why or why not?

10. Are there any challenges in discussing executive skills with participants? If so, how have you addressed them?

FaDSS, Goal4 It!, and LIFT: In some other coaching programs, the coaches talk to the participants about self-regulation skills. These are sometimes referred to as “executive functioning” skills or “soft” skills. They are the skills used to finish tasks, stay organized, and control emotions.

1. Do you train your coaches on the importance of these skills? Why or why not?

2. Do you train your coaches to talk with your participants about these skills? For example, do they talk about time management, emotional regulation, etc.? Do you train them to name the skills? If so, which ones do they name? If they don’t, why not?

4. Discussion of incentives

LIFT and MyGoals: Your program differs from some other coaching programs in that incentives are provided for engaging in the program and [for MyGoals] achieving employment outcomes. We would like to discuss your opinion of these incentives.

1. What do you think are the advantages of offering incentives?

2. Do you think participants change their behavior because of the incentives?

   1. In your opinion, are participants more likely to come to sessions because of the incentives?

   2. For MyGoals: in your opinion, do you think the employment incentives motivate the participants to find a job or stay in a job?

3. What do you think about the structure of the incentives?

   1. Amount

   2. Timing and what participants need to do to receive incentive payments

4. Are there administrative challenges in providing the incentives? If so, what are they?

5. Are there other disadvantages to providing incentives? If so, what are they?

6. In general, do you think offering incentives is a key element of the program? Why or why not?

FaDSS and Goal4 It!: Some programs offer incentives for participating in the program or achieving employment milestones. Your program does not. I’d like to discuss what you think about offering incentives.

1. Could your program offer incentives? Why or why not?

2. If you could, do you think it would be helpful to offer incentives?

3. If so, how much would you think would be a reasonable amount for an incentive? Would you offer them for participating in the program, achieving an employment goal, or achieving another milestone?

4. What do you think are the benefits and challenges of offering incentives?

5. Wrap-Up – Take-Aways from this Study

Thank you so much for sharing this information with us. We sincerely hope that this study can be useful to you. Is there anything else you would like to share with me about the topics we discussed today?

THANK YOU FOR YOUR TIME