Fathers and Continuous Learning in Child Welfare Project

Instrument 1: Semi-Structured Protocol: Improvement Team Group Interview

A Breakthrough Series Collaborative for

This protocol will be used to facilitate group interviews with Improvement Team members, which include child welfare agency staff (such as managers, supervisors, and workers) and community partner staff (such as staff from father engagement organizations). In these interviews, participants will discuss their experience with the BSC, the resources required to participate in the BSC, the perceived benefits of the learning sessions, the barriers and facilitators to implementing strategies based on the Collaborative Change Framework and using the model for improvement, progress toward site-specific aims and short-term outcomes, plans for sustaining the practices identified based on the CCF, and consistency with the five key elements of a BSC.

Public reporting burden for this collection of information is estimated to average one hour and a half. This collection of information is voluntary and all responses collected will be kept private to the extent permitted by law.

An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection is 0970-0531 and the expiration date is 07/31/2022.

Engaging Fathers and Paternal Relatives in Child Welfare

Fathers and Continuous Learning in Child Welfare Project

Semi-Structured Protocol: Improvement Team Group Interview

Note to interviewer: The following detailed script should not be read verbatim. As the interviewer, you must be familiar enough with the script to introduce the study and the focus of the interview without reading word-for-word from the script. Please familiarize yourself thoroughly with the text before you conduct interviews. After introducing the site visit team and the study based on the detailed script, there is a brief script at the end that must be read verbatim to review the fundamental rights of the participants. Detailed Script

Introducing the interviewer and co-interviewer

Thank you for taking the time to speak with us today. We especially appreciate your willingness to participate in this site visit online given these extraordinary circumstances. We are from Mathematica, an independent research firm. My name is [NAME], and my colleague is [NAME].

Explaining the project and the purpose of the group discussion

You should have received a copy of a Consent Information Form by email. If you did not, please let us know and we will send you that information. I’m going to review the content of that form before we begin.

We are conducting the Fathers and Continuous Learning in Child Welfare project for the Office of Planning, Research and Evaluation within the Administration for Children and Families at the U.S. Department of Health and Human Services. This project is designed to support and learn about how child welfare agencies can engage fathers and paternal relatives using the Breakthrough Series Collaborative (BSC) methodology.

We are interested in hearing about your experiences using the BSC to enhance the engagement of fathers and paternal relatives’ with children involved in child welfare. We are interviewing Improvement Team members to learn about your role on the team, the supports you needed to implement the BSC, and how the BSC affected your work with fathers and paternal relatives and changed how engaged they were in child welfare cases. We will use the information you share with us to write a summary of what we have learned, but we will not use your name in our final report. I hope you feel comfortable enough to speak freely about your experience.

Privacy and taping

Our study team will summarize today’s discussion for the purposes of this project only. We will keep the information you provide in these discussions private to the extent permitted by law. We will not use any names or associate your name with a specific statement. We value the information you will share with us, and want to make sure we capture it all by recording this WebEx discussion. We will destroy the recording at the end of the project. If you want to say anything that you do not want recorded, please let me know, and I will be glad to pause the recording.

Before we start, I would like to remind you that participating in this interview is voluntary. We expect this discussion to take about 90 minutes. There are no repercussions if you choose not to participate. If you do not know the answer to a question, please let us know that, and we will simply move on. If you prefer not to answer a question for any reason, please let us know that, and we will move on. Do we have your permission to proceed with the discussion?

Does anyone object to our recording the discussion?

Before we get started, does anyone have any questions?

[The discussions will be tailored to the specific persons interviewed and interview format. To the extent information is not available from other sources collected through the BSC, such as the plan-study-do-act planning form and tracker; the following are guidelines for points for discussion.]

Ground rules for discussion

Before we start, we want to give you some ground rules about how our time together will go.

• Mute yourself when you are not speaking.

• Use your web camera if you are able.

• This will be an informal discussion.

• The discussion will last about 90 minutes. There will be no formal breaks. Please feel free to step away whenever you need to.

• There are no right or wrong answers to the questions. We are interested in learning everyone’s opinion.

• We would like only one person to talk at a time so the recording can pick up everyone. But we hope all of you will tell us your thoughts and feelings throughout the discussion.

• To get your thoughts on all the topics that we’re interested in, I might change the subject or move ahead from time to time.

• We also ask that you keep the discussion private and do not share it with others outside this discussion.

Are there any questions? Is everyone comfortable with these ground rules?

Also, before we begin I need to share with you that an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this information collection 0970-0356 and the expiration date is 07/31/2022.

Brief Verbatim Script

“Before we start, I would like to remind you that participating in this interview is voluntary. We expect this discussion to take about 90 minutes. There are no repercussions if you choose not to participate. If you do not know the answer to a question, please let us know that, and we will simply move on. If you prefer not to answer a question for any reason, please let us know that, and we will move on. Do we have your permission to proceed with the discussion?”

To get things started, I am going to go around the room and ask each of you to introduce yourself. Then we will discuss your experiences related to [agency]’s participation in the BSC. We would like everyone to participate in the discussion.

A. Please introduce yourself bydescribing your role on the Improvement Team and sharing how long you have been working at [agency or partner organization]?

Now, we would like to discuss the process for forming your Improvement Team and how being on the Improvement Team impacts your work.

B. What was the process for forming your Improvement Team?

[Probe on the following if needed]

1. What aspects of the process worked well? What aspects of the process were challenging?

2. Were there criteria for selecting team members? If so, what were they?

3. How were community partners selected? Why were those partners chosen?

C. We know that child welfare staff and their partners have many demands on their time. How are you able to make time to be a member of the Improvement Team?

[Probe on the following if needed]

1. How does it affect the rest of your job?

2. What organizational or other supports did you receive to participate on the Improvement Team?

D. How does bringing the Improvement Team together influence your work more generally?

[Probe on the following if needed]

1. In what ways does participating in the Improvement Team benefit your work with fathers and paternal relatives? Can you provide an example?

2. What do you find challenging about bringing the Improvement Team together?

We would like to learn more about your experience with the Collaborative Change Framework and the Model for Improvement.

E. The Collaborative Change Framework guides the work of improvement teams as part of the BSC. What are your thoughts about the Collaborative Change Framework’s domains?

[Probe on the following if needed]

1. Do you think the Collaborative Change Framework’s selected domains are the most relevant to addressing the challenges of engaging fathers and paternal relatives?

2. What aspects of the Collaborative Change Framework did you find most relevant?

3. Which aspects did you find least relevant?

4. What was missing from the Collaborative Change Framework?

5. Were there any unintended consequences associated with implementing the Collaborative Change Framework?

6. Will the [agency] continue using the strategies associated with the Collaborative Change Framework? Why or why not?

Let’s talk about the Model for Improvement and the strategies you tested next. As part of your participation in the BSC, your team used the Model for Improvement to translate the Collaborative Change Framework into testable strategies and keep focused on your mission and aim of successfully engaging fathers and paternal relatives.

F. How did the team determine the domains to address?

[Probe on the following if needed]

1. What were the key issues and challenges the team experienced when choosing which domains to address? How did you address these key issues and challenges?

G. How did the Improvement Team develop the engagement strategies it implemented?

1. What were the key issues and challenges the team experienced when designing (or adapting) strategies? How did you address these challenges?

2. How did you select other staff, who were not involved in the Improvement Team, to help test and implement the strategies?

3. How do the engagement strategies you tested and implemented over the course of this BSC compare with strategies you used to engage fathers and paternal relatives before the BSC??

4. What resources (infrastructure and supports) from [the agency] and/or its partners do you have to implement the BSC? What additional resources would have been helpful to implement the BSC?

H. How did the Improvement Team develop the data collection plan to track changes in engagement?

[Probe on the following if needed]

5. What were the key issues and challenges the team experienced when developing the data collection plan? How did you address these challenges?

6. What resources (infrastructure and supports) from [the agency] and/or its partners do you have to implement data collection plan? What additional resources would have been helpful to implement the data collection plan?

Let’s talk about learning from other sites. The shared learning environment is intended to inspire sharing of information for rapid improvement in how teams engage fathers and paternal relatives.

I. What is most useful to you about the opportunities you have had to learn from the other sites and the faculty coaches in the BSC? What could be better?

[Probe on the following if needed]

1. What is most useful about the Learning Sessions? What could be better?

2. What is most useful about the Faculty Coaches? What could be better?

3. What is most useful about the Affinity Group calls? What could be better?

4. What is most useful about the collaborative SharePoint site? What could be better?

Thank you for sharing your thoughts—your input is invaluable. As we close our discussion, we are interested in your reflections on the BSC’s impact on your agency and the types of resources that would have made it more helpful to you.

J. How has [agency’s] approach to engaging fathers and paternal relatives changed since you began participating in the BSC?

[Probe on the following if needed]

1. In your opinion, which aspects of participating in the BSC are most rewarding? Which are most challenging?

2. What types of guidance or additional information would have been helpful to you as you implemented the BSC?

Thank you again for sharing your experiences with us today.