2 0906-0092 2_Program Staff Focus Group Protocol 1-Codefin

OMB No: 0906-0095

Expiration Date: 7/31/2027

Public Burden Statement: According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The OMB control number for this information collection is 0906-0095 and it is valid through 7/31/2027. This information collection will be used to assess nominated ECHV programs’ interest in and eligibility for participation in the Assessing and Describing Practice Transitions Among Evidence-Based Home Visiting Programs in Response to the COVID-19 Public Health Emergency (ADAPT-HV) study. The time required to complete this information collection is estimated to average less than 1.5 hours per response, including the time to review instructions, search existing data resources, gather the data needed, to review and complete the information collection. This information collection is voluntary and confidentiality is followed according to law. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: HRSA Information Collection Clearance Officer, 5600 Fishers Lane, Room 14N39, Rockville, MD or [email protected].

Program Staff Focus Group Protocol

Phase 1: Co-Definition, Focus Group 1

Thank you for taking the time to talk with us today. My name is [NAME] and this is [co-facilitator/notetaker NAME(s)]. We work as research analysts with The Policy & Research Group (PRG). On behalf of the Health Resources and Services Administration (HRSA), and in collaboration with the Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF), we are implementing a project designed to identify, develop, study, and disseminate evidence-informed strategies and resources that early childhood home visiting (ECHV) programs may use to strengthen home visiting services, and, ultimately, achieve better outcomes for children and families. The focus of our discussion today is coaching strategies your program has implemented to guide caregivers on how to care for or positively interact with their child.

We are particularly interested in hearing about your experiences with trying to coach caregivers on caring for and interacting with their children, including the strategies you use and why, and your perception of the benefits and challenges to these strategies. Coaching refers to a practice in which the home visitor supports caregiver-led, child-directed skills or activities through verbal guidance or direction. This is in contrast to other approaches such as modeling strategies, when the home visitor demonstrates skills or activities while families observe. We want to identify and discuss strategies that enable more effective uses of coaching –when that technique is appropriate–in order to improve services.

Your participation in our discussion today is completely voluntary, and you can leave the call at any time. There is no penalty if you choose not to participate in this focus group. I may ask you a question directly, but you don’t have to answer anything you don’t want to. This is just an effort to make sure we hear from everyone today. Please know that the information you share today is confidential and only study staff will have access to the notes, recordings, and transcripts. Study staff may summarize what we discuss today to share with interested parties (e.g., staff in your program, HRSA, or other programs participating in the study), but no identifiable information or individual responses will be shared. We ask you to please respect each other’s privacy by not sharing any personal information or experiences discussed during the focus group.

I’m going to record our discussion today so that we can make sure we accurately capture what is said (that we remember the discussion correctly). By joining the discussion, you are agreeing to participate and to have the discussion recorded. Does anyone have any questions before I start recording? [Answer questions.]

As we are getting things set up to record and get our discussion underway – I’m going to send out a link to a quick survey in the Zoom Chat [or via email, if using phone]. The survey asks you to answer a few questions about yourself, so we have an understanding of who participated in our discussions. Like your participation in this discussion, filling out the survey is voluntary. You do not have to respond to all or any of the questions. Your name is not on the survey and will not be connected to your responses. By filling out the survey you are agreeing to allow the study team to see your responses and use your answers to describe (summarize) who was involved in the focus groups we conducted. Is everyone able to click on the link? [troubleshoot as necessary]. Great. If you are willing to fill out the survey, please take a few minutes to do so now and we’ll get started in [5 minutes, at TIME].

Ok. We are set up and I’m going to begin recording now. [Start to record.] Before we get started, I’d like us to set a few ground rules for the conversation.

[If using Zoom]

Since we are using Zoom today, I’ll keep my video on the whole time, but you can choose to have your video on or off. We want to make sure that everyone has the opportunity to have their thoughts and opinions heard. So, to avoid us talking over each other, I’ll be helping to direct the conversation. If you want to speak, you can let me know in the chat, click the “raise hand” button, or just chime in when there is a pause in conversation. Also, I want to emphasize that there are no right or wrong answers and we are interested in hearing both positive and negative comments and opinions – or whatever you think is important to share. We only ask that you do not use the names of clients or staff outside of this focus group to help maintain everyone’s privacy. Those are all of the guidelines I have - does anyone have additional guidelines for our conversation that you would like us to consider? Does anyone have any questions about Zoom or the guidelines for our discussion? [Confirm that everyone understands Zoom features and can mute/unmute.]

[If using Phone]

We want to make sure that everyone has the opportunity to have their thoughts and opinions heard. So, to avoid us talking over each other, I’ll be helping to direct the conversation. If you want to speak, you can chime in when there is a pause in conversation, otherwise, before we move on to different topics, I will provide an opportunity for people to voice their opinions. Also, I want to emphasize that there are no right or wrong answers and we are interested in hearing both positive and negative comments and opinions – or whatever you think is important to share. We only ask that you do not use the names of clients or staff outside of this focus group to help maintain everyone’s privacy. Those are all of the guidelines I have - does anyone have additional guidelines for our conversation that you would like us to consider? Does anyone have any questions about the guidelines for our discussion? [Confirm that everyone understands how to mute/unmute.]

In all, this should take about an hour to an hour and a half. Does anyone have any questions before we get started? [Answer questions.]

Part 1: Program Setting/Background

[This section builds our baseline understanding of the program and its use of strategies for coaching. This section also addresses primary research question 1: What coaching strategies do home visitors use to improve service delivery and promote positive caregiver-child interactions?]

During our previous conversations when we were identifying programs to participate in this study, you provided information about your program and how it operates. I’d like to briefly review that information with you and discuss any additional cultural, community, or program-specific contexts that we should be aware of and that influence your programming.

[Facilitator quickly reviews information on home visiting model used by the program, the setting and target population for the program, services offered to families, program capacity, current program staffing and caseload, and use of virtual service delivery. Ensure that we know:

• population served # home visitors on staff and serving families

• # families enrolled and receiving services

• average or typical case load for home visitors

• frequency of/circumstances for virtual visits

• Important context for providing services, especially virtually, as discussed by staff]

Question: Is all of this information, correct? If not, what needs to be amended?

Question: Is there any other context we should know about how your program works?

• Probe: cultural, community, or program-specific contexts

• Follow-up: Is there anything else we should know about your program structure or service capacity as we start to think about strategies related to coaching?

Question: What strategies do home visitors use to ensure being able to see the family’s environment and the child during virtual visits?

• Probe: guidance given to families to help them position their video

Part 2: Coaching Strategies Implemented

[This section addresses primary research question 1: What coaching strategies do home visitors use to improve service delivery and promote positive caregiver-child interactions? This section is intended to provide a shared foundational knowledge of the strategies used and how they have been implemented by the program.]

Thank you for going over information about your program. Now I am interested in hearing about strategies your program uses or has used to try and coach caregivers, when they are used, and how you use them.

Again, when I talk about coaching strategies, I’m referring to a practice during the home visit when a home visitor provides verbal guidance to a caregiver on specific ways to care for or positively interact with their child. Coaching is intended to build caregiver parenting skills by having the caregiver undertake a specific positive parenting behavior as the home visitor observes and provides feedback to support the caregiver in adequately or accurately exhibiting the skill of focus.

Question: Does anyone have any questions about how we are defining coaching?

Over the course of our discussion, we want to hear about what you have been doing and we will work to understand how your experiences and practices align with the strategy definition we are using for this study. The ultimate goal of the conversation will be to come up with a list of strategies that that your program may be interested in testing during this study and that reflect/encompass the study definition.

Question: Okay. Now that we have discussed our working definition of coaching for this study, I’d like to hear about all of the ways that your program has used coaching with families, when it’s appropriate, to improve the services you provide. When have you used coaching and to what extent have you used different coaching approaches or strategies?

• Probe: caregiving skills; caregiver communication; caregiver -child communication

• Follow-up: Are there any other strategies anyone wants to add?

[Facilitator makes a list as people share, then asks the following questions about the list of strategies broadly.]

Question: What has been the outcome of using [strategy] in your experience (what does it do)?

• Probe: strategy meets following intention – undertake a specific positive parenting behavior as the home visitor observes and provides feedback.

Question: What does [strategy] require in terms of:

• Training/guidance/technical support

• Time for oversight/ongoing guidance

• Time to implement

• Materials

• Rapport or relationship with the family

Question: Considering what [strategy] requires - how practical or feasible is it for your program and home visitors to use this strategy?

• Probe: ease or difficulty for program to provide the training/resources needed to implement; ease or difficulty for home visitors to implement

Question: What guidance is there on this coaching strategy from your home visiting model?

Question: How well does [strategy] fit with the home visiting model that your program uses?

• Follow-up: Are there any issues related to [strategy] and your home visiting model that should be considered?

Question: How accepted is [strategy] by home visitors, administrators, and families?

• Probe: who likes/dislikes it; how well liked or received is it; what is liked/disliked; response of families (negative/positive)

Question: How applicable and appropriate is [strategy] to/for all families and home visits?

• Follow-up: For whom, and in what instances, does [strategy] work best?

  • Probe: Content or purpose of visit, age of children, number of children, mode of visit; rapport between home visitor and family; length of time families have been in the program

• Follow-up: For whom, and in what instances, does [strategy] not work well?

Question: How does [strategy] differ when used in in-person home visits versus virtual visits?

Great. Now, let’s talk about how your practices have changed over time and what you are doing now.

Question: To what extent did home visitors use coaching before the COVID-19 public health emergency (PHE) and how has your use of coaching changed over time?

• Follow-up: How did this change during the PHE?

  • Probe: restrictions on in-person contact; frequency of contact; content of communication; method of communication

Question: How have home visitors applied virtual coaching strategies to in-person services?

• Follow-up: In what situations do you use these strategies with in-person services?

• Follow-up: To what extent are these strategies applicable/adaptable to in-person visits?

Question: How has your/your program’s use of coaching been going recently?

• Follow-up: Can you provide some examples of how you’ve used coaching?

Question: How do you decide when to use coaching?

• Follow-up: How often do you use it?

• Follow-up: How have you decided whether and what strategy to use?

• Probe: caregiver and child characteristics (e.g., age); content or purpose of what you were trying to teach; mode of visit; activity or information being shared; differences in family/child needs

Now that we have discussed the strategies you have used – let’s think bigger picture about the strategies we’ve discussed – especially in terms of how they compare to one another and how you make decisions about which specific strategies to use.

Question: Overall, which coaching strategies do you think work the best for both home visitors and families?

Question: Overall, which strategies do you think work the best for building caregiver skills?

Question: Overall, which strategies do you think have the most potential to be scaled up/implemented at other programs?

Question: Considering the list of strategies we just discussed, can you describe any shared characteristics of these different practices?

• Probe: How and when they are used; purpose; content; expected/intended outcomes

• Follow-up: Can you describe any key differences you see in these practices?

  • Probe: How and when they are used; purpose; content; expected/intended outcomes

Question: Is there anything else anyone wants to share about their experiences using the strategies we’ve been discussing?

[Before moving on to the next question, make sure everyone has had the opportunity to voice their opinions.]

Part 3: Facilitators of and Challenges to Implementation

[This section addresses barriers and facilitators to implementation and helps to respond to primary research question: 2. How can the implementation of coaching strategies be improved? especially sub question 2a.: what facilitated implementation of these strategies? What are the barriers to implementation?

In this section, I would like you to reflect on the strategies we’ve been discussing and share where the barriers and challenges have been. Are there general or program specific barriers to these strategies? I also would like to hear about how your program has succeeded in implementing these strategies and if there are tips or specifics you can think of regarding how to effectively implement coaching during home visits.

Question: Considering the coaching strategies you’ve used [or have recently been using], what barriers or challenges have come up when you’ve used them?

Question: When is coaching easier to implement? Please explain.

• Probe: the setting, family dynamics, coaching on different topics

Question: When is coaching harder to implement? Please explain.

• Probe: the setting, family dynamics, coaching on different topics

Question: Do you ever begin with coaching and switch to modeling?

• Follow-up: What prompted that switch? And can you think of any examples?

Question: If you use coaching in both in-person and virtual visits, are there differences or similarities in challenges you’ve experienced with coaching across settings?

Question: Are there any program-specific barriers or challenges you can think of?

• Follow-up: How good of a fit are each of the strategies for the home visiting model you use?

• Follow-up: Are there different challenges/barriers for coaching during in-person vs virtual home visits?

Question: What family/child/caregiver specific barriers or challenges have come up when implementing each of these strategies?

• Follow-up: Has the age of child or other child factors (e.g., attention, behavior) made implementation harder/easier?

• Follow-up: Has the number of family members present and/or participating in the visit made implementation harder/easier?

Question: Does your program do anything that facilitates or makes implementing each of the strategies easier or more successful?

• Probe: supports (guidance, props, activities) for maintaining child interest/involvement;

supervision of home visitors while using coaching strategies

Question: Considering all of the strategies you’ve tried, can you identify and discuss any challenges or barriers that are common to all or many of the strategies?

• Probe: similarities of barriers and challenges encountered

• Follow-up: Are there any challenges that are more unique or distinct or really only apply to specific strategies?

[Before moving on to the next question, make sure everyone has had the opportunity to voice their opinions.]

Part 4: Perceptions of Strategies

[This section addresses two primary research questions: 1. What coaching strategies do home visitors use to improve service delivery and promote positive caregiver-child interactions? And 2. How can the implementation of coaching strategies be improved?]

In this section I am interested in you sharing details about the strategies you’ve used to coach caregivers and how they have gone. I am interested in your perception of the effect of these strategies on children and families in areas such as caregiver-child interaction, engagement, and rapport.

Considering each of the strategies you’ve used, I’d like you to consider what you see as the utility of the strategy – or what you see as the function, the purpose, or value of the strategy, especially as it relates to caregiver skills, caregiver-child interaction, engagement, and enhancing home visiting.

Question: In your opinion, what is valuable about using coaching with families – what do you see as the benefit of using the strategies we’ve discussed?

• Probe: quality of home visit; quality of caretaker experience; rapport with child/caretaker; internalizing/understanding messages or information provided; satisfaction with program; home visit scheduling/attendance; length of time spent in program; participation in activities outside of home visits

[Facilitator makes a list of perceived benefits to reflect on in next set of questions.]]

Question: In your experience, which strategies would you say are most effective or promising, overall, with regards to the benefits we’ve discussed?

Question: Which strategies would you say are least effective, overall, with regards to the benefits we’ve discussed?

Now that we have an idea of how you see or understand the strategies to be working, I’d like to get a sense of how confident and comfortable you personally feel implementing these strategies.

Question: Which strategies do you feel most confident in your ability to implement?

• Probe: difference in level of confidence in virtual vs. in-person settings

Question: Which strategies do you feel least confident in your ability to implement?

• Probe: difference in level of confidence in virtual vs. in-person settings

Question: How do you feel about the guidance provided by your program and/or model for these strategies?

Question: Is there anything else you want to share about why you do or do not feel confident in your ability to implement a particular strategy?

[Before moving on to the next question, make sure everyone has had the opportunity to voice their opinions.]

Part 5: Wrap-up/Final Questions

[This section is the closing of the focus group and is used to make sure we address any outstanding questions and ensure we ask if anyone has anything additional to share before we end the focus group.]

Before we wrap-up, I want to reflect back on the strategies we’ve discussed and have you consider a “short list” that you are interested in testing or that you think other sites might be interested in testing during the study. Once we’ve talked to all of the sites participating in the study, we’ll send you a list of the strategies you identify here along with what other people are identifying as the most promising or appealing strategies We’ll need to make sure that the list of strategies aligns with or reflects the definition of coaching we discussed previously, so we may need to consider some adjustments to some of the strategies.

Question: Considering our conversation - the strategies we’ve discussed, what you see as the benefits and challenges of those – are there any strategies that you feel are really rising to the top for you – that you definitely think your program should consider testing during this study? If so, can you identify the strategy and discuss why you think it should be kept on the list of strategies to consider?

• Follow-up: By a show of hands who thinks it should be kept on the list. Now a show of hands for people who think it should be dropped [facilitator notes #s].

Question: Are there any strategies that you think definitely should not be considered? Please explain why you think it should be dropped from the list of strategies to consider.

• Follow-up: By a show of hands who thinks it should be kept on the list. Now a show of hands for people who think it should be dropped [facilitator notes #s].

[Facilitator identifies on the running list of strategies, which are to be included not to be included. If there are some strategies that are not identified, ask following question.]

Question: Okay, it looks like there are a few strategies that no one has identified as either “keep” or “drop.” [Facilitator goes through the list one strategy at a time.] For [strategy], let’s see a raise of hands for people who think it should be kept on the list. Now a show of hands for people who think it should be dropped.

Great. Now we need to decide which strategies will remain on the list to consider. I suggest we keep a strategy on the list if a majority of you agree it should remain on the list and no one believes it definitely should be dropped. I’ll go through the list again, just to ensure you all agree with this strategy.

Question: [For each strategy]: [#] people said keep and [#] said drop. I’m going to [keep/drop] based on this – unless you all want to talk it through a little further. Does anyone one to talk about this strategy? [Facilitator asks group to talk out any disagreements and decide amongst themselves whether a strategy should remain.]

Question: Thinking about the study definition [repeat definition if needed], and this list of strategies – are there any that you think we may need to adjust if we want to consider them? If so, please identify the strategy and let’s discuss what adjustments you think could or should be made.

• Follow-up: [facilitator goes through the core elements of the definition one at a time for instance: 1. practiced during the home visit; 2. home visitor provides verbal guidance to a caregiver on specific ways to care for or positively interact with their child; 3. coaching is intended to build caregiver parenting skills by having the caregiver undertake a specific positive parenting behavior as the home visitor observes and provides feedback; 4. the home visitor should not model the behavior (or show the caregiver what to do), but they may provide verbal guidance or direction as needed until the caregiver is able to adequately/accurately exhibit the skill of focus.] Does the strategy meet this part of the definition? If not, can you think of any elements of the strategy that would need to be added or adjusted?

Great. So, based on our conversation – this is the short list of strategies that you would like to consider for the study: [read list]. Again, we will include these along with strategies from other sites as we develop a final set of strategies that all sites can consider for the study. We will send this to you at least a week in advance of our next focus group so you have plenty of time to read through and discuss.

Question: In closing, is there anything else you’d like to share?

Thank you very much for participating in this discussion! If you have questions after you leave the call, feel free to contact me.

Our next focus group will be [provide timing] and will be the second focus group in Phase 1: Co-Definition, of the study process. Before we meet again, we will email everyone here a list of the different strategies discussed today. We will use this list as our starting point for the next focus group, during which we will decide, as a group, which strategy will be chosen for study implementation.

If you have any questions about the study, please contact Teresa Smith at The Policy & Research Group ([email protected] or (225) 281-3783).

If you have questions about your rights as a research volunteer, you can call Health Media Lab Institutional Review Board at (202) 246-8504.

The Policy & Research Group 11