Form 9 Staff Focus Group Installation and Refinement

OMB No: 0906-XXXX

Expiration Date: XX/XX/20XX

Public Burden Statement: According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0906-XXXX. This information collection will be used to assess nominated ECHV programs’ interest in and eligibility for participation in the Assessing and Describing Practice Transitions Among Evidence-Based Home Visiting Programs in Response to the COVID-19 Public Health Emergency (ADAPT-HV) study. The time required to complete this information collection is estimated to average less than 1 hour per response, including the time to review instructions, search existing data resources, gather the data needed, to review and complete the information collection. This information collection is voluntary and confidentiality is followed according to law. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: HRSA Information Collection Clearance Officer, 5600 Fishers Lane, Room 14N39, Rockville, MD or [email protected].

Staff Focus Group Protocol

Phase 2: Installation / Phase 3: Refinement

Thank you for taking the time to talk with us today. My name is [NAME] and this is [co-facilitator/notetaker NAME(s)]. We work as research analysts with The Policy & Research Group (PRG). On behalf of the Health Resources and Services Administration (HRSA), and in collaboration with the Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF), we are implementing a project designed to identify, develop, study, and disseminate evidence-informed strategies and resources that early childhood home visiting (ECHV) programs may use to strengthen home visiting services, and, ultimately, achieve better outcomes for children and families.

The focus of our discussion today is coaching strategies your program has recently implemented. As a reminder, coaching refers to a practice in which the home visitor supports caregiver-led, child-directed skills or activities through verbal guidance or direction. This is in contrast to other approaches such as modeling strategies, when the home visitor demonstrates skills or activities while families observe. We want to identify and discuss strategies that enable more effective uses of coaching –when that technique is appropriate–in order to improve services.

We are particularly interested in hearing about how things have been going and getting your reaction to the data collected thus far in the study. We will share our findings with you, ask for your thoughts on the data gathered thus far, and then ask questions about your experiences and opinions related to implementing the strategies your program is testing. We will conclude by reflecting on what is working and we will identify and discuss potential refinements that you think are needed to improve the strategy(strategies).

Your participation in our discussion today is completely voluntary, and you can leave the call at any time. There is no penalty if you choose not to participate in this focus group. I may ask you a question directly, but you don’t have to answer anything you don’t want to. This is just an effort to make sure we hear from everyone today. Please know that the information you share today is confidential and only study staff will have access to the notes, recordings, and transcripts. Study staff may summarize what we discuss today to share with interested parties (e.g., staff in your program, HRSA, or other programs participating in the study), but no identifiable information or individual responses will be shared. We ask you to please respect each other’s privacy by not sharing any personal information or experiences discussed during the focus group.

I’m going to record our discussion today so that we can make sure we accurately capture what is said (that we remember the discussion correctly). By joining the discussion, you are agreeing to participate and to have the discussion recorded. Does anyone have any questions before I start recording? [Answer questions.]

As we are getting things set up to record and get our discussion underway – I’m going to send out a link to a quick survey in the Zoom chat [or via email, if using phone]. The survey asks you to answer a few questions about yourself, so we have an understanding of who participated in our discussions. Like your participation in this discussion, filling out the survey is voluntary. You do not have to respond to all or any of the questions. Your name is not on the survey and will not be connected to your responses. By filling out the survey, you are agreeing to allow the study team to see your responses and use your answers to describe (summarize) who was involved in the focus groups we conducted. Is everyone able to click on the link? [Troubleshoot as necessary.] Great. If you are willing to fill out the survey, please take a few minutes to do so now and we’ll get started in 5 minutes, at [TIME].

Ok. We are set up and I’m going to begin recording now. [Start to record.] Before we get started, I’d like us to set a few ground rules for the conversation.

[If using Zoom]

Since we are using Zoom today, I’ll keep my video on the whole time, but you can choose to have your video on or off. We want to make sure that everyone has the opportunity to have their thoughts and opinions heard. So, to avoid us talking over each other, I’ll be helping to direct the conversation. If you want to speak, you can let me know in the chat, by clicking the “raise hand” button, or just chiming in when there is a pause in conversation. Also, I want to emphasize that there are no right or wrong answers and we are interested in hearing both positive and negative comments and opinions – or whatever you think is important to share. We only ask that you do not use the names of clients or staff outside of this focus group to help maintain everyone’s privacy. Those are all of the guidelines I have – does anyone have additional guidelines for our conversation that you would like us to consider? Does anyone have any questions about Zoom or the guidelines for our discussion? [Confirm that everyone understands Zoom features and can mute/unmute.]

[If using Phone]

We want to make sure that everyone has the opportunity to have their thoughts and opinions heard. So, to avoid us talking over each other, I’ll be helping to direct the conversation. If you want to speak, you can chime in when there is a pause in conversation; otherwise, before we move on to different topics, I will provide an opportunity for people to voice their opinions. Also, I want to emphasize that there are no right or wrong answers and we are interested in hearing both positive and negative comments and opinions – or whatever you think is important to share. We only ask that you do not use the names of clients or staff outside of this focus group to help maintain everyone’s privacy. Those are all of the guidelines I have – does anyone have additional guidelines for our conversation that you would like us to consider? Does anyone have any questions about the guidelines for our discussion? [Confirm that everyone understands how to mute/unmute.]

In all, this should take about an hour to an hour and a half. Does anyone have any questions before we get started? [Answer questions.]

Part 1: Coaching Strategies Implemented

[This section addresses primary research question 1: What coaching strategies do home visitors use to improve service delivery and promote positive caregiver-child interactions? This section is intended to provide a shared foundational knowledge of the strategies used and how they have been implemented by the program. In Phase 2 and 3, we present our findings from data gathered during the previous cycle and ask participants to reflect and discuss.]

To begin, I’d like to discuss how implementation has been going. I will start by going over the information we’ve gathered so far from your [weekly forms]. I will ask you to reflect on this and share if the findings match your experiences and understandings. Please feel free to add or correct this so we can reach a clear shared understanding of these practices and strategies.

[Facilitator reviews the data from forms and describes what we have learned about the strategies that were implemented]

Question: To what extent do these results reflect your experiences and understandings from this past cycle?

Question: Broadly, how has implementation of this strategy been going?

• Probe: differences in expectations for what was required to implement (e.g., time for training; time for oversight/ongoing guidance; time to implement; additional costs to consider)

Question: Generally, did the families seem to like [strategy] during this cycle? What about home visitors and other staff?

Question: Based on your experience during this cycle, do you think the strategy is appropriate for all families? All home visits?

• Follow-up: For whom, and in what instances, did this strategy work best?

  • Probe: Content or purpose of visit; age of children; number of children; mode of visit; rapport between home visitor and family; length of time families have been in the program

• Follow-up: For whom, and in what instances, does this strategy not work well?

Question: Would you say [strategy] was implemented as you planned? Why or why not?

[Before moving on to the next question, make sure everyone has had the opportunity to voice their opinions.]

Part 2: Facilitators of and Challenges to Implementation

[This section addresses barriers and facilitators to implementation and helps to respond to primary research question: 2.How can the implementation of coaching strategies be improved? and sub question 2.a.: What facilitated implementation of these strategies? What are the barriers to implementation?

In this section, I would like you to reflect on the strategies we’ve been discussing and share where the barriers and challenges have been. Are there general or program specific barriers to these strategies? I also would like to hear about how your program has succeeded in implementing these strategies and if there are tips or specifics you can think of regarding how to effectively implement coaching during home visits.

Question: What barriers or challenges have come up when you’ve used [strategy]? Try to focus on barriers to implementing [strategy] rather than barriers to completing home visits in general.

• Probe: the setting; characteristics of family; family dynamics; coaching on different topics; fit with model; time needed to implement

Question: When has coaching been easier to implement? Please explain.

• Probe: the setting; family dynamics; coaching on different topics

Question: When has coaching been harder to implement? Please explain.

• Probe: the setting; family dynamics; coaching on different topics

Question: Did you ever have instances when you began with coaching and had to switch to modeling?

• Follow-up: If you can think of an example of this, what prompted that switch?

Question: If you used coaching in both in-person and virtual visits, did you experience similar challenges or barriers? Please explain.

Question: Is there anything specific to your program that has facilitated or makes implementing tested strategy/strategies easier or more successful?

• Probe: necessary supports (guidance, props, activities) for maintaining child interest/involvement while you are coaching; supervision of home visitors using coaching strategies; supports that make coaching in virtual visits more effective/easier (e.g., guidance for parents on how to position their camera)

[Before moving on to the next question, make sure everyone has had the opportunity to voice their opinions.]

Part 3: Perceptions of Strategies

[This section addresses two primary research questions: 1. What coaching strategies do home visitors use to improve service delivery and promote positive caregiver-child interactions? And 2. How can the implementation of coaching strategies be improved?]

In this section I am interested in you sharing details about the strategies you’ve used to coach caregivers and how they have gone. I am interested in your perception of the effect of these strategies on children and families in areas such as caregiver-child interaction, engagement, and rapport.

Considering the strategy/strategies you’ve chosen and begun implementing, I’d like you to consider what you see as the utility of the strategy – or what you see as the function, the purpose, or value of the strategy, especially as it relates to caregiver skills, caregiver-child interaction, engagement, and enhancing home visiting.

Question: Generally speaking, what do you think the benefits of using [strategy] you’ve been testing have been? What about drawbacks?

• Probe: [list of benefits discussed in first focus group during co-definition phase]; positive caregiver-child interaction; child engagement; quality of home visit; quality of caretaker experience; rapport with child/caretaker; caretaker internalizing/understanding messages or information provided; caretaker demonstration of new knowledge and skills; child development; child safety/maltreatment; child wellbeing

Question: Considering the strategies you’ve used: How do you think they have affected caregiver skills?

• Follow-up: What types of skills or behaviors have you been able to teach through coaching? Are there some skills/behaviors that are harder to coach?

  • Probe: impact on caregivers’ interactions with their children; caregivers’ confidence in skills; rapport between home visitor and caregiver; children’s interest or engagement in the home visit; [other goals identified during co-definition focus groups]

Question: Has [strategy] impacted engagement in virtual home visits? What about in-person home visits?

• Follow-up: Have you noticed anything about how children engage during and after coaching?

• Follow-up: Have you found any difference in children’s engagement/interest when using coaching strategies?

  • Probe: impact on children’s eagerness to participate when their parents work directly with them; impact on caregiver–child interactions

Now that we have an idea of how you see or understand the strategies to be working, I’d like to get a sense of how confident and comfortable you personally feel implementing these strategies.

Question: How confident are you in your ability to implement [strategy] effectively during home visits?

• Follow-up: What are the reasons you feel more or less confident when using [strategy]?

Question: Do you feel you can follow the guidance provided by your program for [strategy]?

• Follow-up: When you’ve used [strategy], are there any aspects of the guidance that you have had to change to make it work?

[Before moving on to the next question, make sure everyone has had the opportunity to voice their opinions.]

Part 4: Strategy Refinements

[This section addresses the primary research question: How can the implementation of coaching strategies be improved?]

In this section, I will ask you to share your thoughts on how this strategy could be refined to be more effective and/or easier to implement. First, I’m going to review the guidance you currently have for your strategy and the details for how it is being implemented for the purposes of this study, in your program.

[Facilitator briefly reviews guidance]

Question: Thinking about this guidance and how you are implementing [strategy], are there any specific refinement or improvements to what you are doing that you think should be considered?

• Probe: who is implementing; characteristics of families/children used with; when, why, or how it is implemented

• Follow-up: At this point, [X] number of people from your program are involved in the study. Does anyone else have an interest in joining the study?

Question: Now, please reflect on when you have used this strategy during this study and found it most effective versus less effective. What are the differences?

Question: Considering what has made it more or less effective, are there any adjustments or refinements that you think are needed to [strategy] or your guidance to improve consistency and/or effectiveness of implementation?

• Probe: who is implementing; characteristics of families/children used with; when, why, or how it is implemented; specific components of [strategy]; home visit content/purpose

[Facilitator shares challenges or barriers to implementation discussed during this focus group and approaches home visitors are taking to address them.]

Question: It sounds as though [________] are shared barriers across multiple people and [________] are some approaches people are using to address these. Is there interest in refining the study to reflect these?

• Follow-up: Are there any other refinements or adjustments that you think may be needed to address ongoing challenges?

[If they indicate they will make refinements]

Question: We have just discussed several potential refinements to be made [facilitator reviews refinements discussed]. What specific refinements do you want to make to [strategy] or guidance to implement and test in the next cycle of this study?

• Follow-up: How will you define or describe the refinements for implementation purposes?

• Follow-up: What are the main components that make these functional/successful?

• Follow-up: What guidance will you provide for implementing the refinements?

• Follow-up: What considerations will home visitors be asked to make?

• Follow-up: What is needed by the program to make these changes?

  • Probe: staff time; training; physical aids

[Facilitator summarizes what has been discussed]

Question: Does this match your understanding, or should we make adjustments?

• Probe: adjustments to be made

We will write up the details of how the refinements are defined and how they will be implemented based on our conversation. I will send you those details to review, then project staff will work with your staff (those selected to test the strategies) to make sure they understand what they will be doing and how they will be participating in the study in the next phase.

[If they indicate they will not make refinements]

Question: Based on our discussion, it is my understanding that you are not interested in making any refinements to the strategy or guidance. Is that correct? [If yes, continue. If no, go back to the section above.]

Question: Okay, can you tell me why you think refinements are not needed?

Great. Thank you for sharing that. Since you will not be making any refinements, in the next cycle of testing, we will ask that you continue testing the strategy as you initially defined it.

Part 5: Wrap-up/Final Questions

[This section is the closing of the focus group and is used to make sure we address any outstanding questions and ensure we ask if anyone has anything additional to share before we end the focus group.]

Question: In closing, is there anything else you’d like to share?

Thank you very much for participating in this discussion! If you have after you leave the call, feel free to contact me.

Our next focus group will be [provide timing] and will be [Phase 3: Refinement OR Phase 4: Summary] of the study process where we will provide [findings about X] and discuss your reactions and reflections to these findings and the implementation process.

If you have any questions about the study, please contact Teresa Smith at The Policy & Research Group ([email protected] or (225) 281-3783).

If you have questions about your rights as a research volunteer, you can call Health Media Lab Institutional Review Board at (202) 549-1982.

The Policy & Research Group 8