3 PSFGP Codefinition Phase 1 Group 2

OMB No: 0906-XXXX

Expiration Date: XX/XX/20XX

Public Burden Statement: According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0906-XXXX. This information collection will be used to assess nominated ECHV programs’ interest in and eligibility for participation in the Assessing and Describing Practice Transitions Among Evidence-Based Home Visiting Programs in Response to the COVID-19 Public Health Emergency (ADAPT-HV) study. The time required to complete this information collection is estimated to average less than 1.5 hours per response, including the time to review instructions, search existing data resources, gather the data needed, to review and complete the information collection. This information collection is voluntary and confidentiality is followed according to law. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: HRSA Information Collection Clearance Officer, 5600 Fishers Lane, Room 14N39, Rockville, MD or [email protected].

Program Staff Focus Group Protocol

Phase 1: Co-Definition, Focus Group 2

Thank you for taking the time to talk with us today. My name is [NAME] and this is [co-facilitator/notetaker NAME(s)]. We work as research analysts with The Policy & Research Group (PRG). As a reminder, we are implementing a project designed to identify, develop, study, and disseminate evidence-informed strategies and resources that early childhood home visiting (ECHV) programs may use to strengthen home visiting services, and, ultimately, achieve better outcomes for children and families. This project is being implemented on behalf of the Health Resources and Services Administration (HRSA), and in collaboration with the Office of Planning, Research, and Evaluation (OPRE) in the Administration for Children and Families (ACF).

In our last discussion, we talked about coaching strategies your program has implemented to guide caregivers on how to care for or positively interact with their child. As a reminder, coaching refers to a practice in which the home visitor supports caregiver-led, child-directed skills or activities through verbal guidance or direction. This is in contrast to other approaches such as modeling strategies, when the home visitor demonstrates skills or activities while families observe. We want to identify and discuss strategies that enable more effective uses of coaching –when that technique is appropriate–in order to improve services.

The goal of our discussion today is to identify and define specific strategies your program would like to implement and test over the next few weeks. I will begin by reviewing the information we gathered during our first focus group in Phase 1 of this project: Co-Definition. Then, I will lead you through a brainstorming activity. Lastly, I will ask about your thoughts on which strategies you’d like to move forward with in this study and the parameters for defining and implementing these strategies.

Your participation in our discussion today is completely voluntary, and you can leave the call at any time. There is no penalty if you choose not to participate in this focus group. I may ask you a question directly, but you don’t have to answer anything you don’t want to. This is just an effort to make sure we hear from everyone today. Please know that the information you share today is confidential and only study staff will have access to the notes, recordings, and transcripts. Study staff may summarize what we discuss today to share with interested parties (e.g., staff in your program, HRSA, or other programs participating in the study), but no identifiable information or individual responses will be shared. We ask you to please respect each other’s privacy by not sharing any personal information or experiences discussed during the focus group.

I’m going to record our discussion today so that we can make sure we accurately capture what is said (that we remember the discussion correctly). By joining the discussion, you are agreeing to participate and to have the discussion recorded. Does anyone have any questions before I start recording? [Answer questions.]

As we are getting things set up to record and get our discussion underway – I’m going to send out a link to a quick survey in the Zoom Chat [or via email, if using phone]. The survey asks you to answer a few questions about yourself, so we have an understanding of who participated in our discussions. Like your participation in this discussion, filling out the survey is voluntary. You do not have to respond to all or any of the questions. Your name is not on the survey and will not be connected to your responses. By filling out the survey you are agreeing to allow the study team to see your responses and use your answers to describe (summarize) who was involved in the focus groups we conducted. Is everyone able to click on the link? [troubleshoot as necessary]. Great. If you are willing to fill out the survey, please take a few minutes to do so now and we’ll get started in [5 minutes, at TIME].

Ok. We are set up and I’m going to begin recording now. [Start to record.] Before we get started, I’d like us to set a few ground rules for the conversation.

[If using Zoom]

Since we are using Zoom today, I’ll keep my video on the whole time, but you can choose to have your video on or off. We want to make sure that everyone has the opportunity to have their thoughts and opinions heard. So, to avoid us talking over each other, I’ll be helping to direct the conversation. If you want to speak, you can let me know in the chat, click the “raise hand” button, or just chime in when there is a pause in conversation. Also, I want to emphasize that there are no right or wrong answers and we are interested in hearing both positive and negative comments and opinions – or whatever you think is important to share. We only ask that you do not use the names of clients or staff outside of this focus group to help maintain everyone’s privacy. Those are all of the guidelines I have - does anyone have additional guidelines for our conversation that you would like us to consider? Does anyone have any questions about Zoom or the guidelines for our discussion? [Confirm that everyone understands Zoom features and can mute/unmute.]

[If using Phone]

We want to make sure that everyone has the opportunity to have their thoughts and opinions heard. So, to avoid us talking over each other, I’ll be helping to direct the conversation. If you want to speak, you can chime in when there is a pause in conversation, otherwise, before we move on to different topics, I will provide an opportunity for people to voice their opinions. Also, I want to emphasize that there are no right or wrong answers and we are interested in hearing both positive and negative comments and opinions – or whatever you think is important to share. We only ask that you do not use the names of clients or staff outside of this focus group to help maintain everyone’s privacy. Those are all of the guidelines I have - does anyone have additional guidelines for our conversation that you would like us to consider? Does anyone have any questions about the guidelines for our discussion? [Confirm that everyone understands how to mute/unmute.]

In all, this should take about an hour to an hour and a half. Does anyone have any questions before we get started? [Answer questions.]

[Facilitator reviews available information gathered during cycle (e.g., initial staff focus group, focus groups with families, administrative data). At least one week before meeting, the facilitator provides a list of the strategies discussed by the site, and also strategies discussed by other sites or discussed in literature that meet the minimum definition set for the project. During the course of the Co-definition phase, additional parameters may be identified and placed on engagement strategies that improve their feasibility to implement and study within the context of this project.]

[The list created for review will be organized as a table with columns for type (purpose or intent), which will be determined after the list is created The facilitator will provide the list of strategies to all participants via email, prior to the discussion. Additionally, if the discussion occurs via Zoom, the facilitator will use the Whiteboard function or similar feature to share the list so everyone can see it.]

Part 1: Brainstorming Activity

Now let’s transition to the brainstorming activity. We’ll start by reviewing a list of coaching strategies your program has used, as well as strategies used by other home visiting programs.

During our last focus group, your program generated the following list of strategies that you would like to consider. [Display list.] In addition to these, when we sent the strategies list to you to review before our discussion today, we included strategies from other sites as well. Are there any strategies from other sites that you would like to add to your program’s list to be considered for this study?

[Before moving on, make sure everyone has time to share and that all identified strategies of interest are noted and the list is final.]

Okay, this is our list of all the potential strategies to study. Next, we will discuss each strategy, with the goal of refining the list and eventually deciding which strategy we will implement as part of this study. We would like to focus on one strategy so we can clearly implement, study, and refine it throughout the study cycles. Any questions? Let’s begin.

[If using Zoom, show the list in Whiteboard, or similar Zoom feature.]

First, take a look at the list of strategies. I’m going to send out a poll through [Zoom Feature] to get an understanding of your familiarity and experiences with each. I’d like to ask each of you to take the next [3-4 minutes], and for each strategy, first indicate if you have used it [yes/no answer], then indicate how you feel about the strategy based on your experiences with or knowledge of it [mostly positive, neutral, mostly negative responses.]

When you’re deciding what to put next to each strategy, what we are interested in is your experience and opinion. There is no right or wrong answer. You can think about things like how much of a hassle it is to use or how you think it affects the quality of the visit and the experiences of families. If it’s a strategy you haven’t used before, consider what your initial reaction is to the idea.

[Wait until it appears all responses have been received.]

Now, let’s reflect on the activity we just completed.

[If phone]

Please make sure you have a pen/pencil and something to write on. We will walk through the activity together.

First, take a look at the list of strategies. Of these, which are you familiar with, either personally or professionally? For example, which have you used or heard of someone else using? Write the names of the strategies down on the paper in a list. Are there any new strategies on the list you haven’t used but which look promising to you? Write those down too.

Next think about what you know about the experiences you and the families you work with have had with each strategies. For each strategy list, indicate how you feel about the strategy based on these experiences. Use a smiley face for services or resources you feel mostly positive about, a sad face for ones you feel mostly negative about, and a neutral face you feel neutrally or have mixed feelings about.

When you’re deciding what to put next to each strategy, what we are interested in is your experience and opinion. There is no right or wrong answer. You can think about things like how much of a hassle it is to use or how you think it affects the quality of the visit and the experiences of families. If it’s a strategy you haven’t used before, consider what your initial reaction is to the idea.

Now I’m going to go around the group and ask you to tell me whether you have had any experience with it and what you think about it (mostly positive, neutral, mostly negative).

[Starting with the first strategy on this list, go through the list of the strategies one by one and tally answers.]

[Reorganize the list using Zoom feature or table feature available on personal computer. Within each column (for each strategy type), list the strategies in descending order from those receiving most positive to those receiving most negative ratings/assessments. Create separate sections (rows) in the table to delineate strategies that received all positive, mixed, and all negative ratings. If conducted over the phone, read the list aloud to participants.]

I’ve reordered the original list to reflect how you have reviewed or assessed the strategies. Overall, it looks like [point out strategies] received more positive reviews; [point out strategies] received mixed reviews, and [point out strategies] received more negative reviews.

To make the most of our time today, we will limit our discussion to the four strategies with the most positive ratings. We will discuss each of these strategies one by one. I will ask specific questions about each one to make sure we all have an understanding of how and when it works best, the challenges or barriers to each, and what the intended outcomes of the strategy are.

[Ask the following questions for each strategy on the list.]

Question: How many people have used [strategy] before?

[If no one has personal experience with the strategy, skip to the additional questions for new strategies indicated by asterisks in the next section.]

Question: Do you think your program has the capacity to implement [strategy]?

• Probe: addition to home visitors work/caseload; accessibility to all home visitors; other resources that would be needed

Question: Is there anything else we should consider or know about [strategy]?

[Ask the questions for new strategies indicated by an asterisk below, as needed. If there are no added strategies, skip this section and move to the next page, (Selecting top choice for “to be considered” list).]

*Question: Can you tell me why you wanted to include [strategy] on your longer list?

• Follow-up: What appealed to you about [strategy]?

*Question: How do you think [strategy] would work at your site?

• Follow-up: What would implementing it look like for home visitors, families, and program staff?

• Follow-up: Do you think [strategy] will be easy to implement? Why or why not?

• Follow-up: What do you think might be some challenges to implementing [strategy]?

*Question: How do you think families/children will feel about [strategy]?

*Question: What is [strategy] intended to do/help with?

*Question: What would [strategy] require to be implemented in this study?

• Probe: materials required; additional training or support; more time to implement; additional costs associated

*Question: Do you think your program has the capacity to implement [strategy]?

• Follow-up: What will this add to the home visitors work/case load?

• Follow-up: What would be needed? (time, resources, training)

• Follow-up: Is [strategy] accessible to all home visitors? (for example, some may be less comfortable doing virtual visits, some families might not be eligible for the study, etc.)

*Question: Is there anything else we should consider or know about [strategy]?

[Selecting top choice for “to be considered” list.]

[If focus group members are all on their own computers] Now that we have discussed each strategy, I will ask everyone to write down their top choice for which of these strategies to add to the final “to be considered” list. Please send these to me privately in the group chat. Does anyone have any questions about how to send me a private message?

[If focus group members are all in one room together and not all on individual computers] Please write down your top two choices for strategies to add to the final “to be considered” list. You can pass these to [program administrator], without adding your name to the paper. [Program administrator], can you please read out the strategies everyone listed. We will make notes and generate a new list.

[Facilitator generates a new list of final “to be considered” list of strategies and shares this with the group in Whiteboard, or similar Zoom feature]

Question: Based on our discussion, this is the list of strategies you believe should be considered for testing [if phone, list out strategies verbally.]

• Follow-up: Does anyone have any reactions to this list that they would like to share?

• Follow-up: Is there anything final you would like to add or amend about this list?

Question: Could any of these strategies be combined?

[For all potential combinations of strategies]

• Follow-up: What would that look like?

• Follow-up: What are the benefits of combining these strategies?

• Follow-up: Would this combination of strategies significantly change the ask for home visitors and/or other staff?

• Follow-up: Are there any other considerations anyone can think of with this combination of strategies?

[In the eligibility screening calls, study staff will determine who is responsible for making the decision about the strategy or strategies that will be tested. Study staff will also ask this person(s) if they would like for the rest of the staff to remain on the call for the decision-phase of this focus group, or if they would like other staff to leave the call. The next part of the discussion will only include those who were previously identified for this part of the call. This could be only the individual(s) who will make the decision about what strategy/strategies to implement, a smaller subset of the whole group, or the whole group. If it is a subset of the full group, the facilitator will thank everyone else for their time and ask those individuals to stay on the call]

[If some participants will leave the call.]

This has been a very helpful brainstorming session. We appreciate everyone’s participation and thoughtful responses and ideas. At this time, we are going to go through the information we just discussed, and we will work with [leader’s name and title] to identify and define the parameters for the strategy(ies) your program will be testing in the Installation Phase of this project. I will then email you all an update based on that conversation.

Question: Before we wrap up this brainstorming activity, is there anything you would like to add that we have not discussed already?

Thank you again for your time. [individual(s) who will sign off on the strategies for implementation and the parameters for the strategies stay on call.]

Part 2: Strategy Selection

Question: Considering our conversation and what you know about your program, what is your main reaction or takeaway about the strategies your team/staff think are important to consider for testing?

Question: Which of these strategies are you most excited about?

• Follow-up: What is it about that/those strategies that interest(s) you?

Ok, now let’s take another look at the “short” list of strategies your team/staff agreed are best to consider and try to identify strategies of interest, discuss what you see as the most promising element(s) of an engagement strategy ,and try to come up with an initial understanding of a strategy or combination of strategies that are similar, that you want to test out in the Installation phase.

Question: Is there a specific strategy in this list that you think you may be interested in testing? If so, please explain. [If they can’t decide on one go to elaboration below.]

• Follow-up: What do you like about [strategy]?

• Follow-up: Do you think it will be feasible to implement?

• Follow-up: How well does it work with the program model you use?

• Follow-up: How well is it suited for the children and families you serve?

• Follow-up: How promising do you think it is for initiating/maintaining engagement, refocusing, or as a holistic approach to engagement?

• Follow-up: Are there any challenges that you might expect with these?

• Follow-up: What do you see as the benefits?

Question: Is this the strategy (or strategies) that you would like your program to test?

[If yes, go to Definition, starting on the next page. If no, continue with the Elaboration section below.]

Elaboration

Okay, so if you’re not fully certain of a specific strategy – let’s work to identify the elements of a strategy you’d like to test.

Question: Why would you say you are you having a hard time deciding?

• Probe: number of options, similarity of options, intended outcome, feasibility, scope of strategies

Question: What is most appealing or important for you – what do you want the strategy to accomplish or help with? Please explain.

• Probe: initial engagement, maintaining engagement, refocusing, parental involvement in engaging the child, managing the visit

Question: Are there any strategies or elements on the list that you think could accomplish this goal and meet your needs? If so, please explain.

Question: Based on what you are saying is most appealing, it appears that [list strategies] could be a good fit, perhaps with some adjustments to meet the needs of your program. Would you want to test out one (or two) of these strategies? If so, please discuss.

Question: Could any of these strategies be combined to create a more comprehensive or appealing strategy?

• Follow-up: What would that look like?

• Follow-up: What are the benefits of combining these strategies?

• Follow-up: Would this combination of strategies significantly change the ask for home visitors and/or other staff?

• Follow-up: Are there any other considerations anyone can think of with this combination of strategies?

Question: So, based on this conversation it sounds like you are interested in trying [strategy as discussed]?

[Once a strategy or combination of similar strategies have been identified as the one the program would like to implement, move on to Definition section below.]

Definition

[For this set of questions, keep in mind we need to make sure the definition is fully fleshed out – that we understand exactly what the home visitors will be asked to do, when they will be asked to do it, and what considerations they will need to make when deciding how/when to use the strategy]

Okay, now that we have a better understanding of the engagement strategy/strategies you are interested in testing, let’s work to define what this will look like in practice. You indicated you are interested in testing [type of strategy: initiation, maintenance, refocusing, combination, or holistic approach] that [describe what we know about the strategy, including main components and what home visitors will be asked to do.]

Question: Does this match your understanding, or should we adjust this definition?

• Follow-up: What adjustments should be made?

• Follow-up: What do you see as the main components of the strategy you want to test?

Question: To make sure we have a mutual understanding of the strategy, can you define the strategy in your own words?

• Follow-up: How will you explain [strategy] to your home visitors?

Question: How do you see this working in practice?

• Probe: guidance to provide for implementing; most important aspects; boundaries to be set; considerations home visitors will be asked to make

Question: Considering how you have defined the strategy and the parameters you have set for their use, are there any potential challenges for facilitators you see/expect for implementing these?

• Follow-up: What kind of guidance do you think your home visitors will need for dealing with potential challenges?

Question: Are there any specific trainings, materials, or considerations your home visitors need for implementing [strategy]?

• Follow-up: What is a plan for addressing these needs?

Question: What are the outcomes you expect to see from the implementation of [strategy]?

• Follow-up: What do you think “success” looks like for [strategy]?

Now that we have defined the strategy and have discussed guidance for implementing the strategy in the Installation phase. We will write up the details of how the strategy is initially defined and how it will be implemented. I will send you those details to review, then project staff will work with your staff (those selected to test the strategies) to make sure they understand what they will be doing and how they will be participating in the study.

Part 3: Wrap-up/Final Questions

[This section is the closing of the focus group and is used to make sure we address any outstanding questions and ensure we ask if anyone has anything additional to share before we end the focus group.]

Question: Are there any questions about the next steps in this process?

Question: In closing, is there anything else you’d like to share?

Thank you very much for participating in this discussion!

Our next focus group will be [provide timing] and will be Phase 2 (Installation) of the study process where we will provide [findings about X] and discuss your reactions and reflections to these findings and the implementation process. In between now and then we will solidify the plans for this study, ensure your team has what they need to implement the strategy, and begin implementation with cycle one of the study cycles. If you have questions after you leave the call, feel free to contact me.

If you have any questions about the study, please contact Teresa Smith at The Policy & Research Group ([email protected] or (225) 281-3783).

If you have questions about your rights as a research volunteer, you can call Health Media Lab Institutional Review Board at (202) 549-1982.

The Policy & Research Group 11