Instrument 5_EA Focus Group Guide for Nominated Program Staff

Instrument 5:

Evaluability Assessment Focus Group Guide for Nominated Program Staff

Activity 3: NOMINATED PROGRAM STAFF

Introduction

Thank you for talking with us today about your program. We are researchers from [Westat, an independent research firm located in Rockville, MD / the Kempe Center for the Prevention and Treatment of Child Abuse and Neglect, Department of Pediatrics, School of Medicine University of Colorado]. We have been contracted by the Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families (ACF) to conduct an evaluation of Chafee programs serving youth in or transitioning out of foster care (called the Chafee Strengthening Outcomes for Transition to Adulthood [Chafee SOTA] Project).

 

During the conversation today, we are interested in collecting information about how your program works, what its goals and activities are, and how it has evolved over time. At this time, we are not conducting an evaluation that will make conclusions about whether the program is working or not working. Please note that this interview will focus on your program overall and will not ask for information on individual clients.

Your participation in this focus group is voluntary and it should take about an hour and a half to complete. In addition, you do not have to answer any questions you do not want to answer. By being here today, you are giving your consent to participate and that you are here voluntarily.

We will be recording this session for analysis purposes; it also helps us with accurate note-taking. If you are not comfortable being recorded you can either stay quiet during the session or leave now. But focus group recordings will be stored in a secure location and will be destroyed after the recording is transcribed; the only people who will have access to the recordings are members of the evaluation team. In addition, anything that is said during this group stays with the group. We will keep your responses private. We expect that you will not share what others say today with those outside of the group.

Before we get started, do you have any questions for me?

Program Overview/Background

We’d like to begin with a few questions about the organization and your role within it and then we’d like to learn a bit more about the services you provide. When we scheduled this focus group, we asked to speak with staff from [NOMINATED PROGRAM]. We just want to be sure that you all fit that role. Can you confirm that you are program staff of [NOMINATED PROGRAM]?

1. What is your position and role in the organization?

• How long have you been with this organization? In this position?

Program Services

2. Now, we’d like to ask you about the services/activities you provide as part of [NOMINATED PROGRAM].

• How are youth identified or referred to the program? Do you know what the primary sources of referrals are and are they adequate to identify and engage the target population?

• When and how do you first meet with youth/young adults after they are identified and engaged in program services?

• Do you feel the eligibility criteria for the program are sufficient to identify and engage the target population? Why or why not? And how would you change them?

• How often do youth drop out of the program and for what reasons?

• What are the key components of [NOMINATED PROGRAM]?

• How are you trained and supported to implement the program?

• What kind of supervision do you receive to ensure the program is implemented the way it was intended to be implemented?

• What are the main challenges to implementation?

• What are the main facilitators to implementation?

• Are youth engaged around the planning or implementation of [NOMINATED PROGRAM]? In what way are they involved?

• If not, why not? Would you be amenable to changing that? Involving them moving forward?

Service Termination and Followup

3. How and when do you typically stop serving the youth in [NOMINATED PROGRAM])?

• What typically happens to them? That is, do you transfer them to other programs? Do they stop receiving assistance from your organization?

• Do you do any follow-up with youth who exit your services?

Outcomes

4. What short-term and long-term outcomes are you hoping for in serving these youth?

• What are the key components of [NOMINATED PROGRAM] that you feel are most responsible for the outcomes you see in the youth you serve?

• How do you track outcomes for participants in the program?

• What other data do you collect on youth/young adults? In what format is the data?

• Who does the data collection/data entry? How frequently?

• What data are currently available? What format is it in?

Context

5. Are there other similar programs in the area that youth/young adults go to for services? Do they go there if there are no slots in your program? What do youth who do not get into your program do?

6. What other services are youth enrolled in [NOMINATED PROGRAM] typically receiving (Including services outside this organization)?

Work with Partner Agencies

7. In talking with your executive director, the following organizations were identified as your primary partners in delivering services to youth transitioning from foster care [NAME PARTNER AGENCIES]. Do you agree that these are your primary partners?

IF NO:

6a What organizations do you consider to be your primary partners and why?

IF YES:

6b What other organizations do you work with to serve youth

• Other service providers?

• Housing providers?

• Schools?

• Others?

8. How do you work with these organizations to serve youth transitioning from foster care?

• What are the key differences in how your organization serves these youth as compared to these partner agencies?

• What are the challenges in collaborating to serve these youth?

• What facilitators assist you to collaborate to serve these youth?

• How would youth outcomes be affected if they were served by only your organization or the organization that sponsors [NOMINATE PROGRAM] rather than by both?

Evaluation Capacity

Finally, the reason we are interested in learning so much about [NOMINATED PROGRAM] is because we are assessing it for participation in a larger evaluation. This next set of questions asks you to think about various issues related to participating in such an evaluation.

9. First, If [NOMINATED PROGRAM] were to undergo an evaluation, what kinds of questions would be important to ask? What would you hope to learn from an evaluation?

10. How would you suggest to engage youth in helping to plan and implement an evaluation? What challenges might we encounter by involving them?

11. Would you as staff have the time and capacity to be involved in any aspects of the evaluation? For example, if you were asked to collect data on the youth you serve, would you be able to do that?

12. Would you have any interest in/enthusiasm for an evaluation of [NOMINATED PROGRAM]? Why or why not?

13. Finally, what would be the key challenges to evaluating [NOMINATED PROGRAM]? How might we mitigate any of these?

Conclusion

14. Is there any other information about your program that you think would be helpful for us to know either around services or evaluation?

Thank you very much for taking time to speak with us today about your program.

4 Instrument 5:

Evaluability Assessment Focus Group Guide for Nominated Program Staff