Instrument 5 - Site Visit 2 Interview Guide for Administrators

Instrument 5A: Site Visit 2 Interview Guide for Administrators: Child Welfare Agency Administrators

Instructions

Thank you for joining us today. We would like to invite you to participate in an interview that will last about 60 minutes. We are conducting an evaluation of LifeSet, and today we’d like to learn more about the LifeSet program model and service delivery. Specifically, we will talk about the challenges and barriers the program has experienced, key program components, the enrollment and intake process, among other topics. We will use this information to better understand how the LifeSet program operates and serves young adults in New Jersey.

Your participation in this interview is voluntary. You can choose not to answer any question or not participate in the interview at all. There will be no consequences to you if you choose not to participate. We will keep the information you provide private and will not share it with anyone except for research staff working on the study. Additionally, federal law states that an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number for this data collection is 0970-0XXX and the expiration date is XX/XX/XXXX.

Background and Role

Current position, professional experience, main responsibilities and role within LifeSet program

Thank you for taking the time to talk with me today.

[Note to Interviewer: Only ask if this is the first time interviewing this respondent, otherwise go to #10] I’d like to start by learning a little bit about you.

1. What is your current position and how long have you been in it?

2. What are your main responsibilities?

3. [If held current position since before March 2020] Are these the same responsibilities you had pre-COVID? If not how did your responsibilities change?

4. How long have you worked at DCF?

5. What other positions have you held at DCF?

6. What role do you play as part of the LifeSet program?

7. Have you worked with former foster youth or other vulnerable youth populations before working for DCF? If so, in what capacity?

8. Have you held other positions – client facing or management – in social services before? If so, what were they?

9. Is there a degree/credential or experience requirement for your position? What is it?

[Note to interviewer: Only ask if this is a follow-up interview with this respondent] First I’d like to ask if anything changed regarding your current position and main responsibilities since we last spoke on [date]?

10. Please describe any changes in your position or main responsibilities. When the changes occur and why?

Implementation Supports

Next, I’d like to ask you about policies on the implementation of LifeSet in New Jersey and successes and challenges you have encountered during implementation.

Impact of new or current policies on implementation

11. Can you describe any new policies your agency created to support the implementation of LifeSet?

12. What are the YV policies you must follow in order to implement LifeSet?

1. Do those policies make it easier or harder to provide services youth need?

Implementation successes and challenges

13. What challenges, if any, have you encountered implementing LifeSet? How did you overcome these challenges?

14. What would you describe as successes, or what worked well, in implementing LifeSet? What contributed to that success?

Implementation supports and barriers

15. What has been particularly helpful in implementing LifeSet?

    1. Probes: readiness of providers, support from leadership of partner agencies (DCF, YV, providers), policies, technology, funding, use of data for program improvement, monitoring, and decision-making, teaming and communication structure with DCF and YV, etc.

16. What things have been barriers to implementing LifeSet?

    1. Probes: lack of support from leadership, policies, technology, funding, teaming and communication structure with DCF and YV, etc.

17. What DCF resources have been required to implement LifeSet?

    1. What DCF resources have been required to support the evaluation?

Program Model

Now, I would like to learn more about LifeSet’s program model, including the program’s goals and key features.

Program goals and main components

18. What are the goals of LifeSet?

19. Are the goals of LifeSet different from the goals established for youths served by other transition services programs?

20. [Bring/ show copy of the logic model] We have been shown this logic model. Does this reflect inputs, outputs and outcomes for the operation of LifeSet in New Jersey in early 2021? If not, what would you add or take away from this model to make it more accurate?

    1. Inputs are program resources such as staff, caseload size, and tools.

    2. Outputs are program activities such as supervision and consultation, sessions with youth, setting youth goals, and holding group sessions with youth.

    3. Outcomes, or goals, are the changes the program expects will result from the inputs and outputs such as increased education and employment or housing stability.

Next, I would like to learn about any modifications made to the program for New Jersey.

New Jersey-specific modification

21. Do you know if the model and program has been changed in some way in New Jersey? If so, how so?

22. How, if at all, has the evaluation impacted service delivery?

23. How, if at all, has COVID impacted service delivery? How has that changed overtime?

Service Delivery

Next, I’d like to learn more about the eligibility and intake process, and what usual services are provided to youth.

Eligibility and intake

24. What are the eligibility criteria for youth to receive transition support services?

1. Do they need to keep an open case to receive those services?

2. How often are the cases kept open just to continue providing services?

3. Are youth able to close and re-open their case at any time before the age at which services end? If so, how typical is that?

4. What is the minimum age when youth can start receiving transition services?

25. Currently, youth’s eligibility for LifeSet is determined first by using administrative data to identify youth who meet age and case status criteria and then confirmed through a structured screening process with the youth’s caseworker. Do you believe that this is an effective way to identify eligible youth?

    1. How often do you believe this process “misses” or fails to identify youth who are eligible?

    2. How often do you believe the process results in mismatches where youth are screened in but later found to be not eligible?

    3. How often are youth screened in as eligible but become ineligible prior to the provider’s assessment (e.g., due to incarceration)?

26. How are youth selected for LifeSet referred to a provider? Who informs youth that they are eligible for the program?

Services as usual (SAU)

27. What services does DCF offer to help youth transition from foster care to adulthood?

1. Probe to distinguish between services provided by DCF staff versus other community providers that youth are referred to

2. Probe for any restrictions on services based on age.

28. What is the average length of time that you typically work with youth? (That is, from the time you first meet them to when their case closes, or another reason why you stop working with)? Does this depend on the youth?

1. What affects how long you work with the youth?

29. Generally, how available are transition services/service providers for transition aged youth?  

1. How, if at all, does service availability vary across the state? 

2. What affects the availability of services or service providers? [ages served, geography, local conditions, budgets]

3. Are wait lists typical, and generally how long are waitlists?

30. Can you describe in general how youth transitioning out of care or preparing to transition out get referred to services?

1. What factors are considered when deciding what services to refer youth to? Who makes these decisions?

2. Do you believe this is an effective process for getting youth the services that they need?

31. You just told us a bit about how youth are referred to services. If there is one, what is a typical case plan for youth who turn 18 while in care? Can you describe how this plan is determined?

Data Systems and Use

Next, I’d like to know about the data systems you use to track transition-aged youth and services.

32. Is there a data system used to record case information for transition-aged youth?

1. What type of data are in this system? [Probes: case notes, assessments, service plans, demographics, worker activities with youth]

2. Who enters the data? [Probe: DCF staff vs. transition services providers]

33. Are any changes or improvements to the system planned in the short term - e.g., over the next few months or within the next year? If so, what improvements?

34. Did you make any changes to your data system, NJ SPIRIT or SafeMeasures, when you started implementing LifeSet? If yes, what changes were made?

35. How do you use the information collected in your data system?

1. Probe for how it informs service delivery, practices, and the program model

36. What types of reports, specific to transition services, are produced?

1. What information or data are in the reports?

2. Are providers able to access these reports for youth they serve?

37. How does data quality affect LifeSet’s/DCF’s ability to help youth, in your opinion?

Program Improvement (CQI)

I’d now like to learn a bit about how you assess the effectiveness of services provided to transition aged youth.

Assessing effectiveness of services

38. How does DCF assess the effectiveness of services for youth transitioning from care? 

1. Probe for how services are monitored; Reporting requirements to state; Reporting requirements for contracted agencies

2. Are metrics the same for youth over 18 and aged 16-17?

39. Are there some service providers you’ve found to be more effective than others, and which ones are they?

1. What makes them more effective in your view?

40. How effective has DCF/ New Jersey been at achieving positive outcomes for youth transitioning out of care?

1. Is DCF/ New Jersey able to meet the needs of youth transitioning from care and support them in positive development?

Opinion of Program Effectiveness

Lastly, I’d like to discuss your opinion of the effectiveness, strengths, and challenges of LifeSet services in New Jersey.

Opinion of services

41. Overall, how effective do you perceive LifeSet has been at providing services to youth transitioning out of care?

1. What are major strengths? Limitations?

42. Is there anything about LifeSet that you would like to change? If so, what is it?

43. What challenges do youth in LifeSet continue to face after they leave the program?

1. Is there anything you think LifeSet could be doing to better serve and meet the needs of youth? If so, what?

Closing Questions

Thank you for taking the time to talk with me today. I have a couple closing questions.

44. Is there anything that I did not ask about that you think I should know about LifeSet/ services as usual or your experience?

45. Do you have any final questions for me about the study, or about the research team?

The Paperwork Reduction Act Statement: This collection of information is voluntary and will be used to evaluate the programs and services provided to young adults who are currently or were previously in foster care. Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: 0970-XXXX, Exp: XX/XX/XXXX. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to Michael Pergamit at [email protected].