Jurisdiction Lead Interview

Jurisdiction Lead Interview

OMB Control No.: 0970-0401

THE PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) The purpose of this information collection is to gather feedback about your experiences with the Center for States services. Your contribution to the evaluation effort is extremely valuable and will be used to improve future services. Public reporting burden for this collection of information is estimated to average 60 minutes per respondent, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. A Federal agency may not conduct or sponsor, and no individual or entity is required to respond to, nor shall an individual or entity be subject to a penalty or failure to comply with a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless that collection of information displays a currently valid OMB control number. If you have any comments on this collection of information, please contact Christine Leicht, Center for States, by e-mail at [email protected].

Expiration Date: 06/30/2024

Highlights= summarize (not necessary to read word for word)

Bold= read word for word

Italics= probes

*Asterisk questions - high priority question for outcome analysis, always ask a follow-up probe!

Prior to the Interview

In the days leading up to the interview, the interviewer will review the following:

1. Project’s work plan and close out report to help create effective and more specific probes.

2. Project milestones and capacity dimensions identified in the workplan and final report to assist in asking about Center impacts.

3. Project status (i.e., normal close out, ended early, not implemented, the project is continuing), as interview questions and probes will need to be tailored to the project status.

The interview coordinator will share the workplan, final report, and close out report with facilitators and notetakers prior to the date of the interview. These are also available in CapTRACK.

Overview

Briefly discuss the purpose of the interview:

The purpose of our conversation is to gather feedback about your experiences with the Center for States services for [insert project name]. Your contribution to the evaluation effort is extremely valuable and will be used to improve future services.

Convey to each interview participant our privacy policy:

(1) The interview is voluntary; (2) you can decline to answer any questions, or you can stop participating in the interview at any time; (3) data from this interview will be held in private by the evaluation team. The information from today’s interview will be reported and may be shared with the Children’s Bureau and the Capacity Building Collaborative’s 3^rd party evaluator. When reported, this information will be aggregated across all states and your name will not be associated with it. In some instances, however, the information you provide about the success and outcomes of the project might be tied to the project itself in our reporting.

Ask permission to record the interview:

To ensure we capture the discussion accurately and completely, I would like to obtain your permission to audio record the session. This recording will only be accessed by our evaluation team. If you choose not to have the interview recorded, we will take notes. Your name will not be included in any reporting.

Will you allow us to record this interview?

The interviewer asks the notetaker to start the recording.

Thank you, the recording has started.

Ask if they have any questions for you before you begin.

Ask them to provide their verbal consent to participate in the interview.

Do you have any questions before we begin?

Introduction

If applicable, clarify the status of the project and mention whether they may have already participated in an earlier project interview.

[CONTINUATION PROJECTS, FIRST INTERVIEW] I understand that the project is continuing onto a new work plan. However, for today’s interview, we would like you to focus on the most recent workplan that covered [insert date range from last workplan] (because we will be doing separate interviews for the current workplan).

OR

[CONTINUATION PROJECTS, PRIOR INTERVIEWEE] We recognize that you may have already participated in an interview related to this project for a previous workplan. However, we also conduct interviews after completing each workplan. For today’s interview, we would like you to focus on the most recent workplan that covered [insert date range from last workplan]. We appreciate your time and perspective on this interview.

OR

[END OF PROJECT] We recognize that this project has ended. For today’s interview, we would like you to focus on the workplan that covered [insert date range from last workplan]. We appreciate your time and perspective on this interview.

Questions

1. Can you begin by briefly describing your role at the agency?  What was your involvement with Center support and role on the project?

2. *To what extent do you understand the overarching goal(s) of this project? Would you say that you…

• Fully understand project goals

• Mostly understand project goals

• Partially understand project goals

• Do not understand the project goals

*Probe: Please explain why you selected this response.

3. According to the project workplan, the original desired change or primary goal was [insert Jurisdiction's Desired Changes in Child Welfare Practice from workplan]. To what extent did the project make progress towards achieving this desired change?

4. What has been your experience with the Center’s services [or for continuing projects- What has been your experience with the Center’s services so far]?

Probe about these Practice Model phases…

Engagement:

5. *To what extent did the Center help you engage all necessary stakeholders to actively participate and collaborate (e.g., including people with lived experience, federal partners, university partners, community agencies, other agency staff, etc.)? Would you say that the Center helped you…

• Fully engage all necessary stakeholders

• Mostly engage all necessary stakeholders

• Partially engage all necessary stakeholders

• Did not help you engage all necessary stakeholders

*Probe: Please explain why you selected this response.

6. *To what extent was the Center team a “good fit” with your agency? Would you say that they were…

• Entirely a good fit

• Mostly a good fit

• Partially a good fit

• Not a good fit

*Probe: Please explain why you selected this response.

7. *To what extent did your agency feel that it had ownership of this project? Would you say that…

• They felt they had full ownership of the project

• They felt they had substantial ownership of the project

• They felt they had partial ownership of the project

• They felt they had no ownership of the project

*Probe: Please explain why you selected this response.

7. *To what extent do you think your agency’s project team had authority to make project decisions? Would you say that…

• Your agency had full authority to make project decisions

• Your agency had considerable authority to make project decisions

• Your agency had partial authority to make project decisions

• Your agency had no authority to make project decisions

*Probe: Please explain why you selected this response.

7. *To what extent do you think your agency was ready to take on the project work? Would you say that…

• Your agency was fully ready to take on the project work

• Your agency was mostly ready to take on the project work

• Your agency was partially ready to take on the project work

• Your agency was not ready to take on the project work

*Probe: Please explain why you selected this response.

7. *To what extent was there buy-in from your agency’s leadership? Would you say that leadership had…

• Fully bought-in to the project

• Mostly bought-in to the project

• Partially bought-in to the project

• Not bought-in to the project

*Probe: Please explain why you selected this response.

Working as a team:

7. *To what extent do you agree with the following statement, “The Center and my agency effectively worked together as a team.” Would you say that you “agree”, “somewhat agree”, “somewhat disagree” or “disagree” with this statement?

*Probe: Please explain why you selected this response.

Work Planning:

7. How helpful was Center support in thinking about what you wanted to achieve and how you would know if you did achieve your goals? What about it was helpful or not? What could have been improved?

Tracking and Adjusting:

7. How well is the Center helping your state to identify what data were needed and how to secure those data? Can you think of an example of how the team has used data to determine progress and inform planning and decision making?

7. Now I would like you to describe in what ways, if any, Center services benefitted your jurisdiction. [After general description of benefit or no benefit, probe for EVERY improvement area below.]

Did Center support lead to improvements, in:

1. Agency staff knowledge, awareness, or skills? If so, please provide an example. Were these changes you saw right away or later in the project?

2. Agency policy? If so, please provide an example. Were these changes you saw right away or later in the project?

3. Agency staff practices? If so, please provide an example. Were these changes you saw right away or later in the project?

4. Organizational infrastructure (e.g., training systems, staffing, CQI systems)? If so, please provide an example. Were these changes you saw right away or later in the project?

5. Outcomes for children and families (e.g., increased permanency for children, reduced time in foster care, reduced number of placements, reduced recurrence of CAN, etc.)? If so, please provide an example. Were these changes you saw right away or later in the project?

6. Other types of improvements, outcomes, or impacts? If so, please provide an example. Were these changes you saw right away or later in the project?

• Probe: Infusing a racial equity approach (e.g., including a deliberate approach to identifying the systemic and institutional structures, policies, processes, and practices that produce disparate outcomes for people of color). If so, please provide an example. Were these changes you saw right away or later in the project? 

• Probe: Meaningful involvement of individuals or groups with lived experience in child welfare (e.g., youth and family partners with direct, first-hand involvement with the child welfare system were involved). If so, please provide an example. Were these changes you saw right away or later in the project? 

If any impacts mentioned, use additional probes:

7. What factors do you think helped to facilitate the impacts you described (list impacts mentioned by respondent)?

8. Which of the impacts you just described will be sustained?

• Probe: How well is the Center helping prepare your agency to sustain capacity for this work?

7. What factors, if any, impeded project progress?

This next set of questions pertain to the Center for State’s efforts to promote engagement and partnership with individuals who have lived experience and/or lived expertise. For the purposes of this interview, references to individuals with lived expertise and lived experience include individuals who have direct, firsthand experience with the child welfare system. Lived experience can provide insight into patterns, common behaviors, challenges, and barriers among individuals who share similar experiences in the child welfare system¹. Lived expertise refers to “the unique ability to translate personal experiences in the child welfare system into meaningful system change. Developing lived expertise is a process that takes ongoing training, support, and skill building.²”

7. In what ways over the past year, if any, has the Center for States promoted the importance of lived experience and/or lived expertise in its delivery of services?

Probe: Did they use youth or family consultants with lived expertise?  

 

7. In what ways over the past year, if any, did the Center for States encourage your jurisdiction to meaningfully engage with individuals with lived experience and/or lived expertise?  

 

7. To what extent do you agree that individuals with lived expertise assigned to the Center team were meaningfully engaged on this project (i.e., actively contributed and helped to inform decisions)? On a seven-point scale where 1= strongly disagree and 7=strongly agree.  

Strongly Disagree (1)

Disagree (2)

Somewhat Disagree (3)

Neither Agree nor Disagree (4)

Somewhat Agree (5)

Agree (6)

Strongly Agree (7)

Not applicable (individuals with lived expertise were not part of the Center team)

1. What contributed to your response? Are there any (or where are) opportunities to improve?  

2. What are some examples of meaningful engagement you have observed within? 

 

7. To what extent do you agree that the Center is building the capacity of your jurisdiction to engage individuals with lived experience and/or lived expertise? On a seven-point scale where 1= strongly disagree and 7=strongly agree. 

  

Strongly Disagree (1)

Disagree (2)

Somewhat Disagree (3)

Neither Agree nor Disagree (4)

Somewhat Agree (5)

Agree (6)

Strongly Agree (7)

Don't Know (9)

1. What are some examples or activities that have helped increase your jurisdiction’s capacity to engage individuals with lived experience and/or lived expertise?  

This next set of questions relate to any of the Center for State’s efforts to infuse a racial equity approach into its service delivery. A racial equity approach refers to a deliberate approach to identifying the systemic and institutional structures, policies, and practices that produce disparate outcomes for Black, Indigenous, and people of color [and their] communities. This approach requires taking intentional steps to address these inequities.³

7. Overall, to what extent did Center services increase your jurisdiction’s capacity to achieve its planned activities using a racial equity approach?

Probe: Can you provide any examples about why the services were helpful in using a racial equity approach?

Probe: How could Center services related to racial equity better meet your jurisdiction’s needs?

 

7. [END-OF-PROJECT VERSION] How well did the Center help prepare your state to sustain capacity for this work?

Probe if answer is negative: What could have been done differently? [For continuing projects: How did the Center help your jurisdiction prepare for sustaining the progress made thus far with this project?]

[FOLLOW-UP VERSION] How is the jurisdiction sustaining earlier progress/capacity? 

Now I have a few questions about the organizational change and implementation process that the Center used when supporting this project. As you might already know, change management is an evidence-based process for organizational change and implementation that the Center has organized according to the following five phases.

Phase I - Identify and Assess Needs or Opportunities 

Phase II - Develop Theory of Change 

Phase III - Select and Adapt/Design Intervention 

Phase IV - Plan, Prepare, and Implement 

Phase V - Evaluate and Apply Findings 

7. How familiar are you with the organizational change and implementation process used by the Center?

7. Has your agency considered applying organizational change and implementation process in another effort? If so, how? If not, what would have helped your agency implement an evidenced informed change and implementation process?

7. To what extent, if at all, did your agency’s capacity to evaluate practice change improve as a result of support from Center staff?

[If the interviewee reports that their capacity to evaluate practice change improved as a result of support from Center staff, ask] In what ways did Center staff help your agency to improve its capacity to evaluation practice change?

7. How did the Center support you with [insert topical area here]?

Probe: Can you share specific examples of this? How could the Center have improved this support?

7. [If time permits, ask this question]: What were the most helpful activities/strategies used by Center staff in working with your jurisdiction on this project? What were the least helpful?

Possible probes for strategies: Toolkits; assessments, involving frontline staff; cross collaboration with different stakeholders, coaching. The interviewer can also refer to the workplan for specific examples.

Types of strategies:

Most helpful:

Least helpful:

7. What lessons or recommendations do you have for other states beginning to work with the Center?

7. Are there other comments you would like to provide about your experience with the Center for States? 

Thank you for taking the time to share your perspectives with us!

1 Given, L. M. (Ed.). (2008). The SAGE encyclopedia of qualitative research methods (Vol. 2). Thousand Oaks, CA: SAGE. https://dx.doi.org/10.4135/9781412963909

2 Pauter, S. M., Dicharry, L., Cuza, H., Harvey, J., Hernandez, V., McDaniel, S., & Trochtenberg, R. (2019, July). Definition of partnership. In S. Pauter (Chair), Think tank on partnering with youth and young adults in child welfare. Symposium conducted at the Chadwick Center, San Diego, CA. https://www.tipscenter.org/public/uploads/ckeditor/5e98837a743cf1587053434.pdf

3 Capacity Building Center for States (2023). Diversity, racial equity, and inclusion in child welfare: Terms and Definitions. https://capacity.childwelfare.gov/states/resources/rei-terms-and-definitions