Cross-Center Evaluation: Leadership Interview – CIPs

PURPOSE: The purpose of the interview is to obtain information from CIP directors regarding factors that influence their decisions to engage in services with the CB-supported Capacity Building Collaborative (specifically the Center for Courts); perceptions of the capacity building services received; and satisfaction with the Center’s services.

OMB #: 0970-0XXXX

Expiration Date: XX/XX/XXXX

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN: This collection of information will be used to obtain information from Court Improvement Program directors regarding services received from the Capacity Building Collaborative. Public reporting burden for this collection of information is estimated to average 40 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: 0970-XXXX, Exp: XX/XX/XXXX. If you have any comments on this collection of information, please contact Serena L. Williams, ACF, Administration on Children, Youth and Families by e-mail at [email protected].

Evaluation of the Child Welfare Capacity Building Collaborative

Court Improvement Program (CIP) Leadership Interview

The purpose of the interview is to obtain information from CIP Directors regarding factors that influence their decisions to engage in services with the CB-supported Capacity Building Collaborative (specifically the Center for Courts); perceptions of the capacity building services received; and satisfaction with the Center’s services.

Interview Code: C___

Introduction:

Good morning/afternoon. My name is ______________. I am calling from James Bell Associates, an evaluation firm based in Arlington, VA. As you may know, James Bell Associates and ICF have been contracted by the Children’s Bureau to conduct an independent evaluation of the Children’s Bureau’s Capacity Building Collaborative, which includes of the Center for States, Center for Tribes, and Center for Courts.

Directions to interviewer – please inform the interviewee:

This evaluation addresses the role of the Capacity Building Collaborative in helping States, Tribes, and Court Improvement Programs (CIPs) to support and enhance the capacity of their child welfare systems. The Leadership Interview is an important component of the evaluation. We are conducting these telephone interviews with all State child welfare directors as well as CIP directors and Tribal child welfare directors who receive capacity building services from the Centers.

I am going to ask you about the factors that may influence your decisions to use the services of the Capacity Building Collaborative and your experiences with the services you have received from the Center for Courts.

Your participation in this interview is voluntary but important to this evaluation. We anticipate that our discussion with you today will take approximately 40 minutes. Your privacy is important to us. As we have noted in scheduling this interview with you, in order for us to ensure that we accurately capture everything, we would like to audio record this interview. The recording will be destroyed as soon as we transcribe it and we will remove all identifying information from the transcript. Responses that you provide will be combined with answers from other respondents and will not be associated with your agency or you as an individual respondent. Do we have your permission to audio record this conversation? [Interviewer: Wait for the response] Do you have any questions or concerns before we begin?

I will now start the recording. [Interviewer: Begin audio recording]

1. RESPONDENT BACKGROUND

1. What is your official position title?

2. How long have you been in this position?

_______years or _____months

B. THE CAPACITY BUILDING CENTER

First a question about the Capacity Building Center for Courts.

1. How familiar are you with the Capacity Building Center for Courts and its services? Please describe.

C. ASSESSMENT AND WORK PLANNING

Interviewer: CIPs conduct an annual self-assessment that is submitted to CB.

1. Did your program’s understanding/awareness of the strengths and gaps in your court system increase through the self-assessment process? Please briefly explain.

1. Did the Center for Courts conduct any additional assessment of your CIP’s/courts’ capacity building needs?

2. If yes  How useful, or not, was the Center assessment to your program?

   1. Did your CIP’s capacity to self-assess its capacity building needs increase through the Center assessment process? Please briefly explain.

   2. How accurately did the Center assessment summary identify your CIP’s capacity building needs?

2. Was a work plan developed with the Center for Courts?

If no, skip to C3

1. In your opinion, how useful, or not, was the work plan?

2. Did the work planning process help you to achieve things that you would not have achieved without the Center?

3. In addition, did your CIP also participate in any way in the State assessment process that was conducted with the Center for States?

   1. If yes  Was your CIP an active participant in during the State assessment? Please describe.

4. Do you have additional thoughts that you would like to share regarding your CIP’s experiences with the assessment process (or the work plan that was developed with the Center)?

D. WORKING WITH THE CAPACITY BUILDING CENTER

Now, let’s talk about the services that your agency received from the Capacity Building Center for Courts.

D1. According to the Children’s Bureau’s records, the services that your agency received or participated in are listed in the document I sent you. Is that list of services accurate or is something missing?

Prefill from CapTRACK Tailored Services Records 

Assessment and workplans
Assessment service start	Assessment service end	Assessment hours	Workplan service start	Workplan service end	Workplan hours
Projects
Service start date	Date closed	Service hours	Project title		Project related to CFSR/PIP? (Y/N)
Other tailored services
Year	PIC hours	CFSP hours	FFPSA hours
CY20
CY19

D2. What were the key factors that influenced your decision to engage with the Center for Courts to utilize the Center’s services? Please describe.

3. D3. Let’s take a look at the list of potential factors that is included as item D3 in the interview questions summary document I sent you. Which, if any, of these factors also influenced your agency’s decision to use (or not to use) the Center’s services.

Affected decision to use services (+ or -)	Factors	Comments
	CFSR findings or development of the PIP
	Regional Office recommendation
	Federal law or policy change or requirements
	Center’s outreach efforts
	Prior relationship with the Center
	Timeliness in which services can be received
	Center staff’s level of knowledge and skills
	Cultural competency of the Center staff
	Usefulness of the Center’s products/materials
	Child welfare incident in the State (such as a child fatality)
	Availability of free resources from the Center
	Availability of CIP’s staff time
	State child welfare agency leadership
	State/local law or policy change
	Lawsuit/legal settlement
	COVID-19 pandemic

E. SERVICES RECEIVED AND PROGRESS TOWARD OUTCOMES

The next questions (section E in the interview guide that we sent you) pertain to: [Prefill from item D1 brief description(s) of the tailored services]

1. First, could you describe the current status of your CIP’s (court system’s) work with the Capacity Building Center for Courts?

2. In your opinion, what are the primary outcomes that your CIP expected to achieve from the work with the Center?

(Interviewer: Probe also)

1. Are there any additional primary outcomes that your CIP expected to achieve?

Now, let’s talk about the progress that has been made toward the primary outcomes.

[Interviewer: Ask questions E3: A, B, C and repeat these questions for additional outcomes if respondent highlighted more than one outcome]

3. Outcomes:

1. Please describe the progress that has been made toward [outcome from question E2].

2. To what extent did the services provided by the Center contribute toward this outcome?

3. Were there other key factors that also contributed toward this outcome?

4. Was your work with the Center intended to result in any practice changes in your court system?

   1. If yes  In your opinion, are your agency’s practices changing or improving in any way? Please describe.

   2. What do you attribute these changes to?

   3. And are there any changes in outcomes for the children and families served by your child welfare system?

   4. If yes  Please describe. (Interviewer: Probe also for what evidence they have of these changes)

      1. In your opinion, what contributed to these changes?

5. Were there any key factors that may have hindered your CIP (court system) from making progress toward achieving the intended outcomes? If so, please describe. [e.g., organizational or agency factors, community factors, other environmental/contextual factors]

   1. Has the progress toward outcomes been impacted by the COVID-19 pandemic? If yes, please describe.

6. Overall, how would you describe the quality of the services/materials your CIP received from the Center?

7. Based on the services your agency received, how would you describe the expertise of the Center liaisons/consultants?

8. In your opinion, what have been the key benefits of working with the Center?

9. Based on your work with the Center, do you have any suggestions for improving the Center’s services?

IF tailored services have DEI focus or component, probe also:

Do you have any suggestions for improving the Center’s services and support on issues related to diversity, racial equity, and inclusion (including tribal sovereignty) and advancing equity and support for underserved communities?

F. CIP’S EXPERIENCE WITH THE CENTER FOR COURTS

Please provide your level of agreement with the following statements, using the response categories, from 1 to 6: Strongly disagree; Disagree; Somewhat disagree; Somewhat agree; Agree; Strongly agree. These response categories are also listed in your copy of the interview questions document.		1 Strongly disagree	2 Disagree	3 Somewhat disagree	4 Somewhat agree	5 Agree	6 Strongly agree	Do not know
F1.	My CIP is satisfied with the level of accessibility of the Center for Courts
F2.	When working with the Center for Courts on issues related to my child welfare system, my CIP (court system) has been satisfied with the frequency of communication
F3.	My CIP/courts is comfortable disclosing areas of concern or weaknesses of our agency to the Center for Courts’ staff
F4.	When working with the Center for Courts, my CIP’s experience is that our program plays an active part in decision-making
F5.	My CIP is satisfied with the level of follow-through from the Center for Courts
F6.	Overall, my CIP is satisfied with our relationship with the Center for Courts
F7.	Overall, the services that the Center has provided to my agency reflect an understanding of how my CIP and our court system operates
F8.	The services provided by the Center were appropriate and aligned with my CIPs needs
F9.	The services provided by the Center took into account the characteristics and unique context of my CIP and court system

G. IN CONCLUSION

G1. Do you have any additional comments you would like to share regarding CB’s Capacity Building Collaborative or your experiences with the services they provide?

THANK YOU

WE GREATLY APPRECIATE YOUR PARTICIPATION!