Cross-Center Evaluation: Leadership Interview – Tribes

PURPOSE: The purpose of the interview is to obtain information from Tribal Child Welfare Directors regarding factors that influence their decisions to engage in services with the CB-supported Capacity Building Collaborative (specifically the Center for Tribes); perceptions of the capacity building services received; and satisfaction with the Center’s services.

OMB #: 0970-0XXXX

Expiration Date: XX/XX/XXXX

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN: This collection of information will be used to obtain information from tribal child welfare program directors regarding services received from the Capacity Building Collaborative. Public reporting burden for this collection of information is estimated to average 45 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: 0970-XXXX, Exp: XX/XX/XXXX. If you have any comments on this collection of information, please contact Serena L. Williams, ACF, Administration on Children, Youth and Families by e-mail at [email protected].

Evaluation of the Capacity Building Collaborative

Tribal Child Welfare Leadership Interview

The purpose of the interview is to obtain information from Tribal Child Welfare Directors regarding factors that influence their decisions to engage in services with the CB-supported Capacity Building Collaborative (specifically the Center for Tribes); perceptions of the capacity building services received; and satisfaction with the Center’s services.

Interview Code: T___

Introduction:

Good morning/afternoon. My name is ______________. I am calling from James Bell Associates, an evaluation firm based in Arlington, VA. As you may know, James Bell Associates and ICF have been contracted by the Children’s Bureau to conduct an independent evaluation of the Children’s Bureau’s Capacity Building Collaborative, which includes of the Center for States, Center for Tribes, and Center for Courts.

Directions to interviewer – please inform the interviewee:

This evaluation addresses the role of the Capacity Building Collaborative in helping States, Tribes, and Court Improvement Programs (CIPs) to support and enhance the capacity of their child welfare systems. The Leadership Interview is an important component of the evaluation. We are conducting these telephone interviews with all State child welfare directors as well as Tribal directors and CIP directors who receive capacity building services from the Centers.

I am going to ask you about the factors that may influence your decisions to use the services of the Capacity Building Collaborative and your experiences with the services you have received from the Center for Tribes.

[use this paragraph with Tribes that have not yet completed their tailored services projects] Your participation in this interview is voluntary but important to this evaluation. We anticipate that our discussion with you today will take approximately 40 minutes. Your privacy is important to us. As we have noted in scheduling this interview with you, in order for us to ensure that we accurately capture everything, we would like to audio record this interview. The recording will be destroyed as soon as we transcribe it and we will remove all identifying information from the transcript. Responses that you provide will be combined with answers from other respondents and will not be associated with your program/your Tribe or you as an individual respondent. Do we have your permission to audio record this conversation? [Interviewer: Wait for the response] Do you have any questions or concerns before we begin?

[use these two paragraphs only with the Tribes that completed their tailored services – these are specified beforehand:] Your participation in this interview is voluntary but important to this evaluation. We anticipate that our discussion with you today will take approximately 55 minutes. This interview will include two parts. Your responses on part I will be combined with answers from other respondents and will not be associated with your program, your Tribe or you as an individual respondent. The later part II of this interview includes a separate set of discussion topics that we would like to cover on behalf of the Center for Tribes. Your responses on part II will shared with the Center for Tribes evaluators. I will let you know when we get to the start of the part II of this interview.

Your privacy is important to us. As we have noted in scheduling this interview with you, in order for us to ensure that we accurately capture everything, we would like to audio record this interview. The recording will be destroyed as soon as we transcribe it. Do we have your permission to audio record this conversation? [Interviewer: Wait for the response] Do you have any questions or concerns before we begin?

I will now start the recording. [Interviewer: Begin audio recording]

PART I

1. RESPONDENT BACKGROUND

1. What is your official position title?

2. How long have you been in this position?

_______years or _____months

B. THE CAPACITY BUILDING CENTER

First a question about the Capacity Building Center for Tribes.

1. How familiar are you with the Capacity Building Center for Tribes and its services? Please describe.

C. ASSESSMENT AND WORK PLANNING

Interviewer: CBCT uses NAFET 1. If the Tribe did not participate in the assessment, move to Section D.

1. How useful, or not, was the assessment to your program?

   1. Did your child welfare program’s understanding/awareness of the strengths and gaps in your child welfare system increase through the assessment process? Please briefly explain.

   2. Did your child welfare program’s capacity to self-assess its capacity building needs increase through the assessment process? Please briefly explain.

   3. How accurately did the assessment summary identify the Tribal CW system’s capacity building needs?

2. Was a work plan developed as a result of the assessment?

If no, skip to C3

1. In your opinion, how useful, or not, was the work plan?

2. Did the work planning process help you to achieve things that you would not have achieved without the Center?

3. Do you have additional thoughts that you would like to share regarding your child welfare program’s experiences with the assessment process or the work plan that was developed with the Center?

D. WORKING WITH THE CAPACITY BUILDING CENTER

Now, let’s talk about the services that your child welfare program received from the Capacity Building Center for Tribes.

D1. According to the Children’s Bureau’s records, the services that your child welfare program received or participated in are listed in the document I sent you. Is that list of services accurate or is something missing?

Prefill from CapTRACK Tailored Services Records 

Assessment and workplans
Assessment service start	Assessment service end	Assessment hours	Workplan service start	Workplan service end	Workplan hours
Projects
Service start date	Date closed	Service hours	Project title		Project related to CFSR/PIP? (Y/N)
Other tailored services
Year	PIC hours	CFSP hours	FFPSA hours
CY20
CY19

D2. What were the key factors that influenced your decision to engage with the Center for Tribes to utilize the Center’s services? Please describe.

3. Let’s take a look at the list of potential factors that is included as item D3 in the interview questions summary document I sent you. Which, if any, of these factors also influenced your child welfare program’s decision to use (or not to use) the Center’s services?.

Affected decision to use services (+ or -)	Factors	Comments
	CFSR findings or development of the PIP
	Regional Office recommendation
	Federal law or policy change or requirements
	Center’s outreach efforts
	Prior relationship with the Center
	Timeliness in which services can be received
	Center staff’s level of knowledge and skills
	Cultural competency of the Center staff
	Usefulness of the Center’s products/materials
	Child welfare incident in the State or Tribe (such as a child fatality)
	Availability of free resources from the Center
	Availability of child welfare program staff time
	Tribe’s child welfare program leadership
	State/local law or policy change
	Lawsuit/legal settlement
	COVID-19 pandemic

E. SERVICES RECEIVED AND PROGRESS TOWARD OUTCOMES

The next questions (section E in the interview guide that we sent you) pertain to: [Prefill from item D1 brief description(s) of the tailored services]

1. First, could you describe the current status of your Tribe’s work with the Capacity Building Center for Tribes?

2. In your opinion, what are the primary outcomes that your Tribe expected to achieve from the work with the Center?

(Interviewer: Probe also)

1. Are there any additional primary outcomes that your Tribe expected to achieve?

Now, let’s talk about the progress that has been made toward the primary outcomes.

[Interviewer: Ask questions E3: A, B, C and repeat the questions for additional outcomes if respondent highlighted more than one outcome]

3. Outcomes:

1. Please describe the progress that has been made toward [outcome from question E2].

2. To what extent did the services provided by the Center contribute toward this outcome?

3. Were there other key factors that also contributed toward this outcome?

4. Was your work with the Center intended to result in any practice changes in your child Tribal welfare system?

   1. If yes  In your opinion, are your child welfare program’s practices changing or improving in any way? Please describe.

   2. What do you attribute these changes to?

   3. And are there any changes in outcomes for the children and families served by your child welfare system?

   4. If yes  Please describe. (Interviewer: Probe also for what evidence they have of these changes)

      1. In your opinion, what contributed to these changes?

5. Were there any key factors that may have hindered your child welfare program from making progress toward achieving the intended outcomes? If so, please describe. [e.g., organizational or program factors, community factors, other environmental/contextual factors]

   1. Has the progress toward outcomes been impacted by the COVID-19 pandemic? If yes, please describe.

6. Overall, how would you describe the quality of the services/materials your program received from the Center?

7. Based on the services your child welfare program received, how would you describe the expertise of the Center liaisons/consultants?

8. In your opinion, what have been the key benefits of working with the Center?

9. Based on your work with the Center, do you have any suggestions for improving the Center’s services?

IF tailored services have DEI focus or component, probe also:

Do you have any suggestions for improving the Center’s services and support on issues related to diversity, racial equity, and inclusion (including tribal sovereignty) and advancing equity and support for underserved communities?

F. TRIBE’S EXPERIENCE WITH THE CENTER FOR TRIBES

Please provide your level of agreement with the following statements, using the response categories, from 1 to 6: Strongly disagree; Disagree; Somewhat disagree; Somewhat agree; Agree; Strongly agree. These response categories are also listed in your copy of the interview questions document.		1 Strongly disagree	2 Disagree	3 Somewhat disagree	4 Somewhat agree	5 Agree	6 Strongly agree	Do not know
F1.	My child welfare program is satisfied with the level of accessibility of the Center for Tribes
F2.	When working with the Center for Tribes on issues related to my child welfare system, my program has been satisfied with the frequency of communication
F3.	My program is comfortable disclosing areas of concern or weaknesses of our program to the Center for Tribes’ staff
F4.	When working with the Center for Tribes, my program’s experience is that our program plays an active part in decision-making
F5.	My program is satisfied with the level of follow-through from the Center for Tribes
F6.	Overall, my program is satisfied with our relationship with the Center for Tribes
F7.	Overall, the services that the Center has provided to my program reflect an understanding of how my child welfare system operates
F8.	The services provided by the Center were appropriate and aligned with my child welfare system’s needs
F9.	The services provided by the Center took into account the characteristics and unique context of my Tribe

G. IN CONCLUSION

G1. Do you have any additional comments you would like to share regarding CB’s Capacity Building Collaborative or your experiences with the services they provide?

[In interviewing the specific Tribes that have completed their tailored services project – skip this thank you and continue with text below the “thank you”]

THANK YOU

WE GREATLY APPRECIATE YOUR PARTICIPATION!

[In interviewing the Tribes that have completed tailored services project – continue with the following]

Thank you! This concludes the Part I of the interview questions.

***

The next section of the interview (part II) includes topics that we will cover on behalf of the Center for Tribes. Do you have any questions before we start?

PART II

Capacity Building Center for Tribes – Jurisdiction Leadership Interview

Opening

1. What experiences, such as program challenges or opportunities, made you want to start or be involved in [specific project]?

Lessons you would like to share

2. What are you proud of in [specific project]?

3. What was easier to accomplish in [specific project]? Why?

4. What was harder to accomplish in [specific project]? Why?

• Potential topics to expand upon:

  • Specific project tasks

  • Relationships: Internal, external, etc.

  • Levels: Intra-agency, interagency, CBCT, Tribal Leadership, Community, etc.

  • Factors: Communication, timeliness, workforce, funding, COVID, etc.

5. How has work on [specific project] impacted you, your program, or community?

1. What outcomes do you think were a result of [specific project]?

2. What do you think contributed to achieving the outcomes?

6. What do you think would help [specific tribe] build upon [specific project]?

7. If [specific nation] is involved in other Center services: How has your involvement in [specific service] impacted your work in [specific project]?

Experience with CBCT

8. What was your experience like working with the Center for Tribes on [specific project]?

1. Was working with the Center for Tribes on [specific project] what you expected? How so?

2. Are there any stand-out moments when you reflect on working with the Center for Tribes on [specific project]?

9. In what ways did the Center for Tribes provide services in a manner that was culturally-responsive?

• Potential topics to expand upon:

  • Followed [specific nation] protocols and ways

  • Knowledgeable about [specific nation] historical and current contexts

  • Used appropriate language and names

  • Spent time building relationships

  • Used storytelling

  • Worked collaboratively, shared decision-making processes

  • Respected ideas and listened to concerns

10. What can the Center for Tribes do better?

• Potential topics to expand upon:

  • Build relationships

  • Agency (feel like the process was in your control or out of your hands)

  • Slow down, speed up, or clarify process

  • Scheduling

  • Accessibility of services

  • Cultural fit

  • Trauma-informed

  • Evaluation support

11. Are there other Center for Tribes services you would be interested in?

Closing

12. Is there anything else you would like to share or clarify?

13. Do you have any ideas as to how we can make the findings of this [CBCT] evaluation most useful to [specific nation and agency]? (briefs/reports, presentations, discussion of findings, sharing with others, etc.)

THANK YOU.

WE GREATLY APPRECIATE YOUR PARTICIPATION!