Center for Courts (CBCC) Evaluation: Court Improvement Program Capacity Building Feedback Survey

CIP Capacity Building Services Feedback Survey (Center for Courts instrument)

OMB #: 0970-0576

Expiration Date: XX/XX/XXXX

PAPERWORK REDUCTION ACT OF 1995 (Pub. L. 104-13) STATEMENT OF PUBLIC BURDEN: This collection of information will be used to assess usefulness of the Center for Courts’ Academies and participant satisfaction to help the Center make adjustments to improve future Academies. Public reporting burden for this collection of information is estimated to average 15 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. This is a voluntary collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information subject to the requirements of the Paperwork Reduction Act of 1995, unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: 0970-0576, Exp: XX/XX/XXXX. If you have any comments on this collection of information, please contact Scott Trowbridge, ACF, Administration on Children, Youth and Families (ACYF) by e-mail at [email protected].

CIP Capacity Building Services
Feedback Survey

INTRODUCTION AND PURPOSE

This survey seeks feedback from Court Improvement Programs to help assess the Capacity Building Center for Courts (CBCC) services, products, and capacity building assistance. Your feedback is important to ensure the Center best serves you and the community of CIPs. Results will be used to help the Center identify how best to use our resources and improve services to CIPs. This survey is voluntary, and there is no penalty for not participating. The survey takes about 15 minutes to complete. Your answers will be kept private. We ask you to identify your CIP’s jurisdiction so that evaluators can combine data with other sources (e.g., administrative data on services provided) for analysis; but only aggregate data will be reported. Please complete one survey on behalf of the CIP program, consulting with key team members as needed to represent the general experience of your CIP. If you have questions, please contact Kristen Woodruff, CBCC Evaluator, at [email protected].

If you proceed and respond to the survey, you are indicating your consent to participate in the evaluation. Thank you for your participation!

SURVEY QUESTIONS

1. Please indicate the state or U.S. Territory your CIP serves:

[DROP BOX – Listing all states/territories]

Please share your CIP’s experiences with CBCC capacity-building services during the last year, [insert last year].

2. Which 3 services have been MOST helpful to your CIP in moving your court improvement projects forward?

Select 3 MOST helpful services:
	Consultation, coaching, or other one-on-one state-specific (tailored) assistance from my CBCC liaison or other CBCC staff to support a CIP project
	CFSR PIP Intensives (CBCC one-on-one services to facilitate my CIPs work with the child welfare agency and other partners to create a shared vision for the CFSR PIP)
	CQI Workshops
	Judicial Academy
	Constituency Group Calls (Data and Evaluation CG, Family First CG, Hearing Quality CG, ICWA CG, Quality Legal Representation CG, Race Equity CG, Virtual Hearings CG, and New Directors CG)
	CIP All Calls (National)
	CIP Talks
	Online Learning Experiences available on CapLEARN (Child Safety Training for Judges and Attorneys; Hearing Quality: A Guide for CIPs and Court Reformers; Improving the Adoption Court Process: A Guide for CIPs and Court Reformers; Indian Child Welfare Act; Quality Legal Representation; Representing Parents in Child Welfare Cases)
	CBCC’s Online Learning Experience Course Template
	The New CIPSHARE
	CBCC Quick Sheets/Publications
	Other (describe) _________________________________________

3. Which 3 services have been LEAST helpful to your CIP in moving your court improvement projects forward?

Select 3 LEAST helpful services:
	Consultation, coaching, or other one-on-one state-specific (tailored) assistance from my CBCC liaison or other CBCC staff to support a CIP project
	CFSR PIP Intensives (CBCC one-on-one services to facilitate my CIPs work with the child welfare agency and other partners to create a shared vision for the CFSR PIP)
	CQI Workshops
	Judicial Academy
	Constituency Group Calls (Data and Evaluation CG, Family First CG, Hearing Quality CG, ICWA CG, Quality Legal Representation CG, Race Equity CG, Virtual Hearings CG, and New Directors CG)
	CIP All Calls (National)
	CIP Talks
	Online Learning Experiences available on CapLEARN (Child Safety Training for Judges and Attorneys; Hearing Quality: A Guide for CIPs and Court Reformers; Improving the Adoption Court Process: A Guide for CIPs and Court Reformers; Indian Child Welfare Act; Quality Legal Representation; Representing Parents in Child Welfare Cases)
	CBCC’s Online Learning Experience Course Template
	The New CIPSHARE
	CBCC Quick Sheets/Publications
	Other (describe) _________________________________________

4. Please rate your CIP’s overall experiences with CBCC services received during the last year. Indicate your level of agreement with the statement from (1) Strongly Disagree to (5) Strongly Agree.

	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5
Overall, my CIP was satisfied with the services received or used					
CBCC services were well aligned with the needs of my CIP					
My CIP’s relationship with the CBCC liaison or staff makes CBCC services easily accessible					
My CIP gained knowledge or skills through CBCC services that support court improvement efforts					
My CIP applied knowledge or skills acquired through CBCC services					
CBCC services strengthened my CIP’s capacity to implement effective court improvement strategies					
CBCC services strengthened my CIP’s capacity to evaluate court improvement strategies					
My CIP’s limited resources prevented use of CBCC services					
My CIP does not need capacity building services to support our court improvement strategies					

5. What do you want CBCC service providers to know about how best to support your CIP’s work?

6. Please rate your CIP’s experience with your CBCC Liaison and/or CBCC staff providing one-on-one state-specific (tailored) consultation, coaching or other assistance during the last year. Indicate your level of agreement with the statement from (1) Strongly Disagree to (5) Strongly Agree. If your experience with your liaison was different than with other staff you worked with, clarify your experience in the open-ended question #9.

If assistance in the specified area was not needed, select the last option “NA – Not Needed”.

My CBCC liaison and/or staff…	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5	NA – Not Needed
Was easily accessible to my CIP						
Actively listened to my CIP’s needs and request for assistance						
Helped my CIP identify our greatest needs						
Provided subject-matter expertise aligned with my CIP’s needs						
Connected my CIP to tools and resources to support our work						
Enhanced my CIP’s capacity to move a project forward						

7. How helpful was the one-on-one state-specific assistance for your CIP’s work?
   
   
   

Not Helpful	Moderately Helpful	Very Helpful
		

8. How important is it to have one-on-one state specific services to support your CIP?

Not at all Important	Moderately Important	Very Important
		

9. How can CBCC’s one-on-one state-specific services be improved to better support your CIP’s work?

10. Have you or another CIP staff member participated in the monthly CIP All Call during the last year?

Yes

No [If “No” then skip to #13]

11. How helpful were the CIP All Calls, overall, to your CIP’s work?

Not Helpful	Moderately Helpful	Very Helpful
		

12. Please rate your overall experiences with the CIP All Call during the last year, from (1) Strongly Disagree to (5) Strongly Agree.

	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5
The information provided through the CIP All Calls was directly applicable to my work.					
The knowledge gained from the CIP All Call helped move my CIP’s work forward					
The information provided addressed emerging issues in the field					

13. Has your CIP participated in one or more of CBCC’s Constituency Group calls* during the last year?
    
    

*Constituency Groups include: Data and Evaluation CG, Family First CG, Hearing Quality CG, ICWA CG, Quality Legal Representation CG, Race Equity CG, Virtual Hearings CG, and New Directors CG.

Yes

No [If “No” then skip to #20]

14. How helpful were CBCC’s Constituency Group Calls for your CIP’s work during the last year?
    If you/your CIP did not participate in a listed CG, select “NA-Did not participate”.

Constituency Group:	Not Helpful	Moderately Helpful	Very Helpful	NA-Did not participate
Data and Evaluation CG				
Family First CG				
Hearing Quality CG				
ICWA CG				
New Directors CG				
Quality Legal Representation CG				
Race Equity CG				
Virtual Hearings CG				

15. In what ways has your CIP applied the information from one or more Constituency Group Calls in your work? Check all that apply.

 Plan/Implement new strategies  Support program improvement  Support policy development  Train staff/colleagues  Develop bench cards or other practice tools  Develop data collection tools to support evaluation  Conduct research and evaluation

 Provide information to clients/families  Support public awareness and advocacy  Share information with peers  Grant writing/fundraising  Other (describe):__________________________ ____________________________  I have not yet applied the information to my work

16. Which Constituency Group (CG) did you participate in the most during the last year? If you participated equally, select one CG you wish to provide feedback for.

Data and Evaluation CG

Family First CG

Hearing Quality CG

ICWA CG

New Directors CG

Quality Legal Representation CG

Race Equity CG

Virtual Hearings CG

17. Please rate your experiences with the Constituency Group that you selected (above) during the last year, from (1) Strongly Disagree to (5) Strongly Agree.
    
    

	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5
The Constituency Group provided an effective platform for CIPs to share resources and lessons learned about this topic					
I have discovered useful tools, ideas and ways of thinking from the relationships developed during the Constituency Group(s).					
My CIP gained knowledge through the Constituency Group that has helped move our court improvement work forward					
I feel confident in my ability to implement strategies related to the topic as a result of my participation in the Constituency Group.					
As a result of my involvement in the Constituency Group I have improved my connections with peers/colleagues.					

18. What aspects of the Constituency Group calls were most relevant and useful to your work? If you are thinking of a specific Constituency Group, please specify which group.
    
    

19. How can CBCC’s Constituency Group Calls be improved to better support your CIP’s work? If you are thinking of a specific Constituency Group, please specify which group. If there are specific topics you would like, specify those.

20. Have you used the New CIPSHARE?

The New CIPSHARE provides an online platform for CIP network members to share resources, tools and information and download files of interest. We rolled out the New CIPSHARE in April 2021.

Never heard of it [If “Never” then skip to #24, OLE questions]

Heard of it, but haven’t tried it yet [If “haven’t tried it” then skip to #24, OLE questions]

Used it once or twice

Use it regularly

21. How helpful was the New CIPSHARE for your CIP’s work?

Not Helpful	Moderately Helpful	Very Helpful
		

22. Please rate the New CIPSHARE by indicating your overall level of agreement with each statement, from (1) Strongly Disagree to (5) Strongly Agree.

	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5
CIPSHARE is easy to login to.					
It’s easy to find what I’m looking for on CIPSHARE.					
CIPSHARE provides useful resources to support my CIP’s work					
CIPSHARE facilitates peer-to-peer learning and information sharing among CIPs.					

23. What is the one thing that you would change about CIPSHARE?

24. Have you used any of CBCC’s Online Learning Experiences (OLEs)?
    
    

CBCC’s OLEs are online courses available on CapLEARN that cover a range of topics including: Child Safety Training for Judges and Attorneys; Hearing Quality: A Guide for CIPs and Court Reformers; Improving the Adoption Court Process: A Guide for CIPs and Court Reformers; Indian Child Welfare Act; Quality Legal Representation; and Representing Parents in Child Welfare Cases.

Never heard of them [SKIP TO #27]

Heard of the OLEs but have never used them [SKIP TO #27]

Used one or more OLE’s, but it’s been more than a year [PROCEED TO NEXT QUESTION]

Used one or more OLE’s during the last year [PROCEED TO NEXT QUESTION]

25. How helpful was the Online Learning Experience (OLE) course(s) to your CIP’s work?

Not Helpful	Moderately Helpful	Very Helpful
		

26. Has your CIP incorporated one or more of CBCC’s OLE course(s) as a tool in your CIP’s training program for judges, attorneys or other legal professionals?
    
    
    

• Yes

• No

27. Has your CIP used CBCC’s OLE template to build your own online course?
    
    The OLE template provides the ability for CIPs to build their own courses, populating the template with their own course content.

• Yes, my CIP has used CBCC’s OLE template to build our own online course.

• No, I know about the template but my CIP has not used it. [If “No”, skip to 31]

• No, never heard of the template [If “No” skip to 31]

28. Did you receive the support you needed from your CBCC liaison or other CBCC staff when using the template?
    
    
    

• Yes

• No

• Didn’t need support

29. How helpful was the OLE template as a tool in building your CIP’s training for legal professionals?

Not Helpful	Moderately Helpful	Very Helpful
		

30. What is the one thing that you would change about the OLE template?

31. Have you reviewed or used a CBCC Quick Sheet in the last year?

Quick Sheets are CBCC’s brief informational summaries (one to two pages) on topics such as the change management process, developing a theory of change, hearing quality, quality parent representation, ICWA compliance, effective trainings, and data collection and evaluation methods. Quick Sheets are now available in CIPSHARE (search “Quick Sheets”).

Yes, used quick sheet(s) regularly [PROCEED TO NEXT QUESTION]

Yes, used quick sheet(s) once or twice [PROCEED TO NEXT QUESTION]

No [If “No” skip to #33]

32. How helpful were the CBCC Quick Sheets in your CIP’s work?

Not Helpful	Moderately Helpful	Very Helpful
		

33. What are the best ways to provide your CIP with tools, resources, reports, Quick Sheets, and other publications? Check all that apply.

 CIPSHARE

 Distribute during All Calls

 Distribute during CBCC Constituency Group meetings, CQI Workshops and other events

 Email over the listserv

 My CBCC Liaison (when discussing my CIP’s needs)

 Post on the Capacity Building Collaborative web site

34. How would you rate the impact of CBCC’s services on your CIP’s capacity to use the following aspects of the change management process during the last year?

Indicate your level of agreement from (1) Strongly Disagree to (5) Strongly Agree. If your CIP did not need assistance in the designated area this year, mark “NA – Not needed.”

CBCC services strengthened my CIP’s capacity to…	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5	NA – Not needed
Expand our use of data to identify and assess needs						
Develop a clear theory of change to explain how court strategies will lead to improved outcomes						
Select or adapt an intervention that addresses root causes of an identified problem						
Implement new strategies to enhance court performance						
Expand our use of data to monitor progress on project outcomes						
Evaluate the effectiveness of court improvement strategies (e.g., develop evaluation designs, measures/tools, and/or analytic approaches)						

35. How would you rate the impact of CBCC’s services on your CIP’s capacity to improve the quality of legal representation during the last year?

CBCC services increased my CIP’s…	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5	NA - Not needed
Awareness of strategies to improve the quality of legal representation						
Capacity to implement strategies to promote high quality legal representation						
Capacity to evaluate strategies to improve the quality of legal representation						

36. How would you rate the impact of CBCC’s services on your CIP’s capacity to improve hearing quality and judicial practice during the last year?
    

CBCC services increased my CIP’s…	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5	NA - Not needed
Awareness of strategies to improve hearing quality and judicial practice						
Capacity to implement strategies to improve hearing quality and judicial practice						
Capacity to evaluate strategies to improve hearing quality and judicial practice						

37. How would you rate the impact of CBCC’s services on your CIP’s awareness of resources and tools during the last year?
    
    

CBCC services increased my CIP’s…	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5	NA - Not needed
Awareness of tools and resources to implement CIP strategies						
Awareness of tools and resources to evaluate CIP strategies						
Access to tools and resources to implement and/or evaluate CIP strategies						

38. How would you rate the impact of CBCC’s services on your CIP’s peer connections during the last year?

As a result of my CIP’s participation in CBCC services…	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5	NA – Not Needed
My CIP has strengthened peer connections to share ideas, tools, and resources						
My CIP is engaged in a regional community of CIPs to share ideas, tools and resources						
My CIP is engaged in a national community of practice to improve court performance and child and family outcomes						

39. What barrier(s) limit your CIP from connecting with other CIPs to share ideas, tools and resources? Check all that apply.

 Not sure who to contact (for the subject matter I need, or want to share)

 Not sure how to get contact information for other CIPs

 Not enough time

 I have reached out, but other CIP(s) did not respond

 No reason to contact other CIPs

 Other: ________________________________________________________

 None. My CIP has what we need to reach out to CIP peers in other jurisdictions

40. How would you rate the impact of CBCC’s services on your CIP’s capacity to collaborate with the child welfare agency and others during the last year?
    
    

	Strongly Disagree 1	Disagree 2	Neither 3	Agree 4	Strongly Agree 5	NA - Not needed
CBCC services helped my CIP and child welfare agency engage in meaningful collaboration to work toward shared goals						
CBCC services helped my CIP increase interaction with community stakeholders (e.g., youth, families, courts/judges) in planning court improvement efforts						

41. Which topics would you be most interested in seeing future CBCC services focus on?

Addressing bias and racial equity Conducting quality virtual hearings Conducting quality virtual trainings Hearing quality and judicial decision-making (successful strategies, bench cards) High quality legal representation (successful programs, implementation) Reasonable efforts Strategic planning Training curricula Using Title IV-E funds effectively (quality legal representation, prevention, etc.)

Change management process Evaluation methods Evaluation design/planning Effectively evaluating trainings Data collection methods Hearing quality court observation tool Data analysis methods Data visualization Understanding and interpreting data

Describe any other topics and/or more specific topics you are most interested in:

42. Is there anything else you would like to share about your experience with CBCC’s services?

43. Would you be willing to participate in a brief (30 minute) follow up interview with an evaluator to share additional feedback on capacity-building services and discuss how we can improve services to better support CIPs?

Yes, an evaluator may contact the CIP Director/Coordinator in my jurisdiction to arrange an interview

No, I do not wish to participate in an interview

Thank you for your participation!

Your responses are important to us and will help inform CBCC’s delivery of services in the future. If you have any questions or concerns about the survey, please contact Dr. Kristen Woodruff at [email protected]. Thank you!