Evaluation of the Center for Legal and Judicial Innovation and Advancement (CLJIA)
(previously Evaluation of the Capacity Building Collaborative)
OMB Information Collection Request
0970 - 0576
Supporting Statement Part B –
Statistical Methods
June 2025
Type of Request: Revision
Submitted By:
Children’s Bureau
Administration on Children, Youth and Families
Administration for Children and Families
U.S. Department of Health and Human Services
Respondent Universe and Sampling Methods
Data collection for the Center for Legal and Judicial Innovation and Advancement (CLJIA) Academies and Workshops targets the universe of participants in these services, including court improvement programs (CIP) staff and court professionals (e.g., judges, attorneys) who participate in CLJIA services, specifically Academies and Workshops. This data collection uses a census – all participants in the Academies and Workshops are invited to participate in the surveys/assessments—so the universe of respondents and sample size estimates are equivalent. The universe of respondents (and equivalent sample size) is estimated based on the estimated attendance and number of Academies and Workshops to be hosted over the five-year period.
Table 1. Respondent Universe and Sample Size
Instrument |
Universe of Respondents |
Sample Size |
Workshop Feedback Survey |
480 |
480 |
Academy Feedback Survey |
1050 |
1050 |
Pre/Post Academy Assessment |
1050 |
1050 |
During prior data collections response rates for Workshop feedback surveys varied depending on the mode of delivery, with high response rates following in person Workshops (88%) and very low response rates for virtual Workshops that included services over an extended timeframe (8%). We anticipate most Workshops moving forward will be held in person, yielding higher response rates.
The Academy Feedback Surveys have yielded response rates of 57% (Attorney Academies 57% and Judicial Academies 56%). Pre/post assessments, which are part of instructional design, have yielded higher response rates of 82% (Attorney Academies 84% and Judicial Academies 66%).
Procedures for the Collection of Information
Data will be collected from CIP staff and court professionals (e.g., judges, attorneys) who participate in CLJIA Academies and Workshops. This data collection primarily uses web-based instruments but may use paper surveys for in person meetings in some jurisdictions, as described below. Appendix 2 provides the recruitment language, same as provided in the original OMB application.
Participants in workshops will be invited to complete the CQI Workshop Feedback Survey anonymously at the close of the workshop. Workshop participants may include CIP project team members and other workshop participants (e.g., CB staff). Participants in the Academies will be invited to take the Academy Feedback Survey at the close of the academy. Participants may include judges, attorneys, other officers of the court, and other participants (e.g., CB staff). For both surveys, when the Workshop or Academy is held in person, a hard copy survey will be included with meeting materials, and when it is conducted online, a designated CLJIA team member will provide a link to an online survey at the close of the learning experience. Following the learning experience, participants may receive up to two reminder emails to complete the survey, if they have not already done so.
The Pre/Post Academy Assessment will be collected by CLJIA from Academy participants. Prior to attending the Academy, all registrants will receive an invitation to participate in an online learning experience (via CLJIA’s Academy module), which includes the pre-assessment. Registrants will receive up to two reminder emails prior to the Academy if they have not completed the pre- assessment. At the close of the two-day Academy, participants will be invited to return to the Academy website to complete the post-assessment, and may receive up to two reminder emails.
These surveys are designed to limit response options to provided values for closed-ended questions, which eliminates the possibility of multiple answers for a single item when participants complete the online version. For the hard copy, respondents circle their response, which reduces issues with illegibility. The evaluation team will analyze survey data and identify and document any issues with data collection quality and consistency, for discussion with the CLJIA team and amelioration.
Statistical methodology
Since this is a census, and the sample size is the same as the universe, no statistical methodology is used for sample selection or stratification and no estimation procedures apply.
Methods to Maximize Response Rates and Deal with Nonresponse
Response Rates
Maximizing response rates is critical to the administration of these data collection efforts. Though these data collection activities are not designed to produce statistically generalizable findings and participation in the data collection activities is wholly at the respondents’ discretion, response rates will be collected for quality improvement purposes.
Data collection strategies that emphasize flexibility, privacy, and a respect for the respondent’s time facilitate timely participation. The following strategies will be implemented to maximize participation in the data collection.:
Administration: For surveys, reminder emails will be sent (per discussion above) to promote participation and a higher response.
Alternate response methods: Respondents will be given the option to use an alternate method for responding to surveys, such as opting between a paper or web-based survey for in person learning experiences, if this helps to increase participation.
Assurances of data privacy: Respondents to all surveys and interviews will be assured that reported data are aggregated and not attributable to individuals or specific child welfare jurisdictions.
Non-Response
As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated. The evaluation team will, however, track response rates and report them along with findings.
Test of Procedures or Methods to be Undertaken
No tests of procedures are planned, as this request simply proposes to extend the existing data collection activities for ongoing CQI as CLJIA serves more members of the legal and judicial community.
Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
Kristen Woodruff, PhD, MSW
Senior Study Director
Westat / Center for Legal and Judicial Innovation and Advancement
Jennifer Renne, JD
Director
American Bar Association / Center for Legal and Judicial Innovation and Advancement
Attachments
Appendix 1 – Relevant Child Abuse Prevention and Treatment Act (CAPTA) Sections
Appendix 2 – Recruitment and Reminder Language
Instrument 1 – Workshop Feedback Survey
Instrument 2 – Academy Feedback Survey
Instrument 3 – Academy Pre/Post Assessment
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Marneena Evans |
| File Modified | 0000-00-00 |
| File Created | 2025-06-15 |