ACF Responses to Public Comment on CFSR (0970-0214)

CFSR Information Collection

0970-0214

Response Summary

The Department requested comments on:

1. whether the proposed collection of information is necessary for the proper performance of the functions of the agency, including whether the information shall have practical utility;

One commenter expressed that the established timeframe for the period under review does not consider current progress and practice changes made by a system. The Children’s Bureau (CB) chooses to use a 12-month Period Under Review for CFSR onsite case reviews and would contend that the past 12-months constitutes a recent period of time. Information collected helps CB determine whether the state is operating in substantial conformity with federal requirements and using a year accounts for seasonal fluctuations in child welfare data that may not be captured if a shorter Period Under Review is used. Twelve-months has been the general length of Periods Under Review used for all rounds of CFSRs, and twelve-month reporting periods are also used for the statewide data indicators.

Some commenters expressed concerns about the Onsite Review Instrument (OSRI) and the Online Monitoring System (OMS)- the web-based application that automates the OSRI. They noted limitations about the data collected in the OSRI and its usefulness in helping states understand their system, as well as the limitations with the functionality of OMS. In Child and Family Services Review Technical Bulletin #12, August 2020, the Children’s Bureau stated that for Round 3, the OSRI was adapted to improve the types of information collected on practices and services provided to children and families to better assess how their safety, permanency, and well-being needs were met by states. These adaptations will carry forward in Round 4 as well. CB developed and instituted the OMS to support electronic data collection and storage of the OSRI, which includes an array of reports states can run to analyze case review results to assist in determining strengths and areas of need related to practices and services. As such, the logic built into OMS provides the greatest flexibility that is feasible for states to conduct reviews.

We received one comment on a specific case and we acknowledge receipt of the comment.

2. the accuracy of the agency’s estimate of the burden of the proposed collection of information;

While one commenter agreed with the burden estimate for the Statewide Assessment, we received a few others indicating the estimates for the Onsite Review and Program Improvement Plan are too low. The Children’s Bureau provides an “average” estimate for each of these processes, but it will vary based on the capacity and resources of the state. Training, turnover and whether a state opts out of maintaining their Continuous Quality Improvement (CQI) systems- including their case reviews-will impact the amount of burden any given state experiences. CB’s CQI Information Memorandum (IM) (ACYF–CB-IM-12-07), CB encouraged and advised states to maintain their quality assurance systems and enhance them through a more comprehensive continuous quality improvement approach. CB believes that by continuing these efforts will help reduce the burden of “standing up” a new CQI system and facilitate a more seamless transition to the CFSR.

In CFSR Technical Bulletin #12, August 2020 (TB12), CB proposes for Round 4 to replicate many aspects of the Round 3 PIP approach, that was piloted leading which led to PIPs that were developed and approved in a timelier manner. This approach relied on convening a diverse and wide body of stakeholders to review data and challenges identify solutions that could result in improved outcomes. TB12 also notes that since the Round 4 Statewide Assessment (SWA) approach will include an earlier focus on statewide data, along with the more deliberate inclusion of system stakeholders, the essential components driving the PIP pilots will already be in place. The Children’s Bureau will recommend a continued reliance on and augmentation of aggregate data, along with the persistent integration of stakeholder perspective throughout the SWA and the onsite phase of the CFSR. This allows for the initial work conducted in the SWA to be a more integral part of the PIP development.

(c) the quality, utility, and clarity of the information to be collected;

Several comments were submitted about the revisions to the CFSR collection instruments, particularly regarding the OSRI. We received support for the clarification provided in the instructions to the instruments, but some comments received indicated further clarity about OSRI Item 2: Services to Family to Protect Children in the Home and Prevent Removal or Re-Entry into Foster Care is desired. Specifically, there seems to be confusion about how to apply Item 2 as it pertains to applicability of safety plans with alternate caregivers. One commenter suggested that the revisions related to service provision for alternative caregivers is a “new concept” and the expectations surrounding it are unclear. Commenters requested additional guidance in the instructions on determining whether services were “timely” and “appropriate” as well as differentiating between services often captured in Item 2 versus those rated in OSRI Item 12: Needs and Services of Child, Parents, and Foster Parents.

The revisions proposed for the Round 4 OSRI Item 2’s applicability criteria, definitions and instructions incorporated Round 3’s FAQ guidance located in the OMS and the OSRI online training. While no new requirements were added for the item, the applicability is now driven by the need for services to address any safety and/or risk of child abuse and/or neglect and to prevent foster care entry/re-entry. This includes services for children to remain safely in the home or with an alternative caregiver arrangement. The definitions were updated to provide additional clarity and remain flexible enough to account for unique case circumstances. The CB intends to issue a brief on OSRI Item 2 to assist reviewers in applying the item in Round 4, which will include an emphasis on applicability criteria, questions that inform rating determination and making a distinction between appropriate services addressed under Item 2 and other items in the OSRI such as Item 12.

A request was made by one commenter to add back in the two questions in OSRI Item 9 that were removed in the current revisions. These questions were related to inquiring about federally recognized tribal membership and the notification to tribes of their right to intervene in child welfare proceedings- in other words, they were a means to determine compliance with the Indian Child Welfare Act (ICWA) sections 25 CFR §§ 23.107 and 23.111 and in support of Tribal family connections. While CB provides funding to eligible tribes under titles IV-B and IV-E, it does not have authority to ensure compliance with the requirements cited in these question (25 CFR 23.107 and 23.111); these statutes relate to the Bureau of Indian Affairs at the Department of the Interior. After careful consideration, ICWA compliance questions were removed from the OSRI The CB does ensure that family relationships and connections are preserved for children in foster care, including the important connections with Tribes based on possible or confirmed Tribal affiliation or cultural self-identification by a family. The OSRI provides examples of ways we would expect to see that Tribal connection supported for all children in foster care including coordination with Tribes and Tribal courts. In addition, the CFSR is conducted in partnership with state child welfare agencies and other stakeholders and considers Tribes to be an important participant in the review process.

One commenter noted that despite the changes to the OSRI, it still does not fit cases with only “Initial Assessment (investigation) cases”. On August 6, 2021, the Children’s Bureau issued the Child and Family Services Review Technical Bulleting #13 that states for “CFSR Round 4, the in-home services case population will not include families who only received a Child Protective Services (CPS) investigation and/or assessment response.” Thus, the issue of overriding items that are non-applicable will be avoided and cases that were never intended to be open for services will not be in the review sample.

4. ways to minimize the burden of the collection of information on respondents, including through the use of automated collection techniques or other forms of information technology

One commenter requested the continued use of context data and other administrative data as a means to reduce the burden on the collection of information. The commenter suggested If the context data demonstrates performance outside of the national standard in statistically significantly negative deviations, then only those related data points would be reviewed within the context of the OSRI data points. In CFSR Round 4, the CB will return to the planned use of statewide data indicators (SWDI)- data available in states’ Adoption and Foster Care Analysis and National Child Abuse and Neglect System submissions- to assess adjusted state performance on safety and permanency outcomes relative to the observed national performance. For this round, the SWDI will include all seven metrics initially intended for use in Round 3. The SWDI will be supplemented with additional context data, which states may consult as they conduct their statewide assessments. The SWDI will be used to establish substantial conformity on certain safety and permanency outcomes. In addition, those indicators will be used to set baselines and goals for states required to include non-conformant outcomes in their PIPs.

Another commenter requested that states be permitted to complete stakeholder interviews along with the rest of the onsite review in a virtual environment as was done in many states due to the COVID-19 pandemic. It was noted that the virtual reviews created efficiencies that were otherwise not there due to the in-person review and interview process. In May 2020, CB issued the document CFSR/CQI process During the Covid-19 Pandemic (https://www.acf.hhs.gov/sites/default/files/documents/cb/covid19_case_review.pdf); which offered suggestions to states related to case reviews to help agencies maintain their oversight capacity and state connected in a remote-work environment. The CB understands that many lessons are being learned through this cycle of virtual and remote work and that they may be incorporated into the CFSR process in the future.