SSA - Engaging Young People With Lived Experience in the CFSR Process

Engaging Young People With Lived Experience in the Child and Family Services Review (CFSR) Process

Formative Data Collections for Program Support

0970 – 0531

Supporting Statement

Part A - Justification

November 2021

Submitted By:

Children’s Bureau

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

A1. Necessity for the Data Collection

The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval for collecting information about the most effective strategies to engage young people who have lived experience in the child welfare system in the Child and Family Services Review (CFSR) process. The CFSR process is designed to meet the statutory requirement to provide federal oversight of states’ compliance with title IV-B and IV-E plan requirements, strengthen state child welfare programs, and improve safety, permanency, and well-being outcomes for children and families served. The CFSR is a multi-phase process:

1. The state completes a self-assessment

2. The state conducts an onsite review of cases and interviews with stakeholders

3. The Children’s Bureau (CB) provides their findings to the state

4. The state develops and implements a plan to address areas on nonconformity with federal plan requirements.

The CFSR regulations require states to consult with individuals (e.g., parents and children) to develop the CFSR (see 45 CFR 1357.15(l)). There have been three rounds of CFSR development and round four will begin soon. For more information about the CFSR go to www.acf.hhs.gov/cb/monitoring/child-family-services-reviews.The information collection described here will solicit opinions from young people to better understand how they would like to be engaged in all components of the upcoming Round 4 CFSR process.

This proposed information collection meets the following goals of ACF’s generic clearance for formative data collections for program support (0970-0531):

• The development or refinement of program and grantee processes.

• Planning for provision of programmatic or evaluation-related training or technical assistance (T/TA).

Study Background

While persons with lived experience (children, youth, and parents) were involved in previous rounds of the CFSRs, their involvement was limited primarily to providing information during the onsite review phase of the CFSR through case-related interviews and stakeholder interviews. To expand involvement, CB requested that states recruit and train young people with lived experience to be case reviewers. While there were benefits to this model, it was not as successful as CB had hoped to provide valuable information to states to inform development of the CFSR.

After states complete their CFSR, they may be required to develop a Program Improvement Plan (PIP), which outlines strategies for improving safety, permanency and well-being. In the previous round of CFSR development (Round 3), parents and youth with lived experience joined child welfare agency staff and child welfare system partners and stakeholders in a review of CFSR results, state administrative data, and stakeholder data and information to identify root causes (factors) of performance that does not meet CSFR standards and identify and select PIP improvement strategies.

For round 4 reviews, CB has announced changes to the review process. Guidance to states (Technical Bulletin #12¹) provides information on the scheduling of reviews and emphasizes that an important element of the CFSR Round 4 structure and process is to engage states, key stakeholders, and capacity-building providers in a streamlined, focused, and comprehensive PIP development process. The process builds on evidence acquired through the statewide assessment and CFSR, and results in the identification of cross-cutting themes, which when addressed, improve system functioning, child welfare practices, and the resulting outcomes for children, youth, and families. With their unique perspective, young people with lived experience can play a significant role in building the evidence, identifying cross-cutting themes, and developing strategies to address important concerns.

To address limitations identified during Round 3 limitations and to align with Round 4 CFSR requirements, the CB is proposing to conduct focus groups with young persons with lived experience to understand the best methods of recruiting, engaging, supporting, and retaining young persons with lived experience in all aspects of the CFSRs.

Legal or Administrative Requirements that Necessitate the Collection

There are no legal or administrative requirements that necessitate the collection. ACF is undertaking the collection at the discretion of the agency.

A2. Purpose of Survey and Data Collection Procedures

Overview of Purpose and Use

The CB has recognized that all young people should be active participants in shaping their lives and directing their futures. Authentic youth engagement works when young people and supportive adults work together as partners and leads to positive outcomes for youth, families, and communities. It injects young people as valued stakeholders into creating effective and inclusive policies, programs, and environments.²

The purpose of the proposed focus groups is to collect information from young persons with lived experience on the best methods of recruiting, engaging, supporting, and retaining these young people in all aspects of the CFSRs. The information collected will:

• Enable the CB and states to obtain more comprehensive information on how state child welfare systems are functioning, including identifying strengths and challenges;

• Enable the CB and states to better understand the data and information collected during the statewide assessment and onsite review phases of the CFSR and, as a result, be better positioned to develop promising PIP strategies to address areas of nonconformity;

• Better support states in implementation of PIP strategies, including making adjustments as needed to ensure better outcomes; and

• Enhance the CB’s ability to monitor PIP implementation by raising awareness of the perspective of those with lived experience.

The information collected will be used internally for planning, program support, and technical assistance. The CB will also share lessons learned with the states to build their capacity (i.e., themes from the results, but not the full focus group transcripts).

The following components are foundational considerations in implementing this proposal and focusing CB’s efforts. They are posed as questions to be answered:

• What are some effective strategies for the recruitment and engagement of young people with lived experience?

• What are the potential roles or avenues for involvement within the CFSR process that young people with lived experience identify for themselves?

• What training, resources, and supports do young people with lived experience need to be successful and effective within their role(s)?

• What training, resources, and supports do those engaged in the CFSR process (e.g., child welfare agencies, the legal and judicial communities, child welfare professionals, federal staff) need to work effectively with young people with lived experience?

Information Collection Processes

To answer these questions, CB proposes to conduct focus groups with young people who have lived experience in the child welfare system. There will be 2 sets of 4 focus groups, with no more than 10 participants in each set – thus 20 participants total (participant invitation and selection information provided below; the participants in each set will share similar backgrounds/experiences to each other, i.e., lived child welfare experience). Each participant will attend 4 focus groups, the first three cover a specific area of the CFSR process while the fourth is a debrief session. This will provide a rich perspective on young people’s involvement in the CFSRs. The 4 areas include:

• Statewide Assessment and statewide data indicators (focus group 1)

• Onsite Review (focus group 2)

• Program Improvement Plan development and monitoring (focus group 3)

• Wrap-Up and Debrief (focus group 4)

Participant invitation and selection: The two different sets of 10 participants each will be invited and selected from the following:

• Set 1: Set 1 participants will be invited and selected from the pool of current and former National Youth in Transition Database (NYTD) Reviewers and Capacity Building Center for States (CBCS) Young Adult Consultants (YAC). In addition to their lived experience, these individuals have experience working alongside federal staff to provide assistance to states and evaluate state programs. Given that CB does not want to constrain group members’ input by what has been done in the past, we would invite individuals who have no prior experience providing technical assistance to states on CFSRs.

• Set 2: Set 2 participants will be invited and selected from the pool of young people involved with national organizations. These organizations include Congressional Coalition on Adoption- Foster Youth Internship Program, Foster Care Alumni of America (FCAA), Anne E. Casey Foundation/Jim Casey Youth Opportunity Initiatives & Young Fellows, FosterClub, National Foster Care Youth & Alumni Policy Council (Foster Club, FCAA, and Private Foundations), and Think Of Us. These individuals can bring an alternative perspective that is not informed by experiences partnering with federal staff on monitoring efforts and may provide insights on recruiting and engaging persons with lived experience in this work. An email describing the focus groups will be sent to these organizations and we will use convenience sampling to invite and select participants from these organizations on a first come, first serve basis (see Recruitment Appendix).

Focus groups: Each focus group will be scheduled for no more than 60 minutes. CB has contracted with JBS International to conduct this information collection. JBS International staff members of the Child Welfare Reviews Project (CWRP), including coaches (see below), will carry out the participant invitation and selection, focus groups, and analysis.

So that each focus group is most effective, we will prepare information and materials explaining the CFSR process and provide them to the participants before focus groups one, two and three (e.g., the CFSR General Fact Sheet from Round 3). We will gather informal feedback (via a conversation) at the end of each preparation session to understand if we are being effective and to inform subsequent preparation sessions. Preparation sessions will be 60 minutes, including the gathering of informal feedback. In total, participants will be asked to attend the following:

• Preparation session 1: Information about the CFSRs, the statewide assessment, and the statewide data indicators

• Focus group 1: how to engage young people in the statewide assessment and the statewide data indicators

• Preparation Session 2: Information about the onsite review

• Focus group 2: How to engage young people in the onsite review

• Preparation Session 3: Information about Program Improvement Plan (PIP) development and monitoring

• Focus group 3: How to engage young people in the PIP process

• Focus group 4: Wrap-up and Debrief

Recognizing that many young persons have jobs and/or are in college, we will hold each preparation session and each focus group outside of traditional work or class hours, if necessary.

At the beginning of each focus group, participants will be asked to verbally consent to being recorded and to participating in the focus groups. Participation in all the activities is voluntary.

Focus group Facilitators--Coaches: We will model meaningful involvement of young people throughout the process by involving four young people with lived experience as “coaches”. Coaches provided input on the development and implementation of the study, e.g., crafting the invitations and developing the wording for the session topics/questions (see the Recruitment Appendix and the Instruments Appendix). The four coaches will facilitate the focus groups³ and help prepare participants in advance by providing additional context about the purpose of each session. They will also assist with compiling findings. Coaches are contracted staff for JBS International and are part of the project team.

A3. Improved Information Technology to Reduce Burden

Information technology will be used in the following ways:

• The focus groups will occur online using Microsoft Teams. Microsoft Teams was chosen in part because it will work in a mobile environment and youth are likely to be familiar with a mobile environment.

• Communication with participants will occur via email, including an invitation to participate in the focus group along with a web link.

• The focus groups will be recorded for analysis later provided that participants consent to being recorded. A message will be displayed to participants explaining that by entering the meeting, they are giving consent to participate in the focus groups and to be recorded. Participants will also be asked to verbally consent to participating in the focus groups.

This data collection method was chosen to solicit the opinions of participants and it does not require them to fill out forms/surveys or produce documents, which reduces burden. Meeting software such as Microsoft Teams allows focus groups to collect data virtually, which reduces burden.

A4. Efforts to Identify Duplication

This highly specific information about young people with lived experience and the CFSRs is not available elsewhere.

A5. Involvement of Small Organizations

Half of the focus group participants will be invited and selected from small organizations (see set 2, above). Staff from these small organizations (Congressional Coalition on Adoption- Foster Youth Internship Program, Foster Care Alumni of America (FCAA), Anne E. Casey Foundation/Jim Casey Youth Opportunity Initiatives & Young Fellows, FosterClub, National Foster Care Youth & Alumni Policy Council (Foster Club, FCAA, and Private Foundations), and Think Of Us) will be asked to share a recruitment email with relevant participants (i.e., young people with lived child welfare experience—those described in set 2, above). See the Recruitment Appendix for more information.

A6. Consequences of Less Frequent Data Collection

This is a one-time data collection.

A7. Special Circumstances

There are no special circumstances for the proposed data collection efforts.

A8. Federal Register Notice and Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of the overarching generic clearance for formative information collection. This notice was published on November 3, 2020, Volume 85, Number 213, page 69627, and provided a sixty-day period for public comment. During the notice and comment period, no substantive comments were received.

Consultation with Outside Experts

For this project, we consulted with 4 coaches, young people with lived experience. The coaches crafted the recruitment materials (see the Recruitment Appendix) and the focus group topics/questions (see the Instruments Appendix).

A9. Tokens of Appreciation for Respondents

No tokens of appreciation for respondents are proposed for this information collection.

A10. Privacy of Respondents

We will collect participant names, email addresses, and information required to provide renumeration (see A13 for additional information). Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law.

A11. Sensitive Questions

There are no sensitive questions in this data collection.

A12. Estimation of Information Collection Burden

Burden Estimates

Table 1 contains the estimated response burdens for the participants in the focus groups. Each participant will spend time preparing for the session, participating in the session, and then debriefing.

Table 1 – Estimated Burden for Focus group Participants

Task	#Participants	Hours per Task	Total hours	US Average Hourly Wage	Total Annual Cost
Preparation:  Reviewing materials, discussion with coaches.	20	3	60	$40.34	$2,420.4
Session Participation (3 focus groups and one debrief, one hour each)	20	4	80	$40.34	$3,227.2
Total	20	7	140	$40.34	$5,647.60

Cost Estimates

Table 1 also shows the cost estimate for the focus group participants. This was calculated using the Bureau of Labor Statistics (BLS) wage data from May 2020⁴ for the average hourly wage, which is $20.17 per hour. To account for fringe benefits and overhead the rate was multiplied by two which is $40.34.

A13. Cost Burden to Respondents or Record Keepers

Each focus group participant will be provided an honorarium of $250 in recognition of the time and expertise contributed to the data collection. Each participant will spend time preparing for sessions and participating in 4 one-hour long focus groups. This amount is based on the current amount provided to young adult consultants who serve as reviewers for the Children’s Bureau- NYTD. These reviewers receive $35/hour.

All potential participants will be provided with a consent form that details the tasks and activities and includes information for the distribution of the honorarium.

A14. Estimate of Cost to the Federal Government

The total cost for the data collection activities under this current request will be $50,570.00. These costs cover all aspects of internal planning meetings to discuss logistics, project design, instrumentation development; data collection (i.e., focus groups); and use of information technology for online data collection, data management, analysis, and reporting.

A15. Change in Burden

This is for an individual information collection under the umbrella formative generic clearance for program support (0970-0531).

A16. Plan and Time Schedule for Information Collection, Tabulation and Publication

Project Time Schedule. As shown in Table 4, the project will take place during the summer and fall of 2021, pending OMB approval.

Table 4. Project Time Schedule

Activity	Time Schedule
Obtain OMB approval	Fall 2021
Engage and Prepare Support Coaches	Summer 2021
Develop Topics and Questions	Summer 2021
Identify Participants	Fall 2021
Prepare Participants	Fall 2021
Conduct Focus groups	Fall 2021
Synthesize and Analyze Information	Fall 2021

Analysis Plan. Qualitative data will be collected from the focus groups and analyzed inductively to understand how to most effectively engage young people with lived experience in the CFSR process. As the first step in the data-cleaning process, recordings of the Focus groups will be transcribed and cleaned to remove any respondent-identifying information and any transcription mistakes. These transcriptions will serve as the qualitative data used for the study.

The initial step in the inductive analysis process will be reading the cleaned and coded interview transcripts and interview notes to discover underlying themes. The themes will then be grouped into lower order themes based on common topics. Next, following the same coding procedures for grouping interview themes, lower order themes will be grouped into higher order themes. Finally, higher order themes will be grouped into major categories. Qualitative software (i.e., Atlas.ti) will be used to conduct the thematic analysis.

Consensus among analysis team members conducting the analyses will be reached at each step of the analytical process (i.e., interview themes, lower order themes, higher order themes, and major categories) before proceeding to the next step to achieve inter-coder reliability. This process ensures a consistent understanding and interpretation of the data.

A17. Reasons Not to Display OMB Expiration Date

All instruments will display the expiration date for OMB approval.

A18. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are necessary for this information collection.

Attachments

Appendix A: Recruitment Materials

Appendix B: Consent Form

Instruments: Focus Group Guides

1 https://www.acf.hhs.gov/sites/default/files/documents/cb/csfr_technical_bulletin_12.pdf

2 https://www.childwelfare.gov/topics/systemwide/youth/engagingyouth/

3 Evaluation Briefs: Data Collection Methods for Program Evaluation: Focus Groups No. 13, updated August 2018 https://www.cdc.gov/healthyyouth/evaluation/pdf/brief13.pdf

4 https://www.bls.gov/oes/current/oes_stru.htm

10