SSB - Engaging Young People With Lived Experience in the CFSR Process

Engaging Young People With Lived Experience in the Child and Family Services Review (CFSR) Process

Formative Data Collections for Program Support

0970 - 0531

Supporting Statement

Part B

November 2021

Submitted By:

Children’s Bureau

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Danielle McConaga, Child and Family Program Specialist

B1. Objectives

The Administration for Children and Families (ACF) at the U.S. Department of Health and Human Services (HHS) seeks approval for collecting information about the most effective strategies to engage young people who have lived experience in the child welfare system in the Child and Family Services Review (CFSR) process. The CFSR process is designed to meet the statutory requirement to provide federal oversight of states’ compliance with title IV-B and IV-E plan requirements, strengthen state child welfare programs, and improve safety, permanency, and well-being outcomes for children and families served.

Study Objectives

The purpose of the focus groups is to collect information from 20 young persons with lived experience on the best methods of recruiting, engaging, supporting, and retaining these young people in all aspects of the CFSRs. The information collected will:

• Enable the CB and states to obtain more comprehensive information on how state child welfare systems are functioning, including identifying strengths and challenges;

• Enable the CB and states to better understand the data and information collected during the statewide assessment and onsite review phases of the CFSR and, as a result, be better positioned to develop promising PIP strategies to address areas of nonconformity;

• Better support states in implementation of PIP strategies, including making adjustments as needed to ensure better outcomes; and

• Enhance the CB’s ability to monitor PIP implementation by raising awareness of the perspective of those with lived experience.

The information collected will be used internally for planning, program support, and technical assistance. The CB will also share lessons learned with the states to build their capacity (i.e., themes from the results, no individual nor aggregate data).

Study objectives, raised as questions, include asking young people:

• What are some effective strategies for the recruitment and engagement of young people with lived experience?

• What are the potential roles or avenues for involvement within the CFSR process that young people with lived experience identify for themselves?

• What training, resources, and supports do young people with lived experience need to be successful and effective within their role(s)?

• What training, resources, and supports do those engaged in the CFSR process (e.g., child welfare agencies, the legal and judicial communities, child welfare professionals, federal staff) need to work effectively with young people with lived experience?

Generalizability of Results

This study is intended to present an internally-valid description of 20 young peoples’ perceptions, not to promote statistical generalization to other populations. The data are not intended to be representative. Although the data will not be generalizable/representative, the results will have utility for the government because they will provide rich, detailed descriptions from young people about how they would like to be engaged in the CFSRs in their own words.

Appropriateness of Study Design and Methods for Planned Uses

To meet the study objectives, the CB proposes to conduct focus groups with young people who have lived experience in the child welfare system. This data collection method was chosen to solicit the opinions of participants. This highly specific information about young people with lived experience and the CFSRs is not available elsewhere. The data are not intended to be representative. Themes from the focus groups will be used internally for planning, program support, and technical assistance. Key limitations will be included in written products associated with the study.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.  

B2. Methods and Design

Target Population

We will collect information from 20 young people with lived experience. There will be 2 sets of 4 focus groups, with no more than 10 participants in each set; the participants in each set will share similar backgrounds/experiences to each other. The sampling frame for participants from set 1 will be the roster of young adult consultants that support the National Youth in Transition Database (NYTD) Reviews and the Capacity Building Center for States. The sampling frame for participants from set 2 will be young people invited and selected from national organizations. These organizations include Congressional Coalition on Adoption- Foster Youth Internship Program, Foster Care Alumni of America (FCAA), Anne E. Casey Foundation/Jim Casey Youth Opportunity Initiatives & Young Fellows, FosterClub, National Foster Care Youth & Alumni Policy Council (Foster Club, FCAA, and Private Foundations), and Think Of Us. The research team will use non-probability, convenience sampling to identify potential respondents who can provide information on the study’s key constructs. Because participants will be selected on a first come, first serve basis, they will not be representative of the population of young adults with lived experience. Instead, we aim to obtain variation in young peoples’ experiences to understand how to engage them in the CFSRs.

Respondent Invitation and Selection

20 respondents, 10 from each set, each will be invited and selected to participate. We will provide an honorarium ($250.00 each) to respondents for time spent sharing their expertise on the matter at hand. Everyone who we are inviting to participate already meets the recruitment criteria (i.e., being a young person with lived child welfare experience), but we know that not everyone will be available, hence our need to invite a larger number than we will select during the sessions. Please refer to the previous section (target population) for additional details.

B3. Design of Data Collection Instruments

Development of Data Collection Instruments

CB has contracted with JBS International to design the data collection instruments and to conduct this information collection. JBS International staff members of the Child Welfare Reviews Project (CWRP), including consultants who are young adults with lived experience, developed four instruments, one for each focus group. Each focus group and instrument corresponds to a different component of the CFSR, which streamlines the data collection instruments to ask only questions necessary to achieve the objectives of the data collection. Further, data collection and analysis for each instrument is designed to address all objectives of the study.

• Focus group 1 and instrument 1: Statewide Assessment and statewide data indicators

• Focus group 2 and instrument 2: Onsite Review

• Focus group 3 and instrument 3: Program Improvement Plan development and monitoring

• Focus group 4 and instrument 4: Wrap-Up and Debrief

After JBS International Inc developed the instruments, they were reviewed and approved by a team of CB leadership staff.

B4. Collection of Data and Quality Control

A recruitment message in the form of an email (sent by JBS International) describing the series of focus groups will be sent to the sampling frames and we will use convenience sampling to invite and select respondents on a first come, first serve basis. Respondents will be asked to attend 4 focus groups:

• Focus group 1: how to engage young people in the statewide assessment and the statewide data indicators

• Focus group 2: How to engage young people in the onsite review

• Focus group 3: How to engage young people in the PIP process

• Focus group 4: Wrap-up and Debrief

Once an individual confirms they would like to participate, they will be asked to complete a consent form that details the activities and tasks of the study and provides information for the distribution of the honorarium.

Data will be collected through a series of four focus groups. Different components of the CFSRS will correspond with topics covered in the first three focus groups, and the fourth focus group (the wrap-up and debrief) is designed to ensure quality of information, by giving participants another opportunity to provide clarification and additional information about previous responses. To ensure quality and consistency in the data collection, focus group facilitator guides will be distributed to staff conducting the focus groups. The focus groups will be recorded (audio and video) and two notetakers will be present at each session. To ensure quality information is provided by respondents in the focus groups, we will conduct preparation sessions with the respondents to explain the different components of the CFSRs. We will gather informal feedback (via a conversation) at the end of each preparation session to understand if we are being effective and to inform subsequent preparation sessions. Participation in all of the activities is voluntary, and participants can choose not to answer questions during the focus group without any consequences (i.e, their honorarium will not be affected).

B5. Response Rates and Potential Nonresponse Bias

Response Rates

The focus groups are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. Response rates will not be calculated or reported.

NonResponse

As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated.

B6. Production of Estimates and Projections

We do not plan to make policy decisions off of data that is not representative. The data will not be used to generate population estimates, either for internal use or dissemination.

B7. Data Handling and Analysis

Data Handling

The focus groups will be recorded and transcribed. Staff will also take notes during the focus groups to summarize verbal and non-verbal communication (which can add important context to how someone responded to a question/prompt). To minimize any errors, as the first step in the data-cleaning process, recordings of the focus groups will be transcribed and cleaned to remove any respondent-identifying information and any transcription mistakes. These transcriptions, along with the notes from each focus group, will serve as the qualitative data used for the study.

Data Analysis

We will use Atlas.ti software to analyze the qualitative data. The initial step in the inductive analysis process will be reading the cleaned and coded interview transcripts and notes to discover underlying themes. The themes will then be grouped into lower order themes based on common topics. Next, following the same coding procedures for grouping the themes, lower order themes will be grouped into higher order themes. Finally, higher order themes will be grouped into major categories.

Consensus among analysis team members conducting the analyses will be reached at each step of the analytical process (i.e., initial themes, lower order themes, higher order themes, and major categories) before proceeding to the next step to achieve inter-coder reliability. This process ensures a consistent understanding and interpretation of the data. We will also share the results with our consultants (young people with lived experience/coaches) and the focus group participants, which will allow them to share their informal feedback on whether the themes generally summarize what was said during the focus groups.

Data Use

The themes will be compiled into a memo of results. The memo will be used internally for planning, program support, and technical assistance. The CB will also share lessons learned with the states to build their capacity (i.e., themes from the results, not the full focus group transcripts, in the form of a report). Limitations to the data will be noted in any shared materials.

B8. Contact Persons

Hung Q. Pho

Project Director, Child Welfare Reviews Project

JBS INTERNATIONAL, INC.

[email protected]

Attachments

Appendix A: Recruitment Materials

Appendix B: Consent Form

Instruments: Focus Group Guides

7