Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
Youth and Parent Voices for Research
Formative Data Collections for ACF Research
0970 – 0356
Supporting Statement
Part B
June 2024
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officers:
Christine Fortunato and Laura Hoard
Office of Planning, Research, and Evaluation
Part B
The objectives of this information collection for the Preliminary Activities to Support Future Data Collection for the National Survey of Child and Adolescent Well‐Being (“Reimagining NSCAW”) are to:
Gather feedback from parents and youth with lived child welfare experience on constructs and measures proposed for a future NSCAW study design;
Gather feedback from parents and youth with lived child welfare experience on recruitment strategies and other study components proposed for a future NSCAW study design.
The purpose of this information collection is to conduct interviews and focus groups to collect feedback from participants about possible future NSCAW design components and data collections. This feedback will inform a more nuanced and comprehensive understanding of feasible and effective approaches to study design (e.g., instrumentation, administration mode) that are relevant to and resonate with future NSCAW respondents. This is intended to promote a greater level of future NSCAW respondent engagement and participation over time by increasing trust and confidence in study activities (which may then increase response rates).
These activities are intended to assess how specific constructs, measures, and recruitment, engagement, and data collection strategies are perceived and experienced by a sample of youth and families who have lived experience with the child welfare system (CWS LE). This information collection is not intended to promote statistical generalization to other youth and families or populations.
The activities described in this package were designed to inform planning and preparations for future NSCAW data collections by partnering with parents and youth who have CWS LE. The Youth and Parent Voices for Research Study activities were designed to collaborate with these individuals to explore select measures, constructs, and data collection strategies and components proposed in one or more of the design options developed under Reimagining NSCAW. These activities align with Reimagining NSCAW’s goals of providing individuals with CWS LE the opportunity for meaningful and appropriate engagement throughout the project. Information gathered through this effort, intended to serve as a proxy for the perspective and views of a likely future NSCAW respondent, will be used to inform future NSCAW data collections and is not intended to be representative of a specific group or population. Key limitations will be included in any written products associated with the Youth and Parent Voices for Research Study.
Parent participants will not be representative of all family mentors1, and they and their eligible child will not be representative of all youth and families currently involved with the CWS, and as such, results will not be generalizable. Instead, the goal is to gather feedback on proposed constructs, measures, and other study design components from individuals who closely represent a likely future NSCAW respondent. This will be accomplished using qualitative data collection methods, specifically, focus groups and semi-structured interviews.
As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.
The target population will be comprised of parents and youth with CWS LE (i.e., involvement with the CWS) whose CWS cases are currently closed. The sample of parents to be recruited currently serve as professional family mentors to other families entering the CWS. In addition, parent participants must have a child aged 11-17 living in the home. These youth will constitute the eligible sample for the youth target population. The screening questionnaire (Instrument 1) will also ask parents about how many years ago their CWS case closed; the study team will use this information to select participants with the most recent CWS experience.
Because this study is recruiting 32 youth and only 16 parents, there will ultimately be more youth than parents who participate in these activities and therefore not every youth will have a corresponding parent participate. However, there are no research questions intended for dyadic comparisons across parent and youth participants. Using the screening questionnaire (Instrument 1), the study team will determine which screened and eligible sample members to select and invite to participate based on the study’s need (i.e., if the study has met the maximum number of parent participants (n=16) but still needs youth participants, parents may be contacted about their child’s participation in the study).
To identify potential respondents, a targeted selection process will be used to recruit 16 parents who meet the eligibility (i.e., closed case with the CWS, and a child between the ages of 11-17 living at home) and inclusion (willing and able to consent to the study; able to read or speak English proficiently; able to access internet or smartphone with cellular service; and willing to participate in a focus group). This targeted selection will be accomplished using the Parent Screener (Instrument 1).
In addition to meeting those eligibility criteria, the parent screener will also ask parents about their race, ethnicity, and sexual orientation and gender identity (SOGI) [including that of their eligible child], so that recruitment could prioritize and reflect the composition of families involved with the CWS. The goal of asking about parent and youth SOGI as part of the screener is to assist with collecting data from a diverse sample of participants. Any participant's SOGI will not be disclosed in the focus groups or interviews, nor will it be included in analyses. While the reliability and validity of parent-reported information about their child’s SOGI is limited, this information collection effort is intended to explore the extent to which asking this item as part of a screener will assist with capturing a diverse sample of youth.
Parents who complete the screener, meet these criteria, and had a case closed with the CWS less than five years ago will be automatically recruited. Parents who had a case closed with the CWS more than five years ago will have their screener data reviewed by the project team prior to recruitment. This will be done to ensure that the number of years since their case was closed does not result in their child, currently aged 11-17, having been extremely young at the time of their involvement with the CWS, and as such, unable to remember. This recruitment method is used to increase the selection pool while carefully selecting parents and youth who have more recent CWS experience.
Eligible children of the parent respondents will comprise the sampling frame for the youth population, from which up to 12 youth who are 11-14 and up to 20 youth who are 15-17 years old will participate.
Instruments were developed in collaboration with a Technical Workgroup (TWG) - comprised of research experts and individuals with CWS LE - for use with parents and youth aged 11-17 who had prior experience with the CWS. The TWG recommended priority topics and interview questions related to constructs and/or measures deemed potentially problematic for a future NSCAW (e.g., outdated, not inclusive, triggering), in order to elicit reactions and gather feedback that could then be used to inform a future NSCAW design. In addition, the TWG recommended asking questions around general constructs or concepts (e.g., well-being, system experiences) to learn how these individuals with CWS LE think about and define those concepts and experiences in their own words. As a means of understanding issues related to respondent engagement and burden, the project team also developed questions focused on data collection strategies and study design implementation (e.g., mode preferences, tokens of appreciation). Draft instruments underwent several iterations of review by the project team and members of the TWG to refine instruments as a means of minimizing error. Instruments 2, 3, 4, 5, and 6 contain the following:
Instrument 2: Interview Guide for Youth Ages 11-14 – Topic Set 1
Instrument 3: Focus Group Guide for Youth Ages 15-17 – Topic Set 1
Instrument 4: Interview Guide for Youth Ages 11-14 – Topic Set 2
Instrument 5: Focus Group Guide for Youth Ages 15-17 – Topic Set 2
Instrument 6: Focus Group Guide for Parents
Participants will not be asked to read materials or prepare information prior to the focus groups or interviews.
Exhibit 1 presents each instrument along with the corresponding guiding questions and study objectives. While not all participants will be discussing, answering questions, or reacting to prompts about the same topics (i.e., Topic Set 1, Topic Set 2), each instrument will address both study objectives in Section B.1 and all three guiding questions in Section A.2.
Exhibit 1: Data Collection Instruments, Related Guiding Questions and Study Objectives
Data Collection Instruments |
Related Guiding Questions (A.2) |
Related Study Objectives (B.1) |
Instrument 2: Interview Guide for Youth Ages 11-14 – Topic Set 1 |
• How do parents and youth (ages 11-17 years old) with CWS LE understand certain constructs and react to specific measures considered for one or more NSCAW design options?
• From the perspective of parents and youth (ages 11-17) with CWS LE, what factors may support continued engagement in a longitudinal data collection effort?
• From the perspective of parents and youth (ages 11-17) with CWS LE, what factors may support participant’s authentic engagement when responding to survey items? |
Gather feedback from parents and youth with lived child welfare experience on constructs and measures proposed for a future NSCAW study design;
Gather feedback from parents and youth with lived child welfare experience on recruitment strategies and other study components proposed for a future NSCAW study design. |
Instrument 3: Focus Group Guide for Youth Ages 15-17 – Topic Set 1 |
||
Instrument 4: Interview Guide for Youth Ages 11-14 – Topic Set 2 |
||
Instrument 5: Focus Group Guide for Youth Ages 15-17 – Topic Set 2 |
||
Instrument 6: Focus Group Guide for Parents |
Youth Interviews (Instruments 2, 4) and Focus Groups (Instruments 3, 5) are divided by Topic Sets 1 and 2. Prompts and questions comprising each Topic Set were developed using various sources, an overview of which, along with sources of Instrument 6, are provided in Exhibit 2.
Exhibit 2: Sources Used to Develop Instruments
The project team will collect and audio-record all information collections through a virtual (online) meeting platform.
To recruit this sample of 16 parents and 32 youth aged 11-17, the project team will collaborate with up to three entities2, the first of which works directly with family mentor organizations:
The Kent School of Social Work, the evaluator for Kentucky (KY) and other states’ Sobriety and Recovery Teams (STARTs)
Among their staff, START organizations usually have family mentors embedded within child welfare agencies.
The Healing Place (KY), an addiction recovery center network whose nationally recognized program provides the opportunity for clients who successfully complete the program to serve as peer mentors to individuals who are newer to the program.
The Birth Parent National Network (BPNN), a network of birth parents and organizations to support families, communities, and systems.
These organizations will be sent an email and fact sheet to introduce the study goals and to explain the request being made of them (Appendix A), which is to send an e-mail to their staff of family/peer mentors introducing study activities (Appendix B). Interested family/peer mentors can then opt-in to the study via a secure website link where they will be asked to complete a screener (Instrument 1). If they are automatically determined to be eligible to participate in the study (i.e., CWS involvement less than 5 years ago), they will be formally invited to do so. Upon agreement, they will be prompted to complete the consent process for themselves, provide permission for their child(ren) to participate, and have all eligible and willing children complete the agreement to participate form to participate in a focus group or semi-structured interview (Appendix C). Should their involvement with the CWS have been more than five years ago, the project team will assess their circumstances to determine the likelihood that their eligible child would remember his or her involvement with the CWS. If so, they will contact the parent to complete the consent, permission, and youth agreement to participate process. If a parent has multiple children who are eligible, they will be given the option to allow all eligible and willing children to complete the youth agreement to participate form. This does not mean all eligible and willing children will be selected to participate, rather, it will allow the study team to determine who will be invited to participate.
Consent, permission, and youth agreement to participate forms have been scored using a readability algorithm to ensure an appropriate reading level for parents and youth.
Participants must complete a screener (Instrument 1) to determine eligibility. Participating parents will complete a 2-hour focus group; as will participating youth 15-17 years old. Participating youth 11-14 years old will complete an interview that will last up to 1 hour and 30 minutes. Facilitators and interviewers will use a semi-structured focus group/interview guide to structure this information collection (Instruments 2, 3, 4, 5, and 6).
Each focus group will be led by two co-facilitators from the project team (at least one of whom has CWS LE or extensive research experience with individuals who have CWS LE). One individual will serve as the moderator and primary facilitator. The other will be responsible for logistics (e.g., opening the session, monitoring the chat) and note taking. In addition to these focus groups, each semi-structured interview will be conducted by an individual who has expertise interviewing 11-14-year-olds. Prior to the start of data collection, the project team will train interviewers on protocols and procedures to ensure consistency across the study. In addition, all interviewers have received human-subject research training and are experienced in conducting interviews and/or focus groups, respectively. Focus groups and interviews will be audio-recorded and transcribed, as agreed upon by respondents.
The focus groups and interviews are not designed to produce statistically generalizable findings. Additionally, participation is at the respondent’s discretion. Response rates will not be calculated or reported, although information comparing the intended sample with the actual sample may be included in reports and dissemination products.
As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated. Respondent demographics will not be used in any reports or dissemination products except in aggregate form (i.e., not identifiable). However, given that activities are delineated by child age (i.e., interviews for 11–14-year-olds; focus groups for 15–17-year-olds), it may be possible to contextualize findings in reports or dissemination products by age groups.
The data will not be used to generate population estimates, either for internal use or dissemination.
Data will be collected through the Parent Screener (Instrument 1), administered through Blaise 5, an online data management system, and through audio-recordings of the focus groups and interviews, the latter of which will be sent to Rev.com for transcription. After transcripts are received from Rev.com, a quality check will be completed to remove any identifiable information and to ensure that data in the transcripts align with the audio-recording. Where transcripts are marked inaudible, the audio-recording will be reviewed and the manuscript refined. Only cleaned transcripts will be used for analysis. Audio-recordings and transcripts will be destroyed upon completion of the study.
The project team will employ rapid turnaround analysis (RTA), ideally suited for research projects from which team members must rapidly obtain insights to make timely decisions regarding strategy and practice3. The RTA process will entail several steps using de-identified interview and focus group transcripts, including the development of a template to condense and categorize the information into specific themes that align with concepts and measures being tested. Once data are categorized in the template, they will be abstracted into Excel matrices to draw conclusions and distill emergent themes. The project team will engage the TWG and individuals with CWS LE to develop coding strategies and identify emergent themes, after which study staff will use transcripts from interviews and focus groups to apply inductive thematic analyses - well-suited for exploratory work where there is little existing research - for each of the guiding questions. As part of the analytic process, emergent themes (e.g., frequently used words or phrases, thoughts, patterns) specific to each guiding question are identified, recorded, and added as a “code”. This approach, also known as reflexive analysis, is flexible and will allow for code adjustments analyses progress. The proposed analyses support the identification of the most prominent and relevant codes (i.e., emergent themes) for each guiding question, which then leads to a contextually sensitive and nuanced understanding of information collected.
Findings from these activities will be prepared for internal use by ACF as a means of informing future NSCAW data collection efforts. Any external release by ACF will be in the form of presentations, reports, briefs, and/or manuscripts and will not include any generalizable information, but will rather present derived findings, themes, and recommendations that will be beneficial to the public (researchers, federal agencies, other interest-holders). The dissemination of these findings will inform an understanding of feasible and effective approaches to study design and implementation that are relevant to and resonate with future NSCAW respondents. The project team and ACF will make it clear within these dissemination products that the information is not generalizable.
Name |
Title/Affiliation |
|
Melissa Dolan |
Project Director RTI International |
|
Dalia Khoury |
Associate Project Director, RTI International |
|
Heather Ringeisen |
Design Options Task Lead, RTI International |
|
Christine Fortunato |
Contracting Officer’s Representative, ACF |
|
Laura Hoard |
Alternate Contracting Officer’s Representative, ACF |
Instrument 1 – Parent Screener
Instrument 2 – Interview Guide for Youth Ages 11-14 – Topic Set 1
Instrument 3 – Focus Group Guide for Youth Ages 15-17 – Topic Set 1
Instrument 4 – Interview Guide for Youth Ages 11-14 – Topic Set 2
Instrument 5 – Focus Group Guide for Youth Ages 15-17 – Topic Set 2
Instrument 6 – Focus Group Guide for Parents
Appendix A: Outreach to Family Mentor Organization(s)
Appendix B: Outreach Materials for Family Mentors
Appendix C: Consent, Permission, Agreement to Participate Forms
Clark, C.M., Kosciw, J.G. (2022) Considerations for Measuring Sexual Orientation and Gender Identity in Surveys of Secondary School Students (Research Brief). New York: GLSEN. https://www.glsen.org/sites/default/files/2022-05/GLSEN_LGBTQ_Data_Inclusion_Research_Brief.pdf
Connor, K. M., & Davidson, J. R. (2003). Development of a new resilience scale: The Connor‐Davidson resilience scale (CD‐RISC). Depression and anxiety, 18(2), 76-82.
Ferguson, K. M., & Xie, B. (2012). Adult support and substance use among homeless youths who attend high school. Child & Youth Care Forum (Vol. 41, pp. 427-445). Springer, U.S.
Hamilton, A. (2020). Rapid turnaround qualitative research. 17th Annual Research Talk, Qualitative Research Summer Intensive. Research Talks: Cary, North Carolina.
Kiplinger, V. L., & Browne, C. H. (2014). Parents’ assessment of protective factors: User’s guide and technical report. Washington, DC: Center for the Study of Social Policy.
Kosciw, J. G., Clark, C. M., & Menard, L. (2022). The 2021 National School Climate Survey: The experiences of LGBTQ+ youth in our nation’s schools. New York: GLSEN.
Office of Management and Budget (OMB) (a): Recommendations on the best practices for the collection of sexual orientation and gender identity data on federal statistical surveys, Information Collection Request, 2024.
Office of Management and Budget (OMB) (2024b): Revisions to OMB's Statistical Policy Directive No. 15: Standards for Maintaining, Collecting, and Presenting Federal Data on Race and Ethnicity, 89 FR 22182 (Mar 28, 2024), Available at: https://www.federalregister.gov/documents/2024/03/29/2024-06469/revisions-to-ombs-statistical-policy-directive-no-15-standards-for-maintaining-collecting-and
Phinney, J. (1992). The Multigroup Ethnic Identity Measure: A new scale for use with adolescents and young adults from diverse groups. Journal of Adolescent Research, 7, 156-176.
Phinney, J. S., & Ong, A. D. (2007). Conceptualization and measurement of ethnic identity: Current status and future directions. Journal of counseling Psychology, 54(3), 271.
Selwyn, J., Briheim-Crookall , L., & Bright Spots Team. (2022). 10,000 voices. Te Rees Centre, Department of Education, University of Oxford. https://coramvoice.org.uk/wp-content/uploads/2022/11/2410-CV-YLYC-2021-Report-new3-lo-10.10.10.pdf
Slaten, C., Rose, C., Bonifay, W., & Ferguson, J. (2018). Milwaukee youth belongingness scale: item response theory analysis. School Psychology Quarterly.
Titus, J., Feeney, T., Smith, D., Rivers, T., Kelly, L., & Dennis, M. (2013). GAIN-Q3 3.2: Administration, clinical interpretation, and brief intervention. GAIN Coordinating Center, Chestnut Health Systems. https://gaincc.org/instruments/
1 Parents will be recruited from family mentor organizations, therefore the term “family mentor” is associated with the parents recruited for this sample.
2 Should outreach to the START Teams result in a sufficient sample, the Healing Place and BPNN will not be contacted.
3 Hamilton, A. (2020). Rapid turnaround qualitative research. 17th Annual Research Talk, Qualitative Research Summer Intensive. Research Talks: Cary, North Carolina.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Khoury, Dalia |
| File Modified | 0000-00-00 |
| File Created | 2025-05-29 |