SSB Measuring Success rev

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

GenIC: Measuring, Supporting, and Understanding Child and Caregiver well-being through Employment and Self-Sufficiency Research

Under

Umbrella Generic Clearance:

Formative Data Collections for ACF Research

0970 - 0356

Supporting Statement

Part B

March 2023

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers: Erin Cannon, Emily Ross, and Elizabeth Karberg

Part B

B1. Objectives

Study Objectives

The Measuring, Supporting, and Understanding Child and Caregiver well-being through Employment and Self-Sufficiency research (Measuring SUCCESS) project will lay the foundation for integrating a child and family social well-being lens into the U.S. Department of Health and Human Services, Administration for Children and Families (ACF) Office of Planning, Research, and Evaluation’s Welfare and Family Self-Sufficiency Research Portfolio.¹ Specifically, ACF is interested in understanding what aspects of child and family social well-being should be measured in research about programs that aim to increase the economic security of families with low incomes. Through this Generic Information Collection (GenIC) the proposed parent and caregiver focus groups are one of the activities to contribute to the Measuring SUCCESS project. As referenced in Supporting Statement A (A1), other activities include: (1) cataloging and assessing the state of existing child-, parent-, and family-related measurement in OPRE’s Welfare and Family Self-Sufficiency Research Portfolio, (2) reviewing existing literature on whether and how self-sufficiency programs influence well-being, (3) reviewing models of how self-sufficiency programs intend to influence well-being, (4) developing a conceptual framework for understanding the social well-being of children, parents, and families in the context of welfare and family self-sufficiency research, and (5) developing a compendium of relevant measures.

Information collected through the parent and caregiver with lived experience in employment and training programs focus groups are intended to understand how they define well-being in their own words. Specifically, to the goal is to address the following research questions:

1. What do a group of parents and caregivers with lived experience in employment and training programs consider as important for their general social well-being?

2. What do a group of parents and caregivers with lived experience in employment and training programs think shapes their social well-being?

3. From the perspective of a group of parents and caregivers with lived experience in employment and training programs, what are the experiences in TANF programs? What are the structural inequities that affect these parents’ and caregivers’ experiences in the types of programs that studied within OPRE’s Welfare and Family Self-Sufficiency Research Portfolio?

Generalizability of Results

This project is intended to present an internally valid description of a group of parents and caregivers with lived experience in employment and training programs. Although these data are not intended to be representative of the larger population of participants, it is critical to understand the perspectives of the people who are the intended recipients of employment and training services. Through the GenIC, the focus groups are one component of, but not alone to contribute to, the work to develop the conceptual model (referenced in A1) with a small sample of parents and caregivers to ensure their knowledge and perspectives of the most important aspects of child and family well-being are captured in the model.

Appropriateness of Study Design and Methods for Planned Uses

Hearing from individuals with lived experience, in this case parents and caregivers, can challenge deep-rooted assumptions about people participating in programs and help the research team better understand social well-being (Coleman 2021). Through focus groups with parents and caregivers, the research team will be able to talk directly to those whose experiences the programs aim to improve. The conceptual framework and subsequent activities will be informed by the voices of those who have lived experience with programs that aim to increase economic security.

The research team will not use the data to make statistical inferences or to produce generalizable findings. The focus groups are not intended to be a representative sample of all parents and caregivers participating in programs aimed at increasing economic security. The project is also not an impact study and cannot be used to assess an individual’s outcomes. These limitations will be included in the written product associated with this study.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.

B2. Methods and Design

Target Population

For each focus group, the research team (Mathematica) will collect information from parents and caregivers with lived experience in employment and training programs. For the purposes of the focus groups, the research team is defining parents and caregivers as adults who bring up and are responsible for a child. The research team recognizes that families are complex and not all caregivers are biological parents. The research team will use non-probability, purposive sampling to identify potential respondents who can provide information on the project’s research questions. Because participants will be purposively selected, they will not be representative of parents or caregivers with lived experience in employment and training programs. Instead, we aim to obtain variation in parents’ and caregivers’ experiences to understand their perceptions of social well-being within self-sufficiency program contexts.

Respondent Recruitment

To identify the specific sample for this study, the research team will partner with Temporary Assistance for Needy Families (TANF) programs and other organizations that provide ACF programs aimed at increasing economic security. The research team aims to obtain variation in parents’ and caregivers’ interpretation or perceptions of family social well-being by including parents with different background characteristics (e.g., fathers, or Tribal parents and caregivers). The research team will ask the TANF programs and other organizations that provide ACF programs to recommend parents that possess a certain characteristic, for example, parents of children in middle school.² Once the programs have identified potential parents or caregivers for the focus groups, the research team will contact them via email and phone with more information about the project and ask them to participate in the focus group (Appendices B, C, D, and E). The research team anticipates that some potential participants who are introduced to the study by the TANF program may prefer to reach out to the study team directly. The program will provide potential participants with an email or phone number for a study team member. These participants will receive a different outreach email from those whose contact information is provided by the TANF agency (Appendix G).

We will create focus groups so each focus group will include parents and caregivers with a shared pre-identified characteristic, for example parents or caregivers with children who are in middle school. By creating groups based on a characteristic, the research team will identify whether for the parents in the focus group, their definitions and understanding of social well-being is similar or different.

B3. Design of Data Collection Instruments

Development of Data Collection Instrument

The research team developed the data collection instrument to meet the objectives of the project and answer the research questions. The focus group discussion guide (Instrument 1) is designed to minimize burden on respondents by only asking questions that are relevant to the parents in the group. The research team will tailor the questions based on the participants in the group. The focus group discussion guide includes a participant information form that will collect information on the gender identity, age, ethnicity, race, number of children and age of children of the focus group participants. We included gender identity because our technical working group members stated that it will be helpful to obtain the perspective of fathers, who are underrepresented in research about parents and caregivers, as part of the focus groups.

The research team, ACF, and a technical working group member reviewed the data collection instruments for length, content, and flow.

B4. Collection of Data and Quality Control

The research team will conduct all data collection. The research team will ask up to nine local TANF or other employment training program administrators to identify potential parents or caregivers for the focus groups. The TANF/employmentprogram administrators will use a Box site to send a list with names, phone numbers, and email addresses to the research team of parents or caregivers that expressed interest in participating in the focus groups. If potential participants do not want their data shared, they will be able to reach out to the project team directly by email or phone. The research team will then send an email invite to the parents and caregivers that expressed interest. The email will include information about the date of the focus group and location (either on Zoom or in-person).

Prior to each focus group, the research team will meet to discuss the focus group discussion guide and lessons from the previous groups. The focus group facilitator will receive training on the discussion guide and goals of the focus group prior to the focus group. The facilitator will use the discussion guide to ensure that information is collected consistently across the groups. Three members of the research team will attend each focus group and meet to debrief after the focus group. During this debrief, the research team will discuss themes and how they interpreted responses to the discussion guide questions.

The research team has outlined focus group rules and procedures for consent in the focus group discussion guide. All virtual focus group participants will provide verbal consent to participate in the discussion and in-person participants will provide written consent (Appendix A). With the permission of the respondents, the research team will record the interviews to assist with notetaking. If a member of the focus group does not consent to the recording, the research team has a designated notetaker to capture information during the discussion.

The transcriptions of the focus groups will be uploaded to Box for the translation service vendor and will then be saved by the research team from Box to the restricted network once translation services are complete. The research team will delete files from Box once the focus groups are complete. The notes will be compared to the audio recordings to ensure data quality.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

The focus groups are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. Response rates will not be calculated or reported.

The burden estimates account for the possibility that the maximum number of focus group members invited actually attend the focus group. That is, the burden estimates include eight participants for each group so that the research team will not have to exclude respondents if all invited parents and caregivers attend.

Non Response

As participants will not be randomly sampled and findings are not intended to be representative, non-response bias will not be calculated. Respondent demographics will be documented and reported in written materials associated with the data collection. The research team will seek focus group participants who are interested in participating. The research team will plan to recruit up to eight individuals per group. As noted above burden estimates account for the possibility that all eight invitees attend.

B6. Production of Estimates and Projections

The data will not be used to generate population estimates, either for internal use or dissemination.

B7. Data Handling and Analysis

Data Handling

The research team will monitor all data collection. The research team will take notes during the focus groups and record the conversations if participants consent to the recording. The recording will be transcribed and compared to the notes taken during the focus groups. The recordings will be housed on a secure network as described in Supporting Statement A, Section A10.

Data Analysis

The research team will analyze the focus group transcripts for themes that emerge from the discussion based on the questions asked across the focus groups. The focus group discussions will elicit perceptions and experiences of parents and caregivers about their well-being while participating in programs aimed at increasing their economic security. The focus group discussion guide will be used to develop an initial codebook of anticipated themes. The codebook will be iteratively updated and refined as data analysis progresses to ensure codes capture themes that emerge within the groups. The research team may add themes based on the team debriefs after each focus group. The research team will them analyze the focus group transcripts using these themes in Excel. The research team will analyze the responses within and across groups to identify any similarities or differences to the questions based on parent and caregiver characteristics.

Data Use

The data will be used to contribute to the development of a conceptual framework that will use information from multiple information gathering activities (a literature and conceptual model scan and synthesis; input from government, academic, and other experts with lived experience), including focus groups with parents and caregivers with lived experience in employment and training programs. . The conceptual framework will be made public. The data will also be used to produce a brief for the parents and caregivers that participated in the focus groups to share back the themes the research team heard from them on social well-being and inequities they encountered while participating in the programs. Limitations to the findings, such as generalizability, will be noted in materials, as appropriate.

B8. Contact Persons

The following individuals from the contractor will be involved in the design, data collection, and analysis of this project:

Diana McCallum, Project Director, Mathematica, [email protected]

Elizabeth Brown, Task Leader, Mathematica, [email protected]

Erin Cannon, Project Officer, OPRE, [email protected]

Elizabeth Karberg, Project Officer, OPRE, [email protected]

Attachments

Instrument 1: Focus group discussion guide

Appendix A: Focus group consent form

Appendix B: Outreach email to focus group participants

Appendix C: Outreach email 2 to focus group participants

Appendix D: Outreach email confirmation to focus group participants

Appendix E: Reminder email to focus group participants

Appendix F: Focus group FAQs

Appendix G: Outreach email to potential participants who contact the study

References

Coleman, Amanda. “Reimagining Our Approach to Research to Advance Racial Equity - Centering the Perspectives of Program Participants and Other Stakeholders.”[Conference presentation]. 2021 OPRE Methods Meeting on Enhancing Rigor, Relevance, and Equity in Research and Evaluation through Community Engagement. Washing­ton, DC: Office of Planning, Research, and Evaluation, 2021.

1 The project’s working definition of family social well-being considers whether all members of the family: (1) are physically and mentally healthy; (2) are safe and supported in the community; (3) are developing cognitively and socially; (4) have a sense of belonging, inclusion, and respect based on their identity as an individual and role in the family; and (5) are interacting within their family in a healthy and supportive way.

2 We will reach out to up to 9 program administrators, so this effort is not subject to PRA.

7