Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
ACF Behavioral Interventions to Advance Self-Sufficiency Next Generation (BIAS-NG) Project
Generic Information Collection for Qualitative and Descriptive Quantitative Data Collection for Hennepin County Children and Family Services
OMB Information Collection Request
0970 - 0502
Supporting Statement
Part B
March 2022
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officers:
Kim Clum
Victoria Berk Kabak
Part B
B1. Objectives
Study Objectives
Behavioral Interventions to Advance Self-Sufficiency-Next Generation (BIAS-NG) consists of a series of mixed-methods studies to identify, develop, and test interventions related to social services and benefit receipt. This Generic Information Collection (GenIC) request pertains to diagnosis research in one site in the program area of Child Welfare (CW): Hennepin County Children and Family Services (CFS). This IC will conduct a diagnosis study, as indicated in the overarching generic clearance for the BIAS-NG project as Phase 3 is the Diagnosis and Design phase. This diagnosis study will conduct qualitative and descriptive quantitative research to identify and understand the psychological and behavioral factors that can affect the effectiveness of human service programs. The diagnosis research component (this GenIC) provides critical insights to designing an effective intervention, allowing the research team to properly diagnose ways in which agencies are not maximizing their impact for the populations they serve.
Generalizability of Results
This diagnosis study IC is intended to present an internally valid description of family and staff experiences and perspectives in chosen sites for the purpose of behavioral diagnosis, not to promote statistical inference or generalization to other sites or service populations. This IC will contribute to understanding and highlighting lived experiences of parents and on-the-ground experiences of frontline staff.
Appropriateness of Study Design and Methods for Planned Uses
This diagnosis study involves formal interviews and focus groups with CFS-involved families and agency staff (caseworkers, supervisors, administrators, and Family Group Decision Making meeting facilitators who run the county’s formal Family Group Decision Making (FGDM) meetings with participants) from across a representative sample of the agency. These methods will allow the research team to ask questions about family and staff understanding of the current processes plus their perspectives on barriers and facilitators related to those processes, questions that cannot be answered through analysis of administrative data alone.
As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.
B2. Methods and Design
Target Population
The target population to be included in this GenIC include:
Individuals who were or are involved in CFS. We estimate this group to be about 5,500 parents per year.
Staff at the agency including caseworkers and facilitators, supervisors, and administrators.
Sampling
The target respondents to be included in this GenIC include:
Up to 50 parents/guardians with children currently or previously involved with CFS. Depending on the parents’ availability, some parents may be interviewed individually and some may participate in focus groups. These parents may be drawn from community-based organizations (CBOs) where people who have or had personal involvement with Hennepin County CFS work or volunteer, or from parents who are currently involved with CFS that staff identify as being part of a stable case (for example, participating in case management rather than families who are currently being investigated).
20 supervisors and administrators from a sample of all investigations supervisors and administrators across the agency from about 63 total staff at this level. Depending on schedules and availability, some staff may be interviewed individually and some may participate in focus groups.
40 caseworkers and facilitators from all investigations caseworkers and FGDM facilitators, drawn from approximately 100 total staff in these roles. We will aim to gather informants who reflect a mix of case experiences—for example, staff who have been with the program for many years and new staff, and staff who belong to different supervisory units. Depending on schedules and availability, some staff may be interviewed individually and some may participate in focus groups.
40 CBO staff working or volunteering in organizations in Hennepin County. We will identify organizations that serve people who may also be involved in the Children and Family Services system. The CBO staff interviews will provide additional context for public perceptions of CFS practices and for families’ experience of interacting with CFS. Because CBO staff interact with many families, they may be able to share insights on less common case types or experiences that may not be covered in the family interviews. We will identify staff from up to 10 CBOs.
B3. Design of Data Collection Instruments
Development of Data Collection Instruments
Formative focus groups and interviews with fewer than 10 people informed the development of interview/focus group protocols for data collection. The data collection instruments will not be pre-tested at scale but are very similar to other GenICs under the BIAS-NG umbrella generic that have been used successfully to address the five central research questions:
1. What are family perspectives on and experiences with the CW agency and process, including FGDM?
2. What are family perspectives on challenges to engagement with the agency and related service providers?
3. How do site operations work, especially related to family engagement and FGDM?
4. What are staff perspectives on their work and successful family engagement?
5. What are staff perspectives on challenges to family engagement, including FGDM?
B4. Collection of Data and Quality Control
To collect qualitative data, study teams composed of approximately two members will conduct in-person focus groups and interviews with program staff and participants. If this is not possible, the team will conduct interviews virtually. Research staff members are experienced in the process of diagnosis research. Each interview will begin with an introduction that explains the purpose and goals of the BIAS-NG project. The facilitator will inform staff or participants that the conversation will be audio-recorded but that we will stop the recording at any time they do not feel comfortable being recorded. For individual phone or video interviews, the facilitator will begin the conversation with the staff or participant after receiving verbal consent. For in-person interviews, the facilitator will begin after receiving the signed consent form.
At the end of the participant interviews, participants will receive a gift card worth $25. The instruments guiding these participant interviews are provided in Instruments 1 and 2.
To recruit family interviews and focus groups, we will work with local CBOs where families formerly involved in the child welfare system work or volunteer (e.g., in peer mentoring programs). Interviewers will also work with staff from the agency to identify potential parents to interview. The research team will interview these participants in person at a convenient location or via video conference or phone if preferred.
To recruit CFS agency staff for interviews and focus groups, we will similarly work with agency leadership to identify staff members in different supervisory units within the relevant organization divisions.
To recruit CBO staff, we will reach out to local organizations that serve a population with clients who may also be involved in the Children and Family Services system.
Instrument 1 details the family interview guide and Instrument 2 details the staff interview guide.
B5. Response Rates and Potential Nonresponse Bias
Response Rates
The data collection activities in this IC request are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. As respondents are not randomly sampled, we will not calculate a response rate.
For qualitative data collection, the research team will work closely with community-based organizations and/or administrators and staff to recruit families and will work closely with administrators and staff to identify staff for us to interview. Participants in the interviews and focus groups will be identified through a convenience sample identified by agency administrators and staff and/or community-based organization leadership. To further increase the likelihood of participation, we will offer focus group and interview participants and staff gift cards, as discussed in Supporting Statement Part A.
Non-Response
As participants will not be randomly sampled and findings are not intended to be representative, interview and focus group non-response bias will not be calculated.
B6. Production of Estimates and Projections
Findings will only present an internally valid description of the program process from both the administrative and client perspectives in chosen sites. Results will be included in the final public report. The data collected from this IC will not be used to make policy decisions.
B7. Data Handling and Analysis
Data Handling
With study participant permission, we will audio record conversations and use these records to minimize errors due to coding and data processing.
Data Analysis
Interviewers will take detailed notes during the focus groups or interviews. Interviewers will write summary notes after each interview to record early impressions and emerging patterns. These notes will not be considered project data to be analyzed, but they will serve as a record of initial (emerging) themes that will then be followed up on by a more thorough analysis to better understand the program processes and staff and client perceptions. The analysis will focus on answering our five primary diagnosis research questions, listed in section B3.
Data Use
The goal of the BIAS-NG Generic Clearance is to conduct qualitative and descriptive quantitative research to identify and understand the psychological and behavioral factors that can affect the effectiveness of human service programs. The diagnosis research will allow the team to gather structured in-depth information to understand the program process from both the administrative staff and client perspectives. Focus groups and interviews are essential to identifying the points in the outreach and delivery of services, or in the client’s experiences, that are most amenable to a behavioral intervention.
Aside from summarizing findings in public reports as part of the project’s dissemination activities, no other documentation will be released regarding this data collection. Our dissemination goals prioritize sharing findings with practitioners, decisionmakers, and academic scholars at the federal, state, and local levels. We will document the limitations of the resulting data when making anything public.
B8. Contact Persons
Clint Key, MDRC, Project Director and Site Lead – [email protected]
Kim Clum
Senior Social Science Research Analyst
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
330 C St. SW, 4th Floor
Washington, DC 20201
(202) 205-4922
Victoria Berk Kabak
Senior Social Science Research Analyst
Office of Planning, Research & Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
330 C St. SW, 4th Floor
Washington, DC 20201
(202) 401-5871
Attachments
Instrument 1 – Hennepin County Children and Family Services Family Interview and Focus Group Protocol
Instrument 2 – Hennepin County Children and Family Services Staff and Community-Based Organization Staff Interview and Focus Group Protocol
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Liza Rodler |
| File Modified | 0000-00-00 |
| File Created | 2023-08-27 |