SSB - BIAS-NG GenIC: Matrix/Starfish (EHS/HS)

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

ACF Behavioral Interventions to Advance Self-Sufficiency Next Generation (BIAS-NG) Project

Generic Information Collection for Qualitative and Descriptive Quantitative Data Collection for Two Early Head Start/Head Start Sites

OMB Information Collection Request

0970 - 0502

Supporting Statement

Part B

December 2021

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Kim Clum

Victoria Berk Kabak

Part B

B1. Objectives

Study Objectives

Behavioral Interventions to Advance Self-Sufficiency-Next Generation (BIAS-NG) consists of a series of mixed-methods studies to identify, develop, and test interventions related to social services and benefit receipt. This Generic Information Collection (GenIC) request pertains to diagnosis research in two sites in the program area of Early Head Start/Head Start (EHS/HS): Matrix Human Services and Starfish Family Services. This IC will conduct a diagnosis study, as indicated in the overarching generic clearance for the BIAS-NG project as Phase 3 is the Diagnosis and Design phase. This diagnosis study will conduct qualitative and descriptive quantitative research to identify and understand the psychological and behavioral factors that can affect the effectiveness of human service programs. The diagnosis research component (this GenIC) provides critical insights to designing an effective intervention, allowing the research team to properly diagnose ways in which agencies are not maximizing their impact for the populations they serve.

Generalizability of Results

This diagnosis study IC is intended to present an internally valid description of parent and staff experiences and perspectives in chosen sites for the purpose of behavioral diagnosis, not to promote statistical generalization to other sites or service populations. This IC will contribute to understanding and highlighting lived experiences of parents and on-the-ground experiences of frontline staff.

Appropriateness of Study Design and Methods for Planned Uses

This diagnosis study involves formal interviews and focus groups with EHS/HS parents and EHS/HS staff (teachers, family service guides, and center leaders) from across a representative sample of Starfish’s 13 centers and Matrix’s 16 centers. These methods will allow the research team to ask questions about parent and staff understanding of the current processes plus their perspectives on barriers and facilitators related to those processes, questions that cannot be answered through analysis of administrative data alone.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.

B2. Methods and Design

Target Population

The target population to be included in this GenIC include:

• Customers or individuals receiving services from EHS /HS programs, specifically any parent with children enrolled in Matrix or Starfish. We estimate this group to be about 4,000 parents per year.

• Staff at the EHS/HS centers including Matrix and Starfish center leaders, family service guides, and teachers. There are 30 centers across these sites- each with a center leader, one or two family service guides, and a handful of teachers.

Sampling

The target respondents to be included in this GenIC include:

• Up to 50 parents with children in EHS/HS, across multiple Matrix and Starfish centers. Depending on the parents’ availability, some parents may be interviewed individually and some may participate in focus groups. These parents will be drawn from a sample of up to 10 Matrix EHS/HS centers and up to 10 Starfish EHS/HS centers. The targeted sample reflects variation based primarily on center size (for example, larger centers, medium-sized centers and smaller centers) and demographics (specifically, racial and ethnic background of families).

• One center leader from a sample of up to 10 Matrix EHS/HS centers and up to 10 Starfish EHS/HS centers. The targeted sample reflects variation based primarily on center size (for example, larger centers, medium-sized centers and smaller centers) and demographics (specifically, racial and ethnic background of families). Depending on schedules and availability, some staff may be interviewed individually and some may participate in focus groups.

• One or two family service guides from up to 10 Matrix EHS/HS centers and up to 10 Starfish EHS/HS centers (n = 20-40). The targeted sample reflects variation based primarily on center size (for example, larger centers, medium-sized centers and smaller centers) and demographics (specifically, racial and ethnic background of families). Additionally, we will aim to gather informants who reflect a mix of case experiences--for example, staff who have been with the program for many years and new staff. Depending on schedules and availability, some staff may be interviewed individually and some may participate in focus groups.

• One or two teachers from up to 10 Matrix EHS/HS centers and up to 10 Starfish EHS/HS centers (n = 20-40). The targeted sample reflects variation based primarily on center size (for example, larger centers, medium-sized centers and smaller centers) and demographics (specifically, racial and ethnic background of families). Additionally, we will aim to gather informants who reflect a mix of case experiences--for example, staff who have been with the program for many years and new staff. Depending on schedules and availability, some staff may be interviewed individually and some may participate in focus groups.

B3. Design of Data Collection Instruments

Development of Data Collection Instruments

Formative focus groups and interviews with fewer than 10 people informed the development of interview/focus group protocols for data collection. The data collection instruments will not be pre-tested at scale but are very similar to other GenICs under the BIAS-NG umbrella generic that have been used successfully to address the five central research questions outlined in Supporting Statement A.

Research Questions:

1. What are parent perspectives on and experiences with the EHS/HS program?

2. What are parent perspectives on challenges to program engagement and child attendance?

3. How do site operations work, especially related to parent engagement and child attendance?

4. What are staff perspectives on their work, successful parent engagement, and child attendance?

5. What are staff perspectives on challenges to parent engagement and child attendance?

B4. Collection of Data and Quality Control

To collect qualitative data, study teams composed of approximately two members will conduct in-person focus groups and interviews with program staff and participants. If this is not possible, the team will conduct interviews virtually. Research staff members are experienced in the process of diagnosis research. Each interview will begin with an introduction that explains the purpose and goals of the BIAS-NG project. The facilitator will inform staff or participants that the conversation will be audio-recorded but that we will stop the recording at any time they do not feel comfortable being recorded. For individual phone or video interviews, the facilitator will begin the conversation with the staff or participant after receiving verbal consent. For in-person interviews, the facilitator will begin after receiving the signed consent form.

At the end of the participant interviews, participants will receive a gift card worth $25. The instruments guiding these participant interviews are provided in Instruments 1 and 2.

To recruit EHS/HS parent interviews and focus groups, we will use EHS/HS data to select up to 10 EHS/HS centers from the range of EHS/HS centers across Matrix and Starfish. Interviewers will work with staff from up to 20 of those centers to identify potential parents to interview. The research team will interview these participants in person at a convenient location or via video conference or phone if preferred.

To recruit Matrix and Starfish EHS/HS staff for interviews and focus groups, we will similarly work with Matrix and Starfish leadership to identify up to 10 centers from each organization to identify each center’s leader, 1-2 family service guides, and 1-2 teachers. Instrument 2 details the staff interview guide.

B5. Response Rates and Potential Nonresponse Bias

Response Rates

The data collection activities in this IC request are not designed to produce statistically generalizable findings and participation is wholly at the respondent’s discretion. As respondents are not randomly sampled, we will not calculate a response rate.

For qualitative data collection, the research team will work closely with administrators and staff to recruit families and staff for us to interview. Participants in the interviews and focus groups will be identified through a convenience sample identified by Matrix and Starfish administrators and staff. To further increase the likelihood of participation, we will offer focus group and interview participants and staff gift cards as tokens of appreciation, as discussed in Supporting Statement Part A.

Non-Response

As participants will not be randomly sampled and findings are not intended to be representative, interview and focus group non-response bias will not be calculated.

B6. Production of Estimates and Projections

Findings will only present an internally valid description of the program process from both the administrative and client perspectives in chosen sites. Results will be included in the final public report. The data collected from this IC will not be used to make policy decisions.

B7. Data Handling and Analysis

Data Handling

With study participant permission, we will audio record conversations and use these records to minimize errors due to coding and data processing.

Data Analysis

Interviewers will take detailed notes during the focus groups or interviews. Interviewers will write summary notes after each interview to record early impressions and emerging patterns. These notes will not be considered project data to be analyzed, but they will serve as a record of initial (emerging) themes that will then be followed up on by a more thorough analysis to better understand the program processes and staff and client perceptions. The analysis will focus on answering our six primary diagnosis research questions, listed below:

1. What are parent perspectives on and experiences with the EHS/HS program?

2. What are parent perspectives on challenges to program engagement and child attendance?

3. How do site operations work, especially related to parent engagement and child attendance?

4. What are staff perspectives on their work, successful parent engagement, and child attendance?

5. What are staff perspectives on challenges to parent engagement and to child attendance?

Data Use

The goal of the BIAS-NG Generic Clearance is to conduct qualitative and descriptive quantitative research to identify and understand the psychological and behavioral factors that can affect the effectiveness of human service programs. The diagnosis research will allow the team to gather structured in-depth information to understand the program process from both the administrative staff and client perspectives. Focus groups and interviews are essential to identifying the points in the outreach and delivery of services, or in the client’s experiences, that are most amenable to a behavioral intervention.

Aside from summarizing findings in public reports as part of the project’s dissemination activities, no other documentation will be released regarding this data collection. Our dissemination goals prioritize sharing findings with practitioners, decisionmakers, and academic scholars at the federal, state, and local levels. We will document the limitations of the resulting data when making anything public.

B8. Contact Persons

Clint Key, MDRC, Project Director – [email protected]

Helen Lee, MDRC, Site Lead – [email protected]

Kim Clum

Senior Social Science Research Analyst

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

330 C St. SW, 4th Floor

Washington, DC 20201

(202) 205-4922

Victoria Berk Kabak

Senior Social Science Research Analyst

Office of Planning, Research & Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

330 C St. SW, 4th Floor

Washington, DC 20201

(202) 401-5871

Attachments

• Instrument 1 – Matrix and Starfish EHS/HS Parent Interview and Focus Group Protocol

• Instrument 2 – Matrix and Starfish EHS/HS Staff Interview and Focus Group Protocol

7