Alternative Supporting Statement for Information Collections Designed for
Research, Public Health Surveillance, and Program Evaluation Purposes
ACF Behavioral Interventions to Advance Self-Sufficiency Next Generation (BIAS-NG) Project
Generic Information Collection for Qualitative and Descriptive Quantitative Data Collection for Two Early Head Start/Head Start Sites
OMB Information Collection Request
0970 - 0502
Supporting Statement
Part A
December 2021
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officers:
Kim Clum
Executive Summary
Type of Request: This data collection is part of the Behavioral Interventions to Advance Self-Sufficiency Next Generation (BIAS-NG) project. This is a new generic information collection (GenIC) under the umbrella BIAS-NG generic.
Description of Request: This GenIC pertains to the behavioral diagnosis research for Matrix and Starfish, two ACF grantee sites in Wayne County, Michigan within the Early Head Start/Head Start (EHS/HS) domain. The information collected is intended to inform a behavioral diagnosis study, qualitative data collection activities that are critical to understanding the program process from both the administrative and client perspectives. The diagnosis activities will directly contribute to designing interventions that will be evaluated by a randomized controlled trial (RCT) in the next phase of the BIAS-NG study which will assess the impact of at least one behavioral intervention. Proposed data collection includes interviews and focus groups that will collect information essential to identifying the points in the processes used for outreach and delivery of services, or in the client’s experiences of these processes, that are most amenable to a behavioral intervention. This IC will allow the BIAS-NG team to triangulate the insights of behavioral science with the on-the-ground implementation of EHS/HS programs and parent and staff experiences. This IC is intended to yield an internally valid description of the programs’ processes and clients’ experiences for the purposes of behavioral diagnosis for the Starfish Family Services and Matrix Social Services EHS/HS sites, not to promote statistical generalization to other sites or service populations.
A1. Necessity for Collection
The Office of Planning, Research, and Evaluation (OPRE) at the Administration for Children and Families (ACF), U.S. Department of Health and Human Services (HHS), seeks Office of Management and Budget (OMB) approval to conduct interviews and focus groups with administrators, staff, and participants in Matrix and Starfish Early Head Start and Head Start (EHS/HS) programs to understand the points in the processes used for outreach and delivery of services, or in the client’s experiences of those processes, that are most amenable to a behavioral intervention geared towards improving program outcomes. This generic information collection (GenIC) is planned as part of ACF’s Generic Clearance for the Behavioral Interventions to Advance Self-Sufficiency Next Generation (BIAS-NG) project. The goal of the BIAS-NG Generic Clearance is to conduct qualitative and descriptive quantitative research to identify and understand the psychological and behavioral factors that can affect the effectiveness of human service programs, specifically Child Welfare (CW), Temporary Assistance for Needy Families (TANF), and EHS/HS.
There are no legal or administrative requirements that necessitate this IC. ACF is undertaking the collection at the discretion of the agency. ACF has contracted with MDRC to complete this work.
Study Background
The BIAS-NG project builds on a prior OPRE project, the Behavioral Interventions to Advance Self-Sufficiency (BIAS) project, which relied exclusively on administrative data to test the short-term impact of small “nudge” interventions in human services programs. Going beyond the work conducted for BIAS, the BIAS-NG project will test new interventions in more domains and collect a wider range of data.
The study described in this request is a behavioral diagnosis study in collaboration with Starfish and Matrix, two EHS/HS grantees. As indicated in the overarching generic clearance for the BIAS-NG project, Phase 3 is the Diagnosis and Design phase, where we review preexisting administrative data from each site and may complete the first round of interviews/focus groups included under this clearance. For this GenIC, the behavioral diagnosis study, we plan to collect qualitative and quantitative information from program staff and parents to better understand the behavioral bottlenecks, and when and how an intervention would be the most useful, in addition to collecting and reviewing existing administrative data on how the current programs are monitoring and measuring effectiveness.
A2. Purpose
Purpose and Use
The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.
This study is focused on the EHS/HS program goal of improving participant engagement. The goal of the BIAS-NG Generic Clearance is to conduct qualitative and descriptive quantitative research to identify and understand the psychological and behavioral factors that can affect the effectiveness of human service programs. The behavioral diagnosis research component (this GenIC) provides critical insights to designing an effective intervention, allowing the research team to properly diagnose ways in which agencies are not maximizing their impact for the populations they serve. The behavioral diagnosis research will allow the team to gather structured in-depth information to understand the program process from both the administrative and client perspectives. Focus groups and interviews are essential to identifying the points in the outreach and delivery of services, or in the client’s experiences, that are most amenable to a behavioral intervention. They allow the BIAS-NG team to triangulate: the insights of behavioral science; on-the-ground implementation of programs; and parent and staff experiences. The diagnosis activities will directly contribute to designing interventions that will be evaluated by a randomized controlled trial (RCT) in the next phase of the BIAS-NG study which will assess the impact of at least one of the designed behavioral intervention.1 Data collection will take place at two EHS/HS programs in Wayne County, Michigan: Starfish Family Services and Matrix Human Services. This information will be most applicable to these two sites but lessons learned could potentially be applied by other EHS/HS programs.
Research Questions or Tests
This specific GenIC is for data collection related to Phase 3 of the study described in the overarching generic clearance. Phase 3 is to conduct behavioral diagnosis and design, a procedure in which we examine the processes related to the problem of interest (to better understand the factors that may be inhibiting the desired outcomes and design solutions that are informed by behavioral science research to help improve outcomes). For example, using this behavioral diagnosis and design procedure we have identified barriers that TANF recipients may face that contribute to their lack of engagement in welfare-to-work programs. For this work, our goal is to understand staff experiences with the EHS/HS program, especially barriers and opportunities related to parent engagement and child attendance. Our specific research questions include the following:
1. What are parent perspectives on and experiences with the EHS/HS program?
2. What are parent perspectives on challenges to program engagement and child attendance?
3. How do site operations work, especially related to parent engagement and child attendance?
4. What are staff perspectives on their work, successful parent engagement, and child attendance?
5. What are staff perspectives on challenges to parent engagement and child attendance?
Study Design
Through this behavioral diagnosis study we will collect qualitative data from administrators, staff, and clients via focus groups and interviews to inform our intervention design. We will also collect administrative data from agency MIS systems to better understand parent experiences with the program and identify points where service delivery might need improvement.
These qualitative data collection activities are essential to designing an effective intervention, allowing the research team to properly diagnose ways in which agencies are not maximizing their impact for the populations they serve. These activities allow the team to gather structured in-depth information to understand the program process from both the staff and client perspectives.
Please see Instruments 1 and 2 for focus group and interview questions, and Table 1 for details.
Table 1: Research Question and Instrument Matrix
Data Collection Activity |
Instruments |
Respondent, Content, Purpose of Collection |
Mode and Duration |
Parent Focus Group/ Interview |
Instrument 1: Matrix and Starfish EHS/HS Parent Interview and Focus Group Protocol
|
Respondents: Parents participating in Matrix or Starfish EHS/HS program
Content:
Purpose: Understand parent experiences with the EHS/HS program, especially barriers and opportunities related to parent engagement and child attendance |
Mode: Focus group or interview
Duration: 1 hour |
Matrix and Starfish Staff Focus Group/ Interview |
Instrument 2: Matrix and Starfish EHS/HS Staff Interview and Focus Group Protocol
|
Respondents: Matrix and Starfish Head Start program staff
Content:
Purpose: Understand staff experiences with the EHS/HS program, especially barriers and opportunities related to parent engagement and child attendance |
Mode: Focus group or interview
Duration: 1 hour |
In the rest of this document and in Supporting Statement B, we include a description of:
Planned qualitative data collection (see Instruments 1 and 2).
Planned qualitative analyses.
Administrative data that the programs are already collecting and that the study will utilize.
Other Data Sources and Uses of Information
Administrative data will supplement the information collected in participant and staff focus groups/interviews to further understand parent engagement and attendance metrics and trends. This is administrative data that Matrix and Starfish already collect to monitor student attendance and parent engagement. Parent engagement data includes records from ChildPlus where sites capture different forms of staff-parent interactions, such as goal setting, meetings or home visits.
A3. Use of Information Technology to Reduce Burden
Participant and staff focus groups and interviews will be scheduled at convenient times or when they are already planning to be in EHS/HS centers for activities. If a phone or video interview is easier for participant or staff schedules, we may conduct interviews in that mode. Interviews and focus groups will be recorded, with permission from the participant.
A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency
We have worked carefully with the EHS/HS programs (Matrix and Starfish) to understand the data the programs routinely collect. None of the data currently collected by the agency would allow us to assess participant understanding of the current processes or parent and staff perspectives on barriers and facilitators. In addition, the study team will not collect information from the programs that is available from existing public sources.
A5. Impact on Small Businesses
We do not anticipate any small organizations to be affected by this IC. Nonetheless, we will schedule interviews at times that are convenient to participants in order to minimize disruption of daily activities.
A6. Consequences of Less Frequent Collection
Rigorous evaluation of innovative initiatives is crucial to building evidence of what works and how best to allocate scarce government resources. These data collection undertakings represent an important opportunity for ACF to both learn about activities associated with EHS/HS and to design behavioral interventions to improve service delivery and uptake. Not collecting information from the two categories of respondents (staff and clients) during Phase 3 would limit the government’s ability to design appropriately targeted interventions that match the barriers administrators, staff, and clients face in the quest for optimal service delivery.
A7. Now subsumed under 2(b) above and 10 (below)
A8. Consultation
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on May 23, 2017, Volume 82, Number 98, page 23572, and provided a sixty-day period for public comment. An additional comment period was provided in July 2019 (84 FR 33947), requesting public comment on a revision request to add a third potential domain (EHS/HS) to the overarching generic. No substantive comments were received during the notice and comment periods.
The research team has consulted with leading behavioral science experts, stakeholders, and experts on the behavioral diagnosis study data collection instruments.
A9. Tokens of Appreciation
EHS/HS staff and parents who participate in interviews will receive a gift card of $25. We intend for the gift card to help offset the higher out of-pocket costs to respondents for time spent on the interview, additional cell-phone data or phone minutes, or child care costs associated with interviews. Participants involved with EHS/HS are likely to be under some stress and potentially difficult to reach. These participants are also parents, often with young children, which means that they may have to pay out-of-pocket for child care during the time of completing the interview. Based on MDRC’s experience with this population, a $25 gift card will help to offset these incidental costs associated with participation. We do not believe this token of appreciation is so high as to be coercive for participants.
Tokens of appreciation of this amount have been used in prior research activities and approved by the MDRC Institutional Review Board (IRB) and OMB for the BIAS-NG project. For example, with TANF participants in Monroe County, NY, a similar population where the majority of parents enrolled in the study who were approached to participate agreed to the interview.
We have secured IRB approval for the overall BIAS-NG study and for the use of $25 tokens of appreciation for similar interviews in participant implementation research in other BIAS-NG sites. Following OMB approval, the IRB will receive the EHS/HS behavioral diagnosis research study, which we anticipate will be approved quickly, given that other BIAS-NG sites were approved by for similar research efforts.
A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing
Personally Identifiable Information
For interviews conducted remotely, by phone or video, participants’ first names and phone numbers will be collected. Collecting phone numbers is required to contact participants for interviews, and first names are required to conduct interviews.
Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.
Assurances of Privacy
Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. Participants will only be recorded with their permission. As specified in the contract, the Contractor will comply with all Federal and Departmental regulations for private information.
All respondents who participate in research under this clearance will be read a statement that will explain the study and will inform individuals that their participation is voluntary and of the extent of their privacy as respondents (informed consents are included in each of the Instruments). Participants will be told verbally that their conversations will not be shared in a form that identifies the individual with anyone outside the research team. As ACF’s prime contractor, MDRC plans to implement all data collection activities. Information will be kept private to the extent permitted by law and in accordance with current Federal information security standards and other applicable regulations.
Data Security and Monitoring
MDRC qualitative researchers store data temporarily on laptop hard drives that are encrypted at rest and configured to protect from unauthorized access. Also, MDRC provides researchers with HIPAA-compliant recording devices that encrypt audio data at rest. Per MDRC policy, upon return to MDRC, project team members copy these data files to designated secure folders within MDRC’s secure environment and return the laptop or recording device to MDRC’s IT department. IT staff members then run a U.S. Department of Defense-approved hard drive sanitizing utility to securely remove and overwrite all files from the laptop or recording device. MDRC employees are required to maintain and process quantitative and qualitative data in designated project folders on the MDRC network. With the exception of the temporary storage of data during onsite collection, MDRC employees are not allowed to download, keep, or process individual-level data on the hard drives of their MDRC work stations or any other storage.
The project Data Manager will organize BIAS-NG project folders and will supervise storage of BIAS-NG data files. All reports, tables, and printed materials are limited to presentation of aggregate numbers. Original project notes and recordings will be stored in secure folders with limited access rights for use only by authorized behavioral diagnosis study researchers. These original documents will be stored until the published report, to refer to these sources for fact-checking. MDRC will destroy all paper records and electronic records containing PII when no longer needed for research purposes in accordance with funder and contractual requirements, as well as MDRC retention policies.
We plan to share the data we collect and protect the privacy of the individual data collected in the form of Restricted Access Files (RAF) archived with the Inter-university Consortium for Political and Social Research (ICPSR). RAFs deposited with ICPSR are restricted to approved users who have signed a legal agreement tightly limiting their acceptable use, analysis, and disclosure of the data. Per MDRC standard procedure, the Data Librarian and project Data Manager will verify that all incoming files are accounted for at the end of the project – deleted or permanently archived, per agreement with funder and data providers.
A11. Sensitive Information 2
We are asking some sensitive questions in this IC in terms of staff relationships with each other and participant relationships with staff. For example, we ask parents in an interview to discuss their goals and reflect on their relationship with their Head Start center. We also ask staff in interviews to indicate their perceptions of obstacles facing their participants. These answers will help the study team address core diagnosis research questions around participant and staff experiences of the program and its processes. We will assure parents and staff that no one at their sites will not see their responses in any way that can be linked back to them, to encourage honest responses.
MDRC’s IRB has approved the overall BIAS-NG impact study and similar behavioral diagnosis research protocols. The IRB formally reviews study protocols under the project after they receive OMB approval.
A12. Burden
Explanation of Burden Estimates
Across Matrix and Starfish, we expect to speak with a total of up to 50 participants (EHS/HS parents), up to 20 EHS/HS center leaders, up to 40 EHS/HS staff who work as family service guides, and up to 30 HS/EHS teachers. Each interview or focus group is expected to take about 1 hour. The estimate below represents an upper bound on potential burden.
Table 2: Burden Hours for Matrix and Starfish EHS/HS Sites: Matrix and Head Start
Activity |
No. of Respondents (total over request period) |
No. of Responses per Respondent (total over request period) |
Avg. Burden per Response (in hours) |
Total Burden (in hours) |
Average Hourly Wage Rate |
Total Annual Respondent Cost |
EHS/HS Parent Interview/Focus Group (Instrument 1) |
50 |
1 |
1 |
50 |
$9.65 |
$482.50 |
EHS/HS Center Leaders Interview/ Focus Group (Instrument 2) |
20 |
1 |
1 |
20 |
$17.57 |
$351.40 |
EHS/HS Family Service Guides Interview/ Focus Group (Instrument 2) |
40 |
1 |
1 |
40 |
$26.41 |
$1,056.40 |
EHS/HS Teachers Interview/ Focus Group (Instrument 2) |
30 |
1 |
1 |
30 |
$23.85 |
$715.50 |
Total |
140 |
|
|
140 |
|
$2605.80
|
Estimated Annualized Cost to Respondents
We estimate the average hourly wage for Head Start teachers to be the average hourly wage of “Preschool Teachers, Except Special Education” in the Detroit-Warren-Dearborn, Michigan metropolitan area taken from the U.S. Bureau of Labor Statistics, May 2020 Occupational Employment and Wage Statistics ($17.57). We rely on the same data source to estimate the hourly wage for Head Start Center Directors, using the “Education and Childcare Administrators, Preschool and Daycare” category in the Detroit-Warren-Dearborn, Michigan metropolitan area ($26.41). We rely on the same data source to estimate the hourly wage for Head Start Family Support Staff, using the “Community and Social Service Specialists” category in the Detroit-Warren-Dearborn, Michigan metropolitan area ($23.85). To compute the total estimated cost for participants, the total burden hours were multiplied by $9.65, the Wayne County minimum wage as of January 1, 2021.
A13. Costs
The data collections proposed under this generic IC involve imposing time burdens on very busy administrative and frontline staff in human services agencies. Based upon our experience in the field to date under this package, we propose offering a small honorarium of $25 to program staff participating, in recognition of the time and professional expertise they contribute to the studies. These honoraria are intended to both encourage staff participation and recognize their efforts to support a timely and high-quality data collection. A similar honorarium was provided for the last GenIc for the LA TANF site’s implementation research study.
A14. Estimated Annualized Costs to the Federal Government
The total cost for the behavioral diagnosis research data collection, analysis, and reporting activities under this current IC request will be approximately $70,416. Annual costs to the Federal government will be approximately $23,472 (the annual cost over the course of 3 years). There will be no notable costs beyond normal labor costs for staff.
Cost Category |
Estimated Costs |
Behavioral Diagnosis Research Field Work |
$39,268 |
Publications/Dissemination (Diagnosis Research section of final report) |
$31,148 |
Total costs over the request period |
$70,416 |
Annual costs |
$23,472 |
A15. Reasons for changes in burden
This is an individual IC under the BIAS-NG Generic Clearance (0970-0502).
A16. Timeline
Phase 3: Behavioral Diagnosis and Design
During Phase 3, we will collect the qualitative data requested in this IC from staff and clients via focus groups and interviews, to inform our intervention design. Data collection will take place approximately two weeks following OMB approval, for a period of approximately two months. We also collect administrative data from agency MIS systems to better understand client experiences with the program and identify points where service delivery might need improvement.
Phase 4: Evaluation
Phase 4 consists of implementing the behavioral intervention and evaluating it and collecting long-term outcomes. This will occur for approximately two years, starting approximately one year following this OMB approval.
Phase 5: Dissemination
Dissemination efforts during the time of this clearance include site-specific reports, infographics, products aimed at practitioners, sharing findings at conferences, and publicizing our findings and our work on social media. Dissemination efforts are expected to begin after analysis concludes (between 1-3 years after OMB approval).
A17. Exceptions
No exceptions are necessary for this information collection.
Attachments
Instrument 1 – Matrix and Starfish EHS/HS Parent Interview and Focus Group Protocol
Instrument 2 – Matrix and Starfish EHS/HS Staff Interview and Focus Group Protocol
1 Under the umbrella BIAS-NG generic, the next phase of the BIAS-NG study is Phase 4 which relies on the use of administrative data already collected by the programs and new data collected for implementation research. We will submit an individual IC request for implementation research instruments, as we have done with four previous sites under this generic clearance.
2 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Liza Rodler |
| File Modified | 0000-00-00 |
| File Created | 2022-06-14 |