NonSub Change Memo - Increase burden for sample

TO: Jordan Cohen, Office of Information and Regulatory Affairs, Office of Management and Budget

FROM: Nina Philipsen and Bonnie Mackintosh, Office of Planning, Research, and Evaluation, Administration for Children and Families

DATE: May 19, 2022

RE: NonSubstantive Change Request – Early Care and Education Leadership Study (ExCELS) Descriptive Study (OMB: 0970-0582)

Background

The Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families in the U.S. Department of Health and Human Services contracted with Mathematica and its subcontractor, the Institute for Early Education Leadership and Innovation at the University of Massachusetts Boston, to conduct the Early Care and Education Leadership Study (ExCELS). The purpose of ExCELS is to learn about leadership in center-based early care and education (ECE) settings, and better understand how leadership might improve the quality of care and education centers provide and outcomes for staff, children, and families. The ExCELS descriptive study was launched in March 2022 with a plan to recruit 120 center-based ECE settings. These centers need to have at least one primary site leader (e.g., center director) in the building, receive funding from Head Start or the Child Care and Development Fund, and serve children whose ages range from birth to age 5 (but who are not yet in kindergarten). Data collection includes interviews with each center’s primary site leader and surveys for select center managers and all teaching staff. Data from the descriptive study will help develop a psychometrically sound measure of leadership.

The study design includes a purposive sample of 120 center-based ECE settings from four states, selected to achieve variation that is critical for assessing the theory of change for ECE leadership and for psychometric analyses to develop a measure of leadership. The statistical precision of the planned analysis rests primarily on the number of staff respondents to surveys. The team has based assumptions of survey completes on the number of participating centers, the average number of staff in center-based settings, and a high response rate. A sample of 120 centers will support the ability to achieve sample targets for survey respondents that are needed to support the planned analysis.

Since launching recruitment, the study team has had to conduct recruitment calls with a larger number of centers to engage one in the study. We are seeing that 30% of centers with which the team has conducted a recruitment call are agreeing to participate in the study. Centers are reporting being short-staffed, having many demands on their time, and continued staff stress and fatigue, due in part to the ongoing pandemic. We are also finding that larger numbers of centers are being found ineligible than anticipated (30% of those agreeing to participate after the recruitment call). This has required conducting engagement interviews with a larger number of centers than planned to successfully identify eligible centers for the study.

Overview of Requested Changes

We are requesting an increase to the burden estimates to allow for outreach to a larger number of centers and to account for potential attrition at different stages of engagement. These increases will help us complete data collection with 120 centers to support the study goals. We have updated Supporting Statement A to reflect the requested changes to burden estimates by instrument and in total.

The team, in coordination with OPRE, has also made three adjustments to the sampling selection and recruitment targets to be more flexible in bringing centers into the study and determining eligibility. First, the team has removed specific recruitment targets by geographic location (urban or suburban) but continues to release center sample by geographic location to help achieve balance in the sample. Second, the team has revised the thresholds for receipt of Head Start or CCDF funding from 50% to 20% to expand eligibility based on funding. Third, the team collapsed recruitment targets by center size into large and medium/small combined, given the challenge in recruiting small centers with fewer than 25 children. Eligibility for the study continues to specify that centers must have at least two classrooms; this criterion is sometimes at odds with a small center size based on licensed capacity. None of these changes affect the analytical goals of the study, the utility of the data for establishing the psychometric properties of a new measure of leadership, or the uses of the study findings to understand leadership in ECE center-based settings. We have updated Supporting Statement B to reflect these changes.

Time Sensitivity

The study team is working to complete recruitment in June 2022 and data collection in July 2022.