NonSub Change - BASE updates for intervention and to cost workbook

Memorandum

To: Kelsi Feltz

Office of Information and Regulatory Affairs (OIRA)

Office of Management and Budget (OMB)

From: Ann Rivera

Office of Planning, Research and Evaluation (OPRE)

Administration for Children and Families (ACF)

Date: January 18, 2024

Subject: Nonsubstantive Change Request – Build and Sustaining the Child Care and Early Education Workforce Study [OMB # 0970-0615]

This memo requests approval of nonsubstantive changes to the approved information collection for the Building and Sustaining the Child Care and Early Education (BASE) Study [OMB Information Collection Request 0970-0615, approved September 1, 2023].

Background

The BASE project aims to understand factors that affect child care and early education (CCEE) workforce dynamics, including employment recruitment, retention, and advancement, as well as to build the evidence base about strategies that may help to recruit, retain, and advance the CCEE workforce.

The information collection aims to build the evidence base about the implementation, costs, and impacts of strategies aimed at improving the compensation and economic well-being of educators in child care center-based and home-based settings. The BASE project will do so by leveraging two pilot initiatives being implemented by the Colorado Department of Early Childhood (CDEC) that provide additional funding and supports to center-based child care settings to improve teacher compensation and funding for home-based child care settings to support the provision of benefits to owners (referred to as providers hereafter) and staff (referred to as assistants hereafter).

The study includes four substudies:

• Impact Study:

  • Follow-up Surveys (Instrument 1 and 2) of center directors and lead and assistant teachers across research conditions

• Descriptive Study:

  • Follow-up Survey (Instrument 3) of home-based child care providers and assistants across research conditions

• Implementation Study:

  • Interview Protocols (Instruments 4 and 5) with a sample of center directors and lead and assistant teachers in the intervention condition

  • Interview Protocols (Instrument 6) with a sample of home-based care providers and assistants in the intervention condition

  • Interview Protocol (Instrument 7) with a sample of key informants in Colorado implementing agencies

• Cost Study:

  • Cost Workbook (Instrument 8) with a sample of center-based directors or administrators across research conditions

Since approval, CDEC finalized the intervention plans and as a result, the study team has identified updates needed to two instruments for the home-based provider data collection (i.e., the follow-up home-based provider and assistant survey and the one-on-one home-based provider and assistant interview). We have also identified some alternative procedures that may improve data collection efforts for the center-based providers (i.e., offering potential respondents the alternative to complete a shortened cost workbook.)

Data collection with center-based providers: In the center-based intervention, collection of program cost information is ongoing, utilizing the approved cost workbook. However, recruitment of center-based directors has proved challenging, given the time commitment required to complete the workbook. As discussed below, the team is proposing to offer directors the option of an abbreviated data collection instrument and procedure that would capture selected portions of the workbook in less time.

Data collection with home-based providers: The survey and interview protocols for the home-based providers, approved by OMB, were developed assuming the intervention for this group would be similar in nature to that for center-based providers. In late 2023, CDEC finalized the intervention for home-based providers. The intervention improves teachers’ compensation, as intended, but is implemented differently to better fit the structure and needs of home-based child care providers and assistants: annual award amounts for two years that can be used to fund medical benefits (health, dental, vision, HSA/FSA), paid time off (substitute teacher coverage or revenue replacement during closures), and retirement contributions. 

The purpose of the data collection is the same as originally described in the approved OMB materials, and the methods of data collection are the same. The changes to the data collection instruments are only meant to remove items and refine the wording of some items to better align with the design and implementation of the finalized intervention with home-based providers, including language used in CDEC’s initiative outreach materials, and to reduce burden on home-based providers and assistants and on center-based directors. We believe these revisions to these study protocols will enhance the quality and amount of the data collected to address the government’s questions about the impact and implementation of workforce compensation strategies. We request a response to these proposed changes as soon as possible so that we can continue data collection for the center-based cost study and begin data collection for the home-based provider study.

Overview of Requested Changes for Cost Workbook Data Collection for Center-based Providers

The objective of the cost study is to detail the costs associated with teacher recruitment and turnover when operating a center-based child care setting and delivering services. The cost workbook is designed to collect information about time spent on activities related to educator vacancies, recruitment and hiring, and training new educators, and detailed information about salaries and fringe benefits for staff who work on these activities. The goal is to collect this information from 12 providers. The original approved cost workbook was estimated to take respondents 5 hours to complete, including a 30-minute technical assistance (TA) call with research team staff. The original procedure included research team staff walking through the workbook with respondents during the 30-minute call, checking for respondent understanding of items, and addressing any confusion about the information requested in the workbook. Since beginning data collection, respondents have on occasion opted-out of responding to the cost workbook entirely, citing the lengthy time commitment.

Our team proposes adjusting this data collection process by offering respondents two choices:

1. Conduct the full workbook and TA call (as originally approved by OMB);

2. Conduct just the 30-minute TA call with a research team staff member to complete key portions of the workbook together.

We are not proposing a change in the TA call or the workbook itself. We also do not propose a change to the originally estimated burden because we are unsure how many respondents will select the TA call only option. Instead, we are proposing a shortened data collection process using the existing, approved data collection materials and approved TA call—offered as an alternative for respondents. Research team staff members will be trained on which sections of the workbook to prioritize in TA calls, which are listed below in Table 1. Items identified as highest priority will be the primary focus of the TA call, while questions classified as moderate priority will be covered only if appropriate and time allows. Low priority items are unlikely to be collected as part of the abbreviated data collection option, given that the call will only last approximately 30 minutes.

To reflect the different levels of effort between the two options, we propose offering the planned $250 honorarium to respondents who choose to complete the entire cost workbook, including $200 for the time spent completing the workbook and $50 for the time spent completing the TA call. For respondents that opt to complete the TA call only, we propose offering a $50 honorarium for the time spent completing the TA call during which research team staff will assist in the completion of selected portions of the cost workbook.

Table 1. Edits to the cost workbook for the abbreviated cost study protocol

	Cost Workbook Tab	Summary of Changes for TA Call Option	Rationale
Instructions		No changes
Definitions		No changes
Teaching vacancies		Highest priorities:   C1. Center time spent on teaching vacancies  Moderate priorities:  C2. Differences in time by assistant/aide  C3. Differences in time by child age  C5. Expenses (rather than time)  Lowest priorities:  C4. Activities borne by larger organizations  C6. Differences in expenses by assistant/aide C7. Differences in expenses by child age  C8. Expenses borne by larger organizations  C9. Explanatory note	Research team staff will be instructed to fill in the sections on center time spent on teaching vacancies and to ask centers if they have any additional expenses spent on teaching vacancies.     The differences in time by role and age are interesting sub analyses but are not essential to answering the proposed research questions.    In most cases, the expenses borne by larger organization questions have not been relevant to the centers in the sample who have completed the cost workbook so far.   The explanatory note is no longer necessary given the transition from written to collaborative completion with a research team member.
Recruitment and hiring		Highest priorities:   D1. Center time spent on recruitment and hiring activities  Moderate priorities:  D2. Differences in time by assistant/aide  D3. Differences in time by child age  D5. Expenses (rather than time)  Lowest priorities:  D4. Activities borne by larger organizations  D6. Differences in expenses by assistant/aide D7. Differences in expenses by child age D8. Expenses borne by larger organizations  D9. Explanatory note	Same as row C.
Training and onboarding		Highest priorities:   E1. Center time spent on training and onboarding  Moderate priorities:  E2. Differences in time by assistant/aide  E3. Differences in time by child age  E5. Expenses (rather than time)  Lowest priorities:  E4. Activities borne by larger organizations E6. Differences in expenses by assistant/aide E7. Differences in expenses by child age E8. Expenses borne by larger organizations  E9. Explanatory note	Same as row C.
Pilot activities (Treatment centers only)		Highest priority:   F1. Center time spent completing application materials F2. Center time spent processing lead and assistant teachers/aides’ opt-in/opt-out decisions F3. Center time spent on monthly reporting requirements of the pilot initiative F4. Center time spent on monthly pilot initiative wage supplements	No changes.
Salaries		Lowest priority: G1. Staff salaries G2. Payroll tax & fringe benefits G3. Additional benefits G4. Explanatory note G5. Changes to salaries & benefits	Because we will use standardized wages and because salary data are available from other data sources being collected as part of the study, this section can be removed to free up time in the call for higher priority questions.
Your Center		Highest Priority:  H4. Number of educators that ended their employment since January H5. Number of weeks to fill vacancies    Moderate Priority:  H6. Number of educators hired since January H7. Number of current vacancies    Lowest priority:  H1. Number of classrooms open since January H2. Number of educators employed in January H3. Current number of educators employed today	Information on turnover and time to fill positions is sufficient for the cost study, additional data in this section is not essential to answer the proposed research questions and can be down prioritized as needed.

Overview of Requested Changes for Home-Based Providers

The objective of the study for the home-based providers remains the same as originally proposed-- to describe the experiences of home-based child care owners and their assistants with CDEC’s initiative. Given that the intervention for home-based providers is somewhat different from the center-based initiative, the data collection instruments for home-based providers require some adjustment to align with the intervention. The proposed changes remove questions referring to a salary increase intervention and update question wording and response options slightly to better reflect the nature of the intervention for home-based providers. The proposed modifications also aim to reduce the burden of the survey and the interview for respondents. We are not proposing any adjustments to the data collection approach or approved time period. In addition, we have edited all documents to ensure that the language with respect to roles is consistent with CDEC’s initiative and with study outreach materials: owners will be referred to as “providers” and caregivers will be referred to as “assistants”.

1. Follow-up home-based provider and assistant survey

The proposed changes to the follow-up surveys include:

1. removing questions that are not relevant to the current intervention for home-based providers.

2. replacing instances of “owners” with “providers” and “caregivers” with “assistants” throughout the survey.

Within each survey section that is retained, the goal is to capture the construct of interest (e.g., financial well-being, physical/mental health, etc.) and any potential effects of the intervention with fewer questions. Overall, these proposed modifications reduce estimated burden on survey respondents from the planned 45-minute survey to 25 minutes.

Table 2. Edits to the follow-up home-based provider and assistant survey

Section	Questions removed and details of change
Section A: About your family child care home	Providers: Removed 11 questions. Removed questions A4-A6 on home care schedule (pp 2-3). Removed A10 on existence of staff (p 4), because covered in A11. Removed A14-A20 (pp 4-5) on language spoken, use of support staff or specialists.
	Assistants: No changes. Section not administered to assistants.
Section B: About your experiences with the pilot initiative	Providers: Removed entire section Given the final structure of the intervention, this section (pp 7-8) is no longer relevant to home-based providers.
	Assistants: Removed entire section Given the final structure of the intervention, this section (pp 7-8) is no longer relevant to home-based providers.
Section C: About your financial situation and job	Providers: Removed entire section. Removed entire section (pp 8-18), given that questions about role, wages, hours, and benefits offered are not relevant to providers.
	Assistants: Removed 32 questions. Removed questions C1-C5 (pp 8-9) on current employment status, C8 (p 10) on hours worked, C11 (p 10) on schedule, and C17-C42 (pp 12-18) on characteristics of other jobs.
Section D: About children you provide care for	Providers: No changes
	Assistants: No changes. Section not administered to assistants.
Section E: About your professional background	Providers: Removed three questions Removed questions E1 (p 21) and E4-E5 (p 21) about search for a new job.
	Assistants: Removed three questions Removed questions E3-E5 (p 21) about running a child care home and job search.
Section F: Future job plans	Providers: Removed two questions Removed questions F7-F8 (p 23) about professional associations and current enrollment in child care related coursework.
	Assistants: Removed two questions Removed questions F7-F8 (p 23) about professional associations and current enrollment in child care related coursework.
Section G: About your job demands and supports	Providers: Removed six questions Removed G1-G3 (pp 23-24) on job tasks and support from other adults and G5-G7 (pp 24-25) on licensing tasks.
	Assistants: Removed entire section Removed all questions (pp 23-25) on job tasks and support from other adults and licensing tasks.
Section H: Feelings about your job	Providers: Removed 12 questions Removed H2-H4 (p 25) related to job satisfaction, and H6-H14 (pp 26-28) on job self-efficacy and discrimination.
	Assistants: Removed eight questions Removed H2-H4 (p 25) related to job satisfaction, H6-H10 (pp 26-27) on job self-efficacy and discrimination, and H13-H14 (pp 27-28) on receiving a raise.
Section I: About your financial situation	Providers: Removed four questions Removed I10-I13 (p 30) capturing student loans, experiences with eviction, not paying full utilities, perception of adequate resources at the end of the month, and perception of financial well-being.
	Assistants: Removed four questions Removed I10-I13 (p 30), capturing student loans, experiences with eviction, not paying full utilities, perception of adequate resources at the end of the month, and perception of financial well-being.
Section J: Your health and wellbeing	Providers: Removed two questions Removed J3 and J4 (p 31) about anxiety and job stress.
	Assistants: No change
Section K: A little more about you	Providers: No change
	Assistants: No change

2. One-on-one home-based provider and assistant interview

The proposed changes to the provider and assistant interview include:

1. removing questions that are not relevant to the current intervention for home-based providers

2. refining questions to better align with the structure of the final intervention.

Within each Section that is retained, the goal is to capture the construct of interest (e.g., changes due to the pilot, financial well-being, psychological wellbeing/health, etc.) and any potential effects of the intervention. In addition, to better align with the final intervention structure and reduce burden, the interview will only be administered to providers and not to assistants. Overall, these proposed modifications reduced estimated burden on respondents from the planned 90 minutes to 60 minutes. The changes are described in more detail in Table 3.

Table 3. Edits to the one-on-one home-based provider interview protocol

Section	Questions removed and details of change
N/A – throughout	Removed references and sections specific to “all respondents,” “assistant providers” or “those who did not complete an application” because we are now only interviewing providers that are in the Compensation Pilot.
Section 1: Initial exposure/ understanding of pilot initiative	Removed all four questions from Section 1 except for one question about how respondent initially thought and felt about the pilot initiative.
Section 2: Experiences with decision-making (Providers who did not complete the full application)	Removed all questions because decision-making for submitting an application to the pilot is no longer relevant to the intervention.
Section 3: Experiences with decision-making & application process (Providers who completed the application (treatment)	Removed all items because decision-making and process for submitting an application to the pilot is no longer relevant to the intervention.
Section 4: Experiences after initiative started (Providers in treatment group)	Updated questions to be in line with the design of the pilot initiative. Moved entire section (now Section 1 called Experiences after Pilot Started).
Section 5: Experiences on the job (all respondents in treatment group)	Removed questions about how the pilot affected the climate and relationships with others on the job; and the care and support provided for children and families because these are not relevant to the intervention.
Section 6: Experiences with business finances (providers in treatment)	Removed questions about specific operational costs that are no longer relevant in the final intervention (e.g., licensing, marketing, equipment costs).
Section 7: Experiences with personal finances (all respondents in treatment group)	Removed questions about sources of income and public assistance that are no longer relevant to the final intervention.
Section 8: Beliefs, expectations, and looking ahead (all respondents in treatment group)	Removed questions about specific uses of funds from Pilot that are no longer relevant to the final intervention (i.e., they are not allowable uses of the funds.)

Section	Questions revised and details of change
N/A – throughout	Removed “and caregiver” from instrument title. Replaced/updated all references to “[PILOT INITIATIVE]” or “Initiative” to say “Compensation Pilot” or “Pilot” to be consistent with the final intervention.
Introductions	Revised wording to incorporate names of providers and family child care homes. Revised wording since we are interviewing providers only. Updated “date pilot started” to be “July 2023.” Streamlined questioning and reordered some questions to create better conversation flow.
Section 1: Initial exposure/ understanding of pilot initiative	Moved questions from previous Section 4 to revised Section 1. Updated Section 1 title is now “Experiences after Pilot Started”. Added language to clarify use of term “Compensation Pilot”. Moved question about experiencing any other consequences of the pilot from previous Section 5 to revised Section 1.
Section 2: Experiences with decision-making (Providers who did not complete the full application)	Moved questions from previous Section 6 to revised Section 2. Updated Section 2 title is now “Experiences with Business Operations”. Moved two items about business challenges from previous Section 6 (i.e., worry might close, have enough families enrolled) to new Section 2 (pp 13-14). Updated questions about business finances to probe specifically about how they approach time off, how they fill spots, and what other potential sources of revenue they may have, since these aspects of operations may be affected by the final intervention (i.e., initiative may free-up resources that would have been spent on benefits for other activities. Added probe about whether they incurred any expenses related to the Pilot. Updated language in some places to better align with pilot materials. (i.e., financial health and stability)
Section 3: Experiences with decision-making & application process (Providers who completed the application (treatment))	Moved questions from previous Section 5 to revised Section 3. Updated Section 3 title is now “Experiences on the Job”. Updated probes about how pilot affected respondent and staff wellbeing to specifically include “economic or financial wellbeing”. Moved 2 questions about how pilot might change staffing or staff benefits from previous Section 8.
Section 4: Experiences after initiative started (Providers in treatment group)	Moved questions from previous Section 7 to revised Section 4. Updated Section 4 title is now “Experiences with Personal Finances”. Updated language in questions about respondent’s personal and family financial circumstances to better align with intervention (e.g., specifying changes in insurance benefits or job situation). Updated questions to better contrast personal finances before pilot started with after starting pilot.
Section 5: Experiences on the job (all respondents in treatment group)	Moved questions from previous Section 8 to revised Section 5. Updated Section 5 title is now “Beliefs, expectations, and looking ahead”.

Page 1 of 6