NonSub Change - BASE updates for intervention and to cost workbook

BASE_Nonsubstantive change request_OMB_revised 1.17.2024_FINAL.docx

Building and Sustaining the Child Care and Early Education Workforce (BASE)

NonSub Change - BASE updates for intervention and to cost workbook

OMB: 0970-0615

Document [docx]
Download: docx | pdf

Memorandum

To: Kelsi Feltz

Office of Information and Regulatory Affairs (OIRA)

Office of Management and Budget (OMB)


From: Ann Rivera

Office of Planning, Research and Evaluation (OPRE)

Administration for Children and Families (ACF)


Date: January 18, 2024

Subject: Nonsubstantive Change Request – Build and Sustaining the Child Care and Early Education Workforce Study [OMB # 0970-0615]



This memo requests approval of nonsubstantive changes to the approved information collection for the Building and Sustaining the Child Care and Early Education (BASE) Study [OMB Information Collection Request 0970-0615, approved September 1, 2023].


Background

The BASE project aims to understand factors that affect child care and early education (CCEE) workforce dynamics, including employment recruitment, retention, and advancement, as well as to build the evidence base about strategies that may help to recruit, retain, and advance the CCEE workforce.

The information collection aims to build the evidence base about the implementation, costs, and impacts of strategies aimed at improving the compensation and economic well-being of educators in child care center-based and home-based settings. The BASE project will do so by leveraging two pilot initiatives being implemented by the Colorado Department of Early Childhood (CDEC) that provide additional funding and supports to center-based child care settings to improve teacher compensation and funding for home-based child care settings to support the provision of benefits to owners (referred to as providers hereafter) and staff (referred to as assistants hereafter).


The study includes four substudies:

  • Impact Study:

    • Follow-up Surveys (Instrument 1 and 2) of center directors and lead and assistant teachers across research conditions

  • Descriptive Study:

    • Follow-up Survey (Instrument 3) of home-based child care providers and assistants across research conditions

  • Implementation Study:

    • Interview Protocols (Instruments 4 and 5) with a sample of center directors and lead and assistant teachers in the intervention condition

    • Interview Protocols (Instrument 6) with a sample of home-based care providers and assistants in the intervention condition

    • Interview Protocol (Instrument 7) with a sample of key informants in Colorado implementing agencies

  • Cost Study:

    • Cost Workbook (Instrument 8) with a sample of center-based directors or administrators across research conditions


Since approval, CDEC finalized the intervention plans and as a result, the study team has identified updates needed to two instruments for the home-based provider data collection (i.e., the follow-up home-based provider and assistant survey and the one-on-one home-based provider and assistant interview). We have also identified some alternative procedures that may improve data collection efforts for the center-based providers (i.e., offering potential respondents the alternative to complete a shortened cost workbook.)


Data collection with center-based providers: In the center-based intervention, collection of program cost information is ongoing, utilizing the approved cost workbook. However, recruitment of center-based directors has proved challenging, given the time commitment required to complete the workbook. As discussed below, the team is proposing to offer directors the option of an abbreviated data collection instrument and procedure that would capture selected portions of the workbook in less time.

Data collection with home-based providers: The survey and interview protocols for the home-based providers, approved by OMB, were developed assuming the intervention for this group would be similar in nature to that for center-based providers. In late 2023, CDEC finalized the intervention for home-based providers. The intervention improves teachers’ compensation, as intended, but is implemented differently to better fit the structure and needs of home-based child care providers and assistants: annual award amounts for two years that can be used to fund medical benefits (health, dental, vision, HSA/FSA), paid time off (substitute teacher coverage or revenue replacement during closures), and retirement contributions. 


The purpose of the data collection is the same as originally described in the approved OMB materials, and the methods of data collection are the same. The changes to the data collection instruments are only meant to remove items and refine the wording of some items to better align with the design and implementation of the finalized intervention with home-based providers, including language used in CDEC’s initiative outreach materials, and to reduce burden on home-based providers and assistants and on center-based directors. We believe these revisions to these study protocols will enhance the quality and amount of the data collected to address the government’s questions about the impact and implementation of workforce compensation strategies. We request a response to these proposed changes as soon as possible so that we can continue data collection for the center-based cost study and begin data collection for the home-based provider study.


Overview of Requested Changes for Cost Workbook Data Collection for Center-based Providers

The objective of the cost study is to detail the costs associated with teacher recruitment and turnover when operating a center-based child care setting and delivering services. The cost workbook is designed to collect information about time spent on activities related to educator vacancies, recruitment and hiring, and training new educators, and detailed information about salaries and fringe benefits for staff who work on these activities. The goal is to collect this information from 12 providers. The original approved cost workbook was estimated to take respondents 5 hours to complete, including a 30-minute technical assistance (TA) call with research team staff. The original procedure included research team staff walking through the workbook with respondents during the 30-minute call, checking for respondent understanding of items, and addressing any confusion about the information requested in the workbook. Since beginning data collection, respondents have on occasion opted-out of responding to the cost workbook entirely, citing the lengthy time commitment.

Our team proposes adjusting this data collection process by offering respondents two choices:

  1. Conduct the full workbook and TA call (as originally approved by OMB);

  2. Conduct just the 30-minute TA call with a research team staff member to complete key portions of the workbook together.

We are not proposing a change in the TA call or the workbook itself. We also do not propose a change to the originally estimated burden because we are unsure how many respondents will select the TA call only option. Instead, we are proposing a shortened data collection process using the existing, approved data collection materials and approved TA call—offered as an alternative for respondents. Research team staff members will be trained on which sections of the workbook to prioritize in TA calls, which are listed below in Table 1. Items identified as highest priority will be the primary focus of the TA call, while questions classified as moderate priority will be covered only if appropriate and time allows. Low priority items are unlikely to be collected as part of the abbreviated data collection option, given that the call will only last approximately 30 minutes.

To reflect the different levels of effort between the two options, we propose offering the planned $250 honorarium to respondents who choose to complete the entire cost workbook, including $200 for the time spent completing the workbook and $50 for the time spent completing the TA call. For respondents that opt to complete the TA call only, we propose offering a $50 honorarium for the time spent completing the TA call during which research team staff will assist in the completion of selected portions of the cost workbook.

Table 1. Edits to the cost workbook for the abbreviated cost study protocol


Cost Workbook Tab

Summary of Changes for TA Call Option

Rationale

  1. Instructions 

No changes

 

  1. Definitions 

No changes

 

  1. Teaching vacancies 

Highest priorities:  
C1. Center time spent on teaching vacancies 
Moderate priorities: 
C2. Differences in time by assistant/aide 

C3. Differences in time by child age 

C5. Expenses (rather than time) 

Lowest priorities: 

C4. Activities borne by larger organizations 

C6. Differences in expenses by assistant/aide

C7. Differences in expenses by child age 

C8. Expenses borne by larger organizations 

C9. Explanatory note 

Research team staff will be instructed to fill in the sections on center time spent on teaching vacancies and to ask centers if they have any additional expenses spent on teaching vacancies.  
 
The differences in time by role and age are interesting sub analyses but are not essential to answering the proposed research questions. 
 
In most cases, the expenses borne by larger organization questions have not been relevant to the centers in the sample who have completed the cost workbook so far.
 
The explanatory note is no longer necessary given the transition from written to collaborative completion with a research team member.   

  1. Recruitment and hiring 

Highest priorities:  
D1. Center time spent on recruitment and hiring activities 
Moderate priorities: 
D2. Differences in time by assistant/aide 

D3. Differences in time by child age 

D5. Expenses (rather than time) 

Lowest priorities: 

D4. Activities borne by larger organizations 

D6. Differences in expenses by assistant/aide

D7. Differences in expenses by child age

D8. Expenses borne by larger organizations 

D9. Explanatory note 

Same as row C.  

  1. Training and onboarding 

Highest priorities:  
E1. Center time spent on training and onboarding 
Moderate priorities: 
E2. Differences in time by assistant/aide 

E3. Differences in time by child age 

E5. Expenses (rather than time) 

Lowest priorities: 

E4. Activities borne by larger organizations

E6. Differences in expenses by assistant/aide

E7. Differences in expenses by child age

E8. Expenses borne by larger organizations 

E9. Explanatory note 

Same as row C.  

  1. Pilot activities (Treatment centers only)

Highest priority:  
F1. Center time spent completing application materials

F2. Center time spent processing lead and assistant teachers/aides’ opt-in/opt-out decisions

F3. Center time spent on monthly reporting requirements of the pilot initiative

F4. Center time spent on monthly pilot initiative wage supplements

No changes. 

  1. Salaries 

Lowest priority:

G1. Staff salaries

G2. Payroll tax & fringe benefits

G3. Additional benefits

G4. Explanatory note

G5. Changes to salaries & benefits

Because we will use standardized wages and because salary data are available from other data sources being collected as part of the study, this section can be removed to free up time in the call for higher priority questions.

  1. Your Center 

Highest Priority: 
H4. Number of educators that ended their employment since January
H5. Number of weeks to fill vacancies 
 
Moderate Priority: 
H6. Number of educators hired since January

H7. Number of current vacancies 
 

Lowest priority: 

H1. Number of classrooms open since January

H2. Number of educators employed in January

H3. Current number of educators employed today
 

Information on turnover and time to fill positions is sufficient for the cost study, additional data in this section is not essential to answer the proposed research questions and can be down prioritized as needed.  




Overview of Requested Changes for Home-Based Providers

The objective of the study for the home-based providers remains the same as originally proposed-- to describe the experiences of home-based child care owners and their assistants with CDEC’s initiative. Given that the intervention for home-based providers is somewhat different from the center-based initiative, the data collection instruments for home-based providers require some adjustment to align with the intervention. The proposed changes remove questions referring to a salary increase intervention and update question wording and response options slightly to better reflect the nature of the intervention for home-based providers. The proposed modifications also aim to reduce the burden of the survey and the interview for respondents. We are not proposing any adjustments to the data collection approach or approved time period. In addition, we have edited all documents to ensure that the language with respect to roles is consistent with CDEC’s initiative and with study outreach materials: owners will be referred to as “providers” and caregivers will be referred to as “assistants”.


  1. Follow-up home-based provider and assistant survey

The proposed changes to the follow-up surveys include:

  1. removing questions that are not relevant to the current intervention for home-based providers.

  2. replacing instances of “owners” with “providers” and “caregivers” with “assistants” throughout the survey.

Within each survey section that is retained, the goal is to capture the construct of interest (e.g., financial well-being, physical/mental health, etc.) and any potential effects of the intervention with fewer questions. Overall, these proposed modifications reduce estimated burden on survey respondents from the planned 45-minute survey to 25 minutes.


Table 2. Edits to the follow-up home-based provider and assistant survey

Section

Questions removed and details of change

Section A:

About your family child care home

Providers: Removed 11 questions.

Removed questions A4-A6 on home care schedule (pp 2-3).

Removed A10 on existence of staff (p 4), because covered in A11.

Removed A14-A20 (pp 4-5) on language spoken, use of support staff or specialists.

Assistants: No changes. Section not administered to assistants.

Section B: About your experiences with the pilot initiative

Providers: Removed entire section

Given the final structure of the intervention, this section (pp 7-8) is no longer relevant to home-based providers.

Assistants: Removed entire section

Given the final structure of the intervention, this section (pp 7-8) is no longer relevant to home-based providers.

Section C:

About your financial situation and job


Providers: Removed entire section.

Removed entire section (pp 8-18), given that questions about role, wages, hours, and benefits offered are not relevant to providers.

Assistants: Removed 32 questions.

Removed questions C1-C5 (pp 8-9) on current employment status, C8 (p 10) on hours worked, C11 (p 10) on schedule, and C17-C42 (pp 12-18) on characteristics of other jobs.

Section D:

About children you provide care for

Providers: No changes

Assistants: No changes. Section not administered to assistants.

Section E:

About your professional background

Providers: Removed three questions

Removed questions E1 (p 21) and E4-E5 (p 21) about search for a new job.

Assistants: Removed three questions

Removed questions E3-E5 (p 21) about running a child care home and job search.

Section F:

Future job plans




Providers: Removed two questions

Removed questions F7-F8 (p 23) about professional associations and current enrollment in child care related coursework.

Assistants: Removed two questions

Removed questions F7-F8 (p 23) about professional associations and current enrollment in child care related coursework.

Section G:

About your job demands and supports

Providers: Removed six questions

Removed G1-G3 (pp 23-24) on job tasks and support from other adults and G5-G7 (pp 24-25) on licensing tasks.

Assistants: Removed entire section

Removed all questions (pp 23-25) on job tasks and support from other adults and licensing tasks.

Section H:

Feelings about your job

Providers: Removed 12 questions

Removed H2-H4 (p 25) related to job satisfaction, and H6-H14 (pp 26-28) on job self-efficacy and discrimination.

Assistants: Removed eight questions

Removed H2-H4 (p 25) related to job satisfaction, H6-H10 (pp 26-27) on job self-efficacy and discrimination, and H13-H14 (pp 27-28) on receiving a raise.

Section I:

About your financial situation

Providers: Removed four questions

Removed I10-I13 (p 30) capturing student loans, experiences with eviction, not paying full utilities, perception of adequate resources at the end of the month, and perception of financial well-being.

Assistants: Removed four questions

Removed I10-I13 (p 30), capturing student loans, experiences with eviction, not paying full utilities, perception of adequate resources at the end of the month, and perception of financial well-being.

Section J:

Your health and wellbeing

Providers: Removed two questions

Removed J3 and J4 (p 31) about anxiety and job stress.

Assistants: No change

Section K:

A little more about you

Providers: No change

Assistants: No change



  1. One-on-one home-based provider and assistant interview

The proposed changes to the provider and assistant interview include:

  1. removing questions that are not relevant to the current intervention for home-based providers

  2. refining questions to better align with the structure of the final intervention.

Within each Section that is retained, the goal is to capture the construct of interest (e.g., changes due to the pilot, financial well-being, psychological wellbeing/health, etc.) and any potential effects of the intervention. In addition, to better align with the final intervention structure and reduce burden, the interview will only be administered to providers and not to assistants. Overall, these proposed modifications reduced estimated burden on respondents from the planned 90 minutes to 60 minutes. The changes are described in more detail in Table 3.

Table 3. Edits to the one-on-one home-based provider interview protocol

Section

Questions removed and details of change

N/A – throughout

Removed references and sections specific to “all respondents,” “assistant providers” or “those who did not complete an application” because we are now only interviewing providers that are in the Compensation Pilot.

Section 1: Initial exposure/ understanding of pilot initiative

Removed all four questions from Section 1 except for one question about how respondent initially thought and felt about the pilot initiative.


Section 2: Experiences with decision-making (Providers who did not complete the full application)

Removed all questions because decision-making for submitting an application to the pilot is no longer relevant to the intervention.


Section 3: Experiences with decision-making & application process (Providers who completed the application (treatment)

Removed all items because decision-making and process for submitting an application to the pilot is no longer relevant to the intervention.

Section 4: Experiences after initiative started (Providers in treatment group)

Updated questions to be in line with the design of the pilot initiative.

Moved entire section (now Section 1 called Experiences after Pilot Started).

Section 5: Experiences on the job (all respondents in treatment group)

Removed questions about how the pilot affected the climate and relationships with others on the job; and the care and support provided for children and families because these are not relevant to the intervention.

Section 6: Experiences with business finances (providers in treatment)

Removed questions about specific operational costs that are no longer relevant in the final intervention (e.g., licensing, marketing, equipment costs).

Section 7: Experiences with personal finances (all respondents in treatment group)

Removed questions about sources of income and public assistance that are no longer relevant to the final intervention.

Section 8: Beliefs, expectations, and looking ahead (all respondents in treatment group)

Removed questions about specific uses of funds from Pilot that are no longer relevant to the final intervention (i.e., they are not allowable uses of the funds.)



Section

Questions revised and details of change

N/A – throughout

Removed “and caregiver” from instrument title.

Replaced/updated all references to “[PILOT INITIATIVE]” or “Initiative” to say “Compensation Pilot” or “Pilot” to be consistent with the final intervention.

Introductions

Revised wording to incorporate names of providers and family child care homes.

Revised wording since we are interviewing providers only.

Updated “date pilot started” to be “July 2023.”

Streamlined questioning and reordered some questions to create better conversation flow.

Section 1:

Initial exposure/ understanding of pilot initiative

Moved questions from previous Section 4 to revised Section 1. Updated Section 1 title is now “Experiences after Pilot Started”.

Added language to clarify use of term “Compensation Pilot”.

Moved question about experiencing any other consequences of the pilot from previous Section 5 to revised Section 1.

Section 2: Experiences with decision-making (Providers who did not complete the full application)

Moved questions from previous Section 6 to revised Section 2. Updated Section 2 title is now “Experiences with Business Operations”.

Moved two items about business challenges from previous Section 6 (i.e., worry might close, have enough families enrolled) to new Section 2 (pp 13-14).

Updated questions about business finances to probe specifically about how they approach time off, how they fill spots, and what other potential sources of revenue they may have, since these aspects of operations may be affected by the final intervention (i.e., initiative may free-up resources that would have been spent on benefits for other activities.

Added probe about whether they incurred any expenses related to the Pilot.

Updated language in some places to better align with pilot materials. (i.e., financial health and stability)

Section 3: Experiences with decision-making & application process (Providers who completed the application (treatment))

Moved questions from previous Section 5 to revised Section 3. Updated Section 3 title is now “Experiences on the Job”.

Updated probes about how pilot affected respondent and staff wellbeing to specifically include “economic or financial wellbeing”.

Moved 2 questions about how pilot might change staffing or staff benefits from previous Section 8.

Section 4: Experiences after initiative started (Providers in treatment group)

Moved questions from previous Section 7 to revised Section 4. Updated Section 4 title is now “Experiences with Personal Finances”.

Updated language in questions about respondent’s personal and family financial circumstances to better align with intervention (e.g., specifying changes in insurance benefits or job situation).

Updated questions to better contrast personal finances before pilot started with after starting pilot.

Section 5: Experiences on the job (all respondents in treatment group)

Moved questions from previous Section 8 to revised Section 5. Updated Section 5 title is now “Beliefs, expectations, and looking ahead”.



Page 1 of 6


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitleMathematica Memo
Subjectmemo
AuthorPatricia Del Grosso
File Modified0000-00-00
File Created2024-07-20

© 2025 OMB.report | Privacy Policy