Memorandum
Office of Information and Regulatory Affairs (OIRA)
Office of Management and Budget (OMB)
Office of Planning, Research and Evaluation (OPRE)
Administration for Children and Families (ACF)
Subject: Nonsubstantive Change Request – Build and Sustaining the Child Care and Early Education Workforce Study [OMB # 0970-0615]
This memo requests approval of nonsubstantive changes to the approved information collection for the Building and Sustaining the Child Care and Early Education (BASE) Study [OMB Information Collection Request 0970-0615, approved September 1, 2023].
Background
The BASE project aims to understand factors that affect child care and early education (CCEE) workforce dynamics, including employment recruitment, retention, and advancement, as well as to build the evidence base about strategies that may help to recruit, retain, and advance the CCEE workforce.
The information collection aims to build the evidence base about the implementation, costs, and impacts of strategies aimed at improving the compensation and economic well-being of educators in child care center-based and home-based settings. The BASE project will do so by leveraging two pilot initiatives being implemented by the Colorado Department of Early Childhood (CDEC) that provide additional funding and supports to center-based child care settings to improve teacher compensation and funding for home-based child care settings to support the provision of benefits to owners (referred to as providers hereafter) and staff (referred to as assistants hereafter).
The study includes four substudies:
Impact Study:
Follow-up Surveys (Instrument 1 and 2) of center directors and lead and assistant teachers across research conditions
Descriptive Study:
Follow-up Survey (Instrument 3) of home-based child care providers and assistants across research conditions
Implementation Study:
Interview Protocols (Instruments 4 and 5) with a sample of center directors and lead and assistant teachers in the intervention condition
Interview Protocols (Instrument 6) with a sample of home-based care providers and assistants in the intervention condition
Interview Protocol (Instrument 7) with a sample of key informants in Colorado implementing agencies
Cost Study:
Cost Workbook (Instrument 8) with a sample of center-based directors or administrators across research conditions
Since approval, CDEC finalized the intervention plans and as a result, the study team has identified updates needed to two instruments for the home-based provider data collection (i.e., the follow-up home-based provider and assistant survey and the one-on-one home-based provider and assistant interview). We have also identified some alternative procedures that may improve data collection efforts for the center-based providers (i.e., offering potential respondents the alternative to complete a shortened cost workbook.)
Data collection with center-based providers: In the center-based intervention, collection of program cost information is ongoing, utilizing the approved cost workbook. However, recruitment of center-based directors has proved challenging, given the time commitment required to complete the workbook. As discussed below, the team is proposing to offer directors the option of an abbreviated data collection instrument and procedure that would capture selected portions of the workbook in less time.
Data collection with home-based providers: The survey and interview protocols for the home-based providers, approved by OMB, were developed assuming the intervention for this group would be similar in nature to that for center-based providers. In late 2023, CDEC finalized the intervention for home-based providers. The intervention improves teachers’ compensation, as intended, but is implemented differently to better fit the structure and needs of home-based child care providers and assistants: annual award amounts for two years that can be used to fund medical benefits (health, dental, vision, HSA/FSA), paid time off (substitute teacher coverage or revenue replacement during closures), and retirement contributions.
The purpose of the data collection is the same as originally described in the approved OMB materials, and the methods of data collection are the same. The changes to the data collection instruments are only meant to remove items and refine the wording of some items to better align with the design and implementation of the finalized intervention with home-based providers, including language used in CDEC’s initiative outreach materials, and to reduce burden on home-based providers and assistants and on center-based directors. We believe these revisions to these study protocols will enhance the quality and amount of the data collected to address the government’s questions about the impact and implementation of workforce compensation strategies. We request a response to these proposed changes as soon as possible so that we can continue data collection for the center-based cost study and begin data collection for the home-based provider study.
Overview of Requested Changes for Cost Workbook Data Collection for Center-based Providers
The objective of the cost study is to detail the costs associated with teacher recruitment and turnover when operating a center-based child care setting and delivering services. The cost workbook is designed to collect information about time spent on activities related to educator vacancies, recruitment and hiring, and training new educators, and detailed information about salaries and fringe benefits for staff who work on these activities. The goal is to collect this information from 12 providers. The original approved cost workbook was estimated to take respondents 5 hours to complete, including a 30-minute technical assistance (TA) call with research team staff. The original procedure included research team staff walking through the workbook with respondents during the 30-minute call, checking for respondent understanding of items, and addressing any confusion about the information requested in the workbook. Since beginning data collection, respondents have on occasion opted-out of responding to the cost workbook entirely, citing the lengthy time commitment.
Our team proposes adjusting this data collection process by offering respondents two choices:
Conduct the full workbook and TA call (as originally approved by OMB);
Conduct just the 30-minute TA call with a research team staff member to complete key portions of the workbook together.
We are not proposing a change in the TA call or the workbook itself. We also do not propose a change to the originally estimated burden because we are unsure how many respondents will select the TA call only option. Instead, we are proposing a shortened data collection process using the existing, approved data collection materials and approved TA call—offered as an alternative for respondents. Research team staff members will be trained on which sections of the workbook to prioritize in TA calls, which are listed below in Table 1. Items identified as highest priority will be the primary focus of the TA call, while questions classified as moderate priority will be covered only if appropriate and time allows. Low priority items are unlikely to be collected as part of the abbreviated data collection option, given that the call will only last approximately 30 minutes.
To reflect the different levels of effort between the two options, we propose offering the planned $250 honorarium to respondents who choose to complete the entire cost workbook, including $200 for the time spent completing the workbook and $50 for the time spent completing the TA call. For respondents that opt to complete the TA call only, we propose offering a $50 honorarium for the time spent completing the TA call during which research team staff will assist in the completion of selected portions of the cost workbook.
Table 1. Edits to the cost workbook for the abbreviated cost study protocol
|
Cost Workbook Tab |
Summary of Changes for TA Call Option |
Rationale |
|
No changes |
|
|
|
No changes |
|
|
|
Highest
priorities: C3. Differences in time by child age C5. Expenses (rather than time) Lowest priorities: C4. Activities borne by larger organizations C6. Differences in expenses by assistant/aide C7. Differences in expenses by child age C8. Expenses borne by larger organizations C9. Explanatory note |
Research
team staff will be instructed to fill in the sections on center
time spent on teaching vacancies and to ask centers if they have
any additional expenses spent on teaching vacancies. |
|
|
Highest
priorities: D3. Differences in time by child age D5. Expenses (rather than time) Lowest priorities: D4. Activities borne by larger organizations D6. Differences in expenses by assistant/aide D7. Differences in expenses by child age D8. Expenses borne by larger organizations D9. Explanatory note |
Same as row C. |
|
|
Highest
priorities: E3. Differences in time by child age E5. Expenses (rather than time) Lowest priorities: E4. Activities borne by larger organizations E6. Differences in expenses by assistant/aide E7. Differences in expenses by child age E8. Expenses borne by larger organizations E9. Explanatory note |
Same as row C. |
|
|
Highest
priority: F2. Center time spent processing lead and assistant teachers/aides’ opt-in/opt-out decisions F3. Center time spent on monthly reporting requirements of the pilot initiative F4. Center time spent on monthly pilot initiative wage supplements |
No changes. |
|
|
Lowest priority: G1. Staff salaries G2. Payroll tax & fringe benefits G3. Additional benefits G4. Explanatory note G5. Changes to salaries & benefits |
Because we will use standardized wages and because salary data are available from other data sources being collected as part of the study, this section can be removed to free up time in the call for higher priority questions. |
|
|
Highest
Priority: H7.
Number of current vacancies Lowest priority: H1. Number of classrooms open since January H2. Number of educators employed in January H3.
Current number of educators employed today |
Information on turnover and time to fill positions is sufficient for the cost study, additional data in this section is not essential to answer the proposed research questions and can be down prioritized as needed. |
Overview of Requested Changes for Home-Based Providers
The objective of the study for the home-based providers remains the same as originally proposed-- to describe the experiences of home-based child care owners and their assistants with CDEC’s initiative. Given that the intervention for home-based providers is somewhat different from the center-based initiative, the data collection instruments for home-based providers require some adjustment to align with the intervention. The proposed changes remove questions referring to a salary increase intervention and update question wording and response options slightly to better reflect the nature of the intervention for home-based providers. The proposed modifications also aim to reduce the burden of the survey and the interview for respondents. We are not proposing any adjustments to the data collection approach or approved time period. In addition, we have edited all documents to ensure that the language with respect to roles is consistent with CDEC’s initiative and with study outreach materials: owners will be referred to as “providers” and caregivers will be referred to as “assistants”.
Follow-up home-based provider and assistant survey
The proposed changes to the follow-up surveys include:
removing questions that are not relevant to the current intervention for home-based providers.
replacing instances of “owners” with “providers” and “caregivers” with “assistants” throughout the survey.
Within each survey section that is retained, the goal is to capture the construct of interest (e.g., financial well-being, physical/mental health, etc.) and any potential effects of the intervention with fewer questions. Overall, these proposed modifications reduce estimated burden on survey respondents from the planned 45-minute survey to 25 minutes.
Table 2. Edits to the follow-up home-based provider and assistant survey
Section |
Questions removed and details of change |
Section A: About your family child care home |
Providers: Removed 11 questions. Removed questions A4-A6 on home care schedule (pp 2-3). Removed A10 on existence of staff (p 4), because covered in A11. Removed A14-A20 (pp 4-5) on language spoken, use of support staff or specialists. |
Assistants: No changes. Section not administered to assistants. |
|
Section B: About your experiences with the pilot initiative |
Providers: Removed entire section Given the final structure of the intervention, this section (pp 7-8) is no longer relevant to home-based providers. |
Assistants: Removed entire section Given the final structure of the intervention, this section (pp 7-8) is no longer relevant to home-based providers. |
|
Section C: About your financial situation and job
|
Providers: Removed entire section. Removed entire section (pp 8-18), given that questions about role, wages, hours, and benefits offered are not relevant to providers. |
Assistants: Removed 32 questions. Removed questions C1-C5 (pp 8-9) on current employment status, C8 (p 10) on hours worked, C11 (p 10) on schedule, and C17-C42 (pp 12-18) on characteristics of other jobs. |
|
Section D: About children you provide care for |
Providers: No changes |
Assistants: No changes. Section not administered to assistants. |
|
Section E: About your professional background |
Providers: Removed three questions Removed questions E1 (p 21) and E4-E5 (p 21) about search for a new job. |
Assistants: Removed three questions Removed questions E3-E5 (p 21) about running a child care home and job search. |
|
Section F: Future job plans
|
Providers: Removed two questions Removed questions F7-F8 (p 23) about professional associations and current enrollment in child care related coursework. |
Assistants: Removed two questions Removed questions F7-F8 (p 23) about professional associations and current enrollment in child care related coursework. |
|
Section G: About your job demands and supports |
Providers: Removed six questions Removed G1-G3 (pp 23-24) on job tasks and support from other adults and G5-G7 (pp 24-25) on licensing tasks. |
Assistants: Removed entire section Removed all questions (pp 23-25) on job tasks and support from other adults and licensing tasks. |
|
Section H: Feelings about your job |
Providers: Removed 12 questions Removed H2-H4 (p 25) related to job satisfaction, and H6-H14 (pp 26-28) on job self-efficacy and discrimination. |
Assistants: Removed eight questions Removed H2-H4 (p 25) related to job satisfaction, H6-H10 (pp 26-27) on job self-efficacy and discrimination, and H13-H14 (pp 27-28) on receiving a raise. |
|
Section I: About your financial situation |
Providers: Removed four questions Removed I10-I13 (p 30) capturing student loans, experiences with eviction, not paying full utilities, perception of adequate resources at the end of the month, and perception of financial well-being. |
Assistants: Removed four questions Removed I10-I13 (p 30), capturing student loans, experiences with eviction, not paying full utilities, perception of adequate resources at the end of the month, and perception of financial well-being. |
|
Section J: Your health and wellbeing |
Providers: Removed two questions Removed J3 and J4 (p 31) about anxiety and job stress. |
Assistants: No change |
|
Section K: A little more about you |
Providers: No change |
Assistants: No change |
One-on-one home-based provider and assistant interview
The proposed changes to the provider and assistant interview include:
removing questions that are not relevant to the current intervention for home-based providers
refining questions to better align with the structure of the final intervention.
Within each Section that is retained, the goal is to capture the construct of interest (e.g., changes due to the pilot, financial well-being, psychological wellbeing/health, etc.) and any potential effects of the intervention. In addition, to better align with the final intervention structure and reduce burden, the interview will only be administered to providers and not to assistants. Overall, these proposed modifications reduced estimated burden on respondents from the planned 90 minutes to 60 minutes. The changes are described in more detail in Table 3.
Table 3. Edits to the one-on-one home-based provider interview protocol
Section |
Questions removed and details of change |
N/A – throughout |
Removed references and sections specific to “all respondents,” “assistant providers” or “those who did not complete an application” because we are now only interviewing providers that are in the Compensation Pilot. |
Section 1: Initial exposure/ understanding of pilot initiative |
Removed all four questions from Section 1 except for one question about how respondent initially thought and felt about the pilot initiative.
|
Section 2: Experiences with decision-making (Providers who did not complete the full application) |
Removed all questions because decision-making for submitting an application to the pilot is no longer relevant to the intervention.
|
Section 3: Experiences with decision-making & application process (Providers who completed the application (treatment) |
Removed all items because decision-making and process for submitting an application to the pilot is no longer relevant to the intervention. |
Section 4: Experiences after initiative started (Providers in treatment group) |
Updated questions to be in line with the design of the pilot initiative. Moved entire section (now Section 1 called Experiences after Pilot Started). |
Section 5: Experiences on the job (all respondents in treatment group) |
Removed questions about how the pilot affected the climate and relationships with others on the job; and the care and support provided for children and families because these are not relevant to the intervention. |
Section 6: Experiences with business finances (providers in treatment) |
Removed questions about specific operational costs that are no longer relevant in the final intervention (e.g., licensing, marketing, equipment costs). |
Section 7: Experiences with personal finances (all respondents in treatment group) |
Removed questions about sources of income and public assistance that are no longer relevant to the final intervention. |
Section 8: Beliefs, expectations, and looking ahead (all respondents in treatment group) |
Removed questions about specific uses of funds from Pilot that are no longer relevant to the final intervention (i.e., they are not allowable uses of the funds.) |
Section |
Questions revised and details of change |
N/A – throughout |
Removed “and caregiver” from instrument title. Replaced/updated all references to “[PILOT INITIATIVE]” or “Initiative” to say “Compensation Pilot” or “Pilot” to be consistent with the final intervention. |
Introductions |
Revised wording to incorporate names of providers and family child care homes. Revised wording since we are interviewing providers only. Updated “date pilot started” to be “July 2023.” Streamlined questioning and reordered some questions to create better conversation flow. |
Section 1: Initial exposure/ understanding of pilot initiative |
Moved questions from previous Section 4 to revised Section 1. Updated Section 1 title is now “Experiences after Pilot Started”. Added language to clarify use of term “Compensation Pilot”. Moved question about experiencing any other consequences of the pilot from previous Section 5 to revised Section 1. |
Section 2: Experiences with decision-making (Providers who did not complete the full application) |
Moved questions from previous Section 6 to revised Section 2. Updated Section 2 title is now “Experiences with Business Operations”. Moved two items about business challenges from previous Section 6 (i.e., worry might close, have enough families enrolled) to new Section 2 (pp 13-14). Updated questions about business finances to probe specifically about how they approach time off, how they fill spots, and what other potential sources of revenue they may have, since these aspects of operations may be affected by the final intervention (i.e., initiative may free-up resources that would have been spent on benefits for other activities. Added probe about whether they incurred any expenses related to the Pilot. Updated language in some places to better align with pilot materials. (i.e., financial health and stability) |
Section 3: Experiences with decision-making & application process (Providers who completed the application (treatment)) |
Moved questions from previous Section 5 to revised Section 3. Updated Section 3 title is now “Experiences on the Job”. Updated probes about how pilot affected respondent and staff wellbeing to specifically include “economic or financial wellbeing”. Moved 2 questions about how pilot might change staffing or staff benefits from previous Section 8. |
Section 4: Experiences after initiative started (Providers in treatment group) |
Moved questions from previous Section 7 to revised Section 4. Updated Section 4 title is now “Experiences with Personal Finances”. Updated language in questions about respondent’s personal and family financial circumstances to better align with intervention (e.g., specifying changes in insurance benefits or job situation). Updated questions to better contrast personal finances before pilot started with after starting pilot. |
Section 5: Experiences on the job (all respondents in treatment group) |
Moved questions from previous Section 8 to revised Section 5. Updated Section 5 title is now “Beliefs, expectations, and looking ahead”. |
Page
File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
File Title | Mathematica Memo |
Subject | memo |
Author | Patricia Del Grosso |
File Modified | 0000-00-00 |
File Created | 2024-07-20 |