Supporting Statement A - SCOPE Follow up

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Study of Coaching Practices in Early Care and Education Settings 2021: Follow-up

OMB Information Collection Request

0970 - 0515

Supporting Statement

Part A

April 2021

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officer:

Wendy DeCourcey

Part A

Executive Summary

• Type of Request: This Information Collection Request is for a reinstatement with change. We are requesting 1 year of approval.

• Description of Request: This is a primary data collection request for the Study of Coaching Practices in Early Care and Education 2021: Follow-up (SCOPE 2021: Follow-up). This descriptive study aims to examine, using web-based surveys and qualitative interviews, the extent to which coaching, and professional development more broadly, have supported early care and education (ECE) settings in providing care for children and families as the COVID-19 pandemic has progressed. The study will focus on both ECE centers and family child care homes that serve low income children, with a primary target of settings that serve children supported by Child Care and Development Fund subsidies or a Head Start grant. The sample frame will be comprised of respondents to the Study of Coaching Practices in Early Care and Education Settings 2019 (SCOPE 2019) web-based surveys (OMB# 0970-0515).

We do not intend for this information to be used as the principal basis for public policy decisions.

A1. Necessity for Collection

Coaching has emerged as one of the most common approaches to professional development in early care and education (ECE) because of its potential to improve teachers’ and caregivers’ classroom practices and provide quality services to children and families. The COVID-19 pandemic has created barriers and necessitated changes to the usual practice of coaching, and likely changed the way that ECE professionals engage in professional development more generally. Notably, there is likely much more use of remote or virtual strategies. However, little is known about the depth or breadth of these changes. This information collection is necessary to inform the ECE field about how to successfully support ECE teachers’ and caregivers’ ability to provide high quality services to children with the support of coaching and professional development during a time of change and crisis, and to provide lessons about what changes would be beneficial to sustain after the crisis ends.

A2. Purpose

Purpose and Use

In 2019, the Study of Coaching Practices in Early Care and Education (SCOPE; OMB #0970-0515, approved September 18, 2018; referred to as SCOPE 2019 throughout the supporting statements) conducted web-based surveys with coaches, center directors, teachers, and family child care providers to examine and understand the structure and process of coaching in ECE settings. Appendix A includes findings from the SCOPE 2019 web-based surveys. We propose a follow-up data collection effort (SCOPE 2021: Follow-up) using web-based surveys and qualitative interviews to follow up with coaches, center directors, and family child care (FCC) providers who participated in the 2019 web-based surveys. The purpose of this descriptive research is to understand the practice and processes of coaching—and professional development more broadly—in supporting ECE settings throughout the COVID-19 pandemic, and how those practices and processes have been adapted or changed in the context of the COVID-19 pandemic. There will be a particular focus on understanding the use of remote versus in-person strategies for coaching and professional development.

The information from the web-based surveys and qualitative interviews will be used to help ACF and the ECE field more broadly understand whether coaching and professional development activities have been sustained through and influenced by the COVID-19 pandemic and whether there are changes to coaching and professional development that could be beneficial to maintain after the pandemic. Ultimately, these results are intended to provide lessons to support ACF in its efforts to improve the quality of care for young children in community-based child care, Head Start settings, and FCC homes. The information collected is meant to contribute to the body of knowledge on ACF programs. It is not intended to be used as the principal basis for a decision by a federal decision-maker, and is not expected to meet the threshold of influential or highly influential scientific information.

Research Questions or Tests

This project has four research questions:

1. What features of coaching are evident during the COVID-19 pandemic? What has changed or remained the same since COVID-19 emerged?

2. What is the role of coaches during the pandemic and how have they been supported?

3. What has been the role of coaching, and professional development more broadly, in supporting ECE settings during the COVID-19 pandemic? As the pandemic has progressed?

4. What is the perceived value and role of coaching, professional development, and quality improvement more generally among ECE coaches, directors, and FCC providers during the COVID-19 pandemic? As the pandemic has progressed?

Study Design

SCOPE 2021: Follow-up will conduct one-time web-based surveys with coaches (100), center directors (66), and FCC providers (38) who responded to the SCOPE 2019 web-based surveys and one-time qualitative interviews with a subset of those who respond to the SCOPE 2021: Follow-up web-based surveys included in this information collection request. For center directors only, if we determine there is a high rate of turnover (that is, the 2019 respondent has left their position but the center is still open and operating), we will aim to recruit the new center director to do the 2021 web-based survey if they have been in their position for at least four months. More details about the recruitment of respondents for SCOPE 2021: Follow-up are available in Section B2 of Part B under Respondent Recruitment and Site Selection. Data collection for SCOPE 2019 is complete; data collection for SCOPE 2021: Follow-up will take place during spring 2021, following OMB approval. The SCOPE 2019 web-based surveys targeted ECE coaches, center directors, teachers, and FCC providers who worked in or with ECE centers and FCC homes that served low-income preschool-age children, with a primary focus on settings that serve children with Child Care and Development Fund (CCDF) subsidies or a Head Start grant.¹ To identify respondents for the SCOPE 2019 web-based surveys, we used a purposive sampling approach to ensure variation in the state policy context, the ECE setting type, and setting funding sources, as well as funders and providers of coaching and features of the coaching they provide. This purposive approach ensured we had sufficient variation in coaching approaches to answer research questions about how coaching features were implemented and combined, and to develop lessons learned relevant to a range of ECE settings serving low-income, preschool-age children. This purposive approach will provide the same benefit to the current information collection (that is, lessons learned will be applicable to a wide range of ECE settings). In addition, by recruiting the same respondents we are uniquely positioned to examine changes to ECE coaching throughout the pandemic.

The SCOPE 2021: Follow-up qualitative interviews will provide important context for interpretation of web-based survey data given the complexity the COVID-19 pandemic has introduced to the ECE field and provide additional lessons learned to fully understand various coaching and professional development approaches currently in use. The study’s key potential limitation is that our respondents might not ultimately include the full range of approaches currently in use and, thus, might not entirely address the information needs of the range of stakeholders aiming to improve the quality of ECE through coaching and professional development. More details about the rationale of our study deign are available in Section B1 of Part B under Appropriateness of Study. More details about the purposive selection criteria used to identify the SCOPE 2019 participants are described in previously approved Supporting Statements (OMB #0970–0515).² More details about the recruitment of respondents for SCOPE 2021: Follow-up are available in Section B2 of Part B under Respondent Recruitment and Site Selection.

Table A.1. Data collection activities

Data Collection Activity	Instruments	Respondent, Content, Purpose of Collection	Mode and Duration
Web-based survey data collection	Coach Survey (Instrument 1) Center Director Survey (Instrument 2) FCC Provider Survey (Instrument 3)	Respondents: ECE coaches, center directors, and FCC providers who completed a SCOPE 2019 web-based survey. Content: Screener questions to confirm eligibility, questions on coaching workload and support, questions on professional development, coaching activities, coach communication, and coaching challenges at the centers and FCC homes. Purpose: To understand how coaching has adapted in the context of the COVID-19 pandemic and how coaching and professional development have supported early care and education settings throughout the pandemic.	Mode: Web-based survey Duration: 20 minutes for each instrument
Qualitative interview data collection	Coach Interview (Instrument 4) Center Director Interview (Instrument 5) FCC Provider Interview (Instrument 6)	Respondents: A subset of ECE coaches, center directors, and FCC providers who complete the SCOPE 2021: Follow-up web-based surveys, are participating in coaching and/or professional development and (for the center directors and FCCs) are in settings providing services to families. Content: Interview questions to understand the context of coaching, communication with coaches, support for professional development, challenges to providing coaching and challenges to providing quality care. Purpose: To understand how coaching has adapted in the context of the COVID-19 pandemic and how coaching and professional development have supported early care and education settings throughout the pandemic.	Mode: Telephone interview Duration: 45 minutes for each interview interview (a shorter version will be available for respondents who cannot commit to 45 minutes; that version is expected to take about 30 minutes and questions will be prioritized based on survey responses)

Other Data Sources and Uses of Information

We will draw on data from SCOPE 2019 to help answer the SCOPE 2021: Follow-up research questions. Specifically, we will compare responses about how coaching was implemented before the pandemic to responses about how coaching is being implemented as the pandemic has progressed. We will also draw on information from expert consultation and stakeholder engagement (fewer than 10 individuals).

A3. Use of Information Technology to Reduce Burden

The data collection plan is designed to obtain information in an efficient way that minimizes respondent burden. When feasible, we will use information from SCOPE 2019.

We will ask ECE coaches, center directors, and FCC providers to complete a web-based survey. The web-based surveys will enable respondents to complete the data collection instrument at a location and time of their choice, and its built-in editing checks and programmed skip patterns will reduce the level of response errors.

We will conduct qualitative interviews by telephone. After we obtain permission from each participant, we will record all interviews to ensure that we capture information accurately at one time point.

A4. Use of Existing Data: Efforts to reduce duplication, minimize burden, and increase utility and government efficiency

We have not identified any other current or planned efforts to collect information on how coaching, and professional development more broadly, have been sustained or changed during the COVID-19 pandemic in ECE settings.

None of the study instruments ask for information that can be obtained from alternative data sources. We will use previously collected information from SCOPE 2019 to examine how coaching has changed over time.

A5. Impact on Small Businesses

Most of the ECE centers and all of the FCC homes included in the study will be small organizations, including community-based organizations and other nonprofits. We will minimize burden for respondents by offering a web-based survey that respondents can complete at their convenience and by restricting the length of the web-based survey. We will schedule the phone interviews at times that are convenient for the respondents.

A6. Consequences of Less Frequent Collection

This is a one-time data collection.

A7. Now subsumed under 2(b) above and 10 (below)

A8. Consultation

Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and Office of Management and Budget (OMB) regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to request an OMB review of this information collection activity. This notice was published on December 21, 2020, Volume 85, Number 245, pages 83090-83091, and provided a sixty-day period for public comment. During the notice and comment period, no comments were received.

Consultation with Experts Outside of the Study

We consulted with experts to complement the knowledge and experience of the team (Table A.2). Consultants included researchers with expertise in the systems through which ECE coaching and professional development are provided to both center-based settings and FCC homes, and in the actual provision of the coaching. Consultants provided input on the study’s goals, topics that would be of value to the study, and consideration for data collection. In addition, two members of the study team (April Crawford and Susan Landry) who are coaching and professional development experts/practitioners were consulted on these topics and reviewed the data collection instruments.

Table A.2. Expert advisors

Name	Affiliation
Juliet Bromer	Herr Research Center for Children and Social Policy, Erikson Institute
Sherri Drake	Teachstone
Bridget Hamre	Teachstone and University of Virginia’s Center for Advanced Study of Teaching and Learning
Debi Mathias	BUILD Initiative
Chris Sciarrino	The Early Learning Lab
Rebecca Shearer	Department of Psychology, University of Miami
April Crawford	Children’s Learning Institute (CLI) at the University of Texas Health Science Center at Houston
Susan Landry	Children’s Learning Institute (CLI) at the University of Texas Health Science Center at Houston

A9. Tokens of Appreciation

There are no tokens of appreciation proposed for respondents in this data collection.

A10. Privacy: Procedures to protect privacy of information, while maximizing data sharing

Personally Identifiable Information

As part of the web-based surveys we are collecting preferred contact information from study respondents in order to send an honorarium (discussed in Section A.13 on costs) and to support recruitment of participans for the qualitative interviews. Information will not be maintained in a paper or electronic system from which data are actually or directly retrieved by an individuals’ personal identifier.

Assurances of Privacy

Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As specified in the contract, the Contractor will comply with all Federal and Departmental regulations for private information. The study will seek IRB approval before data collection begins.

Data Security and Monitoring

As specified in the contract, the Contractor shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The Contractor has developed a Data Safety and Monitoring Plan that assesses all protections of respondents’ personally identifiable information (PII). The Contractor shall ensure that all of its employees, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy issues and comply with the above requirements. 

As specified in the evaluator’s contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of PII during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Processing Standard.  The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system; establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process PII. Any data stored electronically will be secured in accordance with National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, the Contractor must submit a plan for minimizing to the extent possible the inclusion of PII on paper records and for the protection of any paper records, field notes, or other documents that contain PII that ensures secure storage and limits on access.  

A11. Sensitive Information ³

No sensitive information collected.

A12. Burden

Explanation of Burden Estimates

Table A.3 provides an estimate of time burden for the data collections, broken down by instrument and respondent. These estimates are based on our experience collecting information with these types of staff. Data collection will take place over about three months. We expect the total annual burden to be 104 hours.

All data collection using all previously approved materials under OMB #0970-0515 is complete.

Estimated Annualized Cost to Respondents

The study team based average hourly wage estimates for deriving total annual costs on data from the Bureau of Labor Statistics, Occupational Employment Statistics (2020). For each instrument included in Table A.3, the team calculated the total annual cost by multiplying the annual burden hours by the average hourly wage.

The mean hourly wage of $33.26 for instructional coordinators (occupational code 25-9031) is used for coaches. The mean hourly wage of $25.81 for education administrators of preschool and child care centers or programs (occupational code 11-9031) is used for ECE center directors. The mean hourly wage for childcare workers (occupational code 39-9011) of $12.27 is used for FCC providers. Tables from which these wages were drawn are available at the following links:

• Coaches: https://www.bls.gov/oes/current/oes259031.htm

• Center director: https://www.bls.gov/oes/current/oes119031.htm

• FCC providers: https://www.bls.gov/oes/current/oes399011.htm

Table A.3. Estimated annualized burden and cost to respondents

Instrument	No. of Respondents (total over request period)	No. of Responses per Respondent (total over request period)	Avg. Burden per Response (in hours)	Annual Burden (in hours)	Average Hourly Wage	Annual Respondent Cost
Coach Survey (Instrument 1)	100	1	.33	33	$33.26	$1,097.58
Center Director Survey (Instrument 2)	66	1	.33	22	$25.81	$567.82
FCC Provider Survey (Instrument 3)	38	1	.33	13	$12.27	$159.51
Coach Interview (Instrument 4)	12	1	.75	9	$33.26	$299.34
Center Director Interview (Instrument 5)	24	1	.75	18	$25.81	$464.58
FCC Provider Interview (Instrument 6)	12	1	.75	9	$12.27	$110.43
Totals:				104		$2,699.26

A13. Costs

We propose to offer respondents to the web-based surveys and qualitative interviews an honorarium to acknowledge their contribution to a timely and complete data collection. Across all respondents, we expect that participating in the web-based surveys and qualitative interviews will disrupt their schedules.

We will offer a $20 honorarium to each respondent for completing a web-based survey and a $20 honorarium to each respondent participating in an interview in recognition of the time and expertise provided to the information collection. The respondents we aim to recruit are uniquely positioned to answer the study’s research questions as they all participated in the SCOPE 2019 web-based surveys, so we can compare the information they share in the current information collection with information shared in the SCOPE 2019 web-based surveys. This will allow us to answer questions about how coaching has changed as the COVID-19 pandemic has progressed.

To develop honoraria amounts, we considered the length of the data collection activities, and the potential disruption to the schedules of the targeted respondents from participation. Respondents to the SCOPE 2019 web-based surveys also received an honorarium. We also received approval to give an honorarium for participation in qualitative interviews as part of in-person case studies for SCOPE 2019 (OMB #0970-0515), but those case studies had to be suspended due to COVID-19.

A14. Estimated Annualized Costs to the Federal Government

Estimated costs to the Federal government are based on the costs per hour (including overhead) for contractor staff expected to carry out the information collection and the total hours each staff person is expected to contribute to the project. In addition to labor-related costs, costs include operational expenses (such as equipment, overhead, printing, and staff support), web-hosting fees, a dedicated toll-free number and email for the study, IRB fees, honoraria for respondents, and any other expenses which would not have been incurred without this collection of information.

Table A.4. Estimated annualized costs

Cost Category	Estimated Costs
Instrument Development and OMB Clearance	$111,095
Field Work	$178,026
Analysis	$107,470
Publications/Dissemination/Data Archiving	$143,172
Total costs over the request period	$539,764

A15. Reasons for changes in burden

This request is for new data collection activities (web-based surveys and qualitative interviews) with respondents who previously completed SCOPE 2019 web-based surveys. All data collection using previously approved materials is complete. The burden estimate reflects these new data collection activities.

A16. Timeline

Table A.5 presents the timeline for SCOPE 2021: Follow-up activities. Data collection activities will begin immediately upon OMB approval, and extend for up to four months. We aim to complete all data collection activities during spring 2021, as many ECE settings close down for the summer (consistent with a typical school year). Analysis will begin upon completion of each data collection activity (four to five months following OMB approval), and dissemination seven months following OMB approval. We plan to submit study data to the Child & Family Data Archive or a similar data archive at the end of the study so it can be used by other researchers.

Table A.5. SCOPE 2021: Follow-up timeline

Activity	Timing
Recruitment and data collection
Web-based survey recruitment and data collection	To begin immediately after OMB’s approval and extend for 2 to 3 months
Qualitative interview recruitment and data collection	To begin approximately one month after OMB’s approval and extend for 3 months
Analysis
Web-based survey analysis	To begin 4 months after OMB’s approval and extend for 2 to 3 months
Qualitative interview analysis	To begin 5 months after OMB’s approval and extend for 2 to 3 months
Dissemination
Reports and/or briefs	To begin 7 months after OMB’s approval and extend for 5 months
Briefings and/or presentations	To begin 9 months after OMB’s approval and extend for 2 months

A17. Exceptions

No exceptions are necessary for this information collection.

References:

Bureau of Labor Statistics, U.S. Department of Labor. “Occupational Employment Statistics.” [www.bls.gov/oes/]. March 2020.

Attachments

Instruments:

Instrument 1: Coach survey

Instrument 2: Center director survey

Instrument 3: Family child care provider survey

Instrument 4: Coach interview protocol

Instrument 5: Center director interview protocol

Instrument 6: Family child care provider interview protocol

Appendices:

Appendix A1: Findings from the SCOPE 2019 web-based surveys – summary of findings

Appendix A2: Findings from the SCOPE 2019 web-based surveys – NRCEC presentation of findings

Appendix B1: Survey recruitment materials

Appendix B2: Interview recruitment materials

Appendix C: Survey item by research question crosswalk

Appendix D: Interview item by research question crosswalk

1 The approved OMB package for SCOPE 2019 also included 12 in-person site visits that included qualitative interviews with center directors, center teachers, FCC providers, coaches, and coach supervisors as well as an observation of a coaching session in each site; these site visits were scheduled to occur when the COVID-19 pandemic began and therefore had to be suspended.

2 Materials related to SCOPE 2019 are accessible at: https://www.reginfo.gov/public/do/PRAViewICR?ref_nbr=201803-0970-005

3 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.

13