SSB - HomeEc Early Childhood Home Visiting Special Topics Substudy Formative GenIC

Alternative Supporting Statement for Information Collections Designed for

Research, Public Health Surveillance, and Program Evaluation Purposes

Supporting Family Economic Well-Being through Home Visiting (HomeEc) – Early Childhood Home Visiting Special Topics Substudy

Formative Data Collections for ACF Research

0970 - 0356

Supporting Statement

Part B

December 2022

Submitted By:

Office of Planning, Research, and Evaluation

Administration for Children and Families

U.S. Department of Health and Human Services

4^th Floor, Mary E. Switzer Building

330 C Street, SW

Washington, D.C. 20201

Project Officers:

Pooja Gupta Curtain

Part B

B1. Objectives

Study Objectives

The Supporting Family Economic Well-Being through Home Visiting (HomeEc) – Early Childhood Home Visiting Special Topics Substudy, overseen by the Administration for Children and Families (ACF) Office of Planning, Research, and Evaluation (OPRE), is designed to better understand the influence of the COVID-19 pandemic on promoting family economic well-being services in home visiting. The study includes a literature review, a brief web-based survey, and discussions with one representative from up to nine early childhood home visiting (ECHV) programs.

This proposed information collection intended to inform the substudy by collecting background information from ECHV programs through a web-based survey to identify nine programs that best meet the study’s eligibility and prioritization criteria to participate in the targeted discussions. This includes understanding how the programs are supporting family economic well-being and the communities they serve.

Generalizability of Results

The study is intended to present an internally valid description of the influence of the COVID-19 pandemic on promoting family economic well-being services in ECHV, but not to promote statistical generalization to other sites or service populations.

Appropriateness of Study Design and Methods for Planned Uses

The study’s sampling and quantitative methods are appropriate for identifying and collecting basic information from ECHV programs about their family economic well-being services during the COVID-19 pandemic. The data from the short web-based survey will inform the selection of up to nine programs to participate follow up discussions to better understand how ECHV programs have operated and adapted during the COVID-19 pandemic to address the economic needs of the families they serve. The information from the discussions will expand the study team’s initial findings from the literature review and fill in any gaps in answering the study’s research questions.

As noted in Supporting Statement A, this information is not intended to be used as the principal basis for public policy decisions and is not expected to meet the threshold of influential or highly influential scientific information.  

B2. Methods and Design

Target Population

The study team will solicit participation in the survey through the Home Visiting Applied Research Collaborative (HARC) Practice Based Research Network (PBRN) project leadership contacts. HARC is a practice-based research network for conducting research with home visiting programs. Programs that provide home visiting for expectant families and/or families with children birth to five years are able to join, as are directors of local networks and state networks, and researchers who have an interest in home visiting research. HARC membership includes hundreds of home visiting programs and networks, as well as more than 200 researchers. To achieve the study’s objectives, we will collect survey data from respondents from up to 60 ECHV programs. Interested programs will volunteer to participate and complete the survey. The programs will not be representative of the population of early childhood home visiting programs.

Sampling

Following Office of Management and Budget (OMB) approval, the study team will send an email (using Appendix A) through the HARC PRBN listserv to introduce the study and ask representatives from interested programs to complete a brief web-based survey. Up to 60 potential programs that currently implement family economic well-being components in their home visiting programs and who may be interested in participating in the substudy are expected to complete the web-based survey. This estimate is based on the study team’s knowledge of the HARC network’s reach and responses to previous data collection requests. However, we will accept responses from more than 60 ECHV programs if they came through during the data collection window and our burden estimate in Table A2 accounts for up to 80 respondents.

B3. Design of Data Collection Instruments

Development of Data Collection Instrument

We developed a web-based survey (Instrument 1) to collect this formative information for the HomeEc – Early Childhood Home Visiting Special Topics Substudy. The web-based survey only includes questions related to the selection and eligibility criteria so the study team can assess a program’s eligibility for participating in a follow-up qualitative data collection. Table A.1 in Supporting Statement A provides details about the respondent, content, purpose and mode.

The questions in the web-based survey are new because they reflect a new area of research, and the constructs under study cannot be measured using existing instruments. The web-based survey was developed by content experts at Mathematica and its subcontractor, JBA, and informed by the key information the study team would need to select the programs best suited to participate in the qualitative data collection.

B4. Collection of Data and Quality Control

The study team will collect all data through a web-based survey programmed in the platform QuestionPro. Section B3 describes the development of the recruitment protocols. To facilitate buy-in, we have prepared an informative recruitment email for program directors (Appendix A), which describes the study, includes a link to the web-based survey, and provides the estimated amount of time required to complete the web-based survey. This email will be disseminated through the HARC listserv.

Throughout the recruitment and data collection periods, the study team will conduct regular meetings to exchange information and strategies and troubleshoot challenges. We will monitor the survey response rate regularly and conduct regular quality assurance checks on the data. If HARC permits multiple emails about the substudy, over the course of the data collection period, we will send out periodic reminder emails to complete the survey through the listserv (Appendix B).

B5. Response Rates and Potential Nonresponse Bias

Response Rates

The web-based survey is not designed to produce statistically generalizable findings, and participation will be wholly at the respondent’s discretion. Response rates will not be calculated or reported.

NonResponse

We anticipate that most programs on the HARC listserv will not participate in the survey. However, because we are only seeking responses from programs that think they are eligible to and interested in participating in the qualitative data collection, nonresponse is not problematic. Because participants will not be randomly sampled and findings are not intended to be representative, we will not calculate nonresponse bias.

B6. Production of Estimates and Projections

We will not use the data to generate population estimates, either for internal use or dissemination. This information collection is intended to aid in the selection of up to nine ECHV programs to participate in follow up discussions with nine programs. The discussions are intended to present an in-depth description of how some ECHV programs responded to the COVID-19 pandemic, addressed challenges to family economic well-being, and changed to their approach to supporting family economic well-being during the COVID-19 pandemic. The study is not intended to promote statistical generalization to other programs. We do not plan to make policy decisions based on data that are not representative and will not publish population estimates. Information reported from the substudy will clearly state that results are not meant to be generalizable. No information from this formative data collection will be reported outside of ACF; it is for internal project purposes only.

B7. Data Handling and Analysis

Data Handling

The study team will program the survey into QuestionPro, a web survey platform. The use of a web-based survey will make completion easier for respondents and support efforts to maintain accurate data. The web-based survey will not allow respondents to enter out-of-range or inconsistent responses. Weekly reviews of the web-based survey data will allow us to identify potential errors and follow-up with respondents before the end of data collection.

Data Analysis

The web-based survey included in this OMB package will yield data that we will analyze using both qualitative and quantitative methods. For the questions yielding quantitative data, we will examine descriptive statistics, such as the frequencies, for the overall sample. For the questions yielding qualitative data, we will import the responses into Excel to review them.

Data Use

Once analysis is complete, we will use the data from the web-based survey to select the up to nine programs that supported family economic well-being through the COVID-19 pandemic, serve MIECHV priority populations, and represent variation, both in terms of the regions served and the model type employed.

B8. Contact Persons

The following individuals at ACF and Mathematica are leading the study team:

Pooja Gupta Curtin

Social Science Research Analyst

Office of Planning, Research, and Evaluation

[email protected]

Sarah Avellar

Mathematica

[email protected]

Katie Eddins

Mathematica

[email protected]

Attachments

Instruments

Instrument 1: HomeEc Special Topic Study - Program Eligibility Screener

Appendices

Appendix A: Program Director Recruitment and Survey Invitation Email

Appendix B: Program Director Reminder Email

5