The Home-Based Child Care Toolkit for Nurturing School-Age Children Pilot Study
Pre-testing of Evaluation Data Collection Activities
0970 – 0355
Supporting Statement
Part A
December 2022
Submitted By:
Office of Planning, Research, and Evaluation
Administration for Children and Families
U.S. Department of Health and Human Services
4th Floor, Mary E. Switzer Building
330 C Street, SW
Washington, D.C. 20201
Project Officers: Ann Rivera and Bonnie Mackintosh
Type of Request: This Information Collection Request (ICR) is for a generic information collection under the umbrella generic, Pre-testing of Evaluation Data Collection Activities (0970-0355).
Description of Request: This current request is for the Home-Based Child Care Toolkit for Nurturing School-Age Children (HBCC-NSAC Toolkit) Pilot Study, which is part of the Home-Based Child Care Supply and Quality (HBCCSQ) project’s measures development. Home-based child care is child care and early education (CCEE) offered in a provider’s or child’s home.
The HBCC-NSAC Toolkit is intended to help home-based child care providers, who care for at least one school-age child, identify strengths and areas for growth in providing care for children and partnering with their families. It consists of a self-administered provider questionnaire and family questionnaire. This is a request to pilot test the HBCC-NSAC Toolkit. The pilot will reach a diverse group of HBCC providers (and the families they care for) who regularly care for at least one school-age child (age 5 and in kindergarten, or ages 6 through 12), meaning they care for the school-age child(ren) at least 10 hours per week. These providers may also care for under school-age children (ages birth through 5 and not yet in kindergarten). Pilot study data are not intended to produce generalizable statistical information; the pilot study’s findings will be used to revise the instruments in the HBCC-NSAC Toolkit and implement any procedural changes prior to the larger validation study (full ICR to be submitted). We do not intend for this information to be used as the principal basis for public policy decisions.
Time Sensitivity: The study team will use the pilot study data to understand the variation in responses and refine the instruments in the HBCC-NSAC Toolkit before the larger validation study planned for fall 2023 (recruitment) through winter/spring 2024 (data collection). A separate ICR will be submitted to the Office of Management and Budget for the validation study. To adhere to the project timeline, the pilot study data collection requested under this generic ICR must be completed by April 2023. This schedule will allow the team to analyze the data and make any necessary revisions to the HBCC-NSAC Toolkit instruments in time to begin sampling and recruitment for the future validation study.
Home-based child care (HBCC), or child care and early education (CCEE) offered in a provider’s or child’s home, is the most common form of nonparental child care in the United States and is essential for many families. Families of color, those from immigrant backgrounds, those with low incomes who work nontraditional hours, and those living in rural areas are more likely to use HBCC than center-based care.1
In launching the Home-Based Child Care Supply and Quality (HBCCSQ) project, the Office of Planning, Research, and Evaluation (OPRE) within the Administration for Children and Families (ACF) aims to fill gaps in the understanding of HBCC supply and address challenges defining and measuring quality in HBCC settings. The HBCCSQ project conducted a literature review and measures review and found that existing measures used in HBCC settings are often based on or designed to parallel measures of center-based CCEE. Existing measures primarily focus on children served before they reach school age even though over half of HBCC providers are estimated to care for school-age children, and many unlisted HBCC providers (who in many cases are unlicensed, and often family, friends, and neighbors) care for one to two school-age children.2 In addition, few measures were developed to account for features that may be more likely to take place in or to be implemented differently in HBCC settings, such as children’s racial and ethnic socialization, cross-age peer interactions, and nontraditional-hour care.3 These features that are missing from existing measures may meet the CCEE needs of families who use HBCC and might support positive outcomes for these children and families.
To fill these gaps, the HBCCSQ project is developing the Home-Based Child Care Toolkit for Nurturing School-Age Children (HBCC-NSAC Toolkit). The instruments in the HBCC-NSAC Toolkit include a provider questionnaire (Instrument 3) and a family questionnaire (Instrument 4) and will be available in English and Spanish. The measures in the provider questionnaire focus on practices that may be more likely to take place in or be implemented differently in HBCC settings, such as racial and ethnic socialization, cross-age interactions (between school-age children and younger children) and similar-age interactions, as well as nontraditional-hour care. The HBCC-NSAC Toolkit is intended for HBCC providers who regularly care for at least one school-age child (age 5 and in kindergarten, or ages 6 through 12), meaning they care for the school-age child(ren) at least 10 hours per week. These providers may also care for younger children (ages birth through 5 and not yet in kindergarten). Its primary purpose is to help HBCC providers who regularly care for at least one school-age child identify and reflect on their strengths and areas of growth related to their caregiving practices and how they partner with children’s families.
To further develop this toolkit, the study team is requesting approval to conduct the Home-Based Child Care Toolkit for Nurturing School-Age Children Pilot Study. The pilot study will include HBCC providers (and the families they care for) who are licensed or regulated by states to provide CCEE, commonly referred to as family child care (FCC) providers, and those who are unlicensed, or legally exempt from state licensing or other state regulations for CCEE, commonly referred to as family, friend, and neighbor (FFN) providers.
This pilot is necessary to test the HBCC-NSAC Toolkit instruments and data collection procedures and materials with a diverse group of HBCC providers and the families they care for before a larger validation study (full Information Collection Request (ICR) to be submitted to the Office of Management and Budget (OMB)).
There are no legal or administrative requirements that necessitate this collection. ACF is undertaking the collection at the discretion of the agency.
Purpose and Use
The purpose of this information collection is to pilot test the HBCC-NSAC Toolkit to examine its psychometric properties (including reliability and validity) and ensure the statements and questions are appropriate for HBCC providers with diverse characteristics and for families who receive care from an HBCC provider. The pilot study will also test the recruitment and data collection procedures and the protocol for completing the HBCC-NSAC Toolkit. The study team will use the pilot data to understand responses to items and refine and improve the quality of the instruments and procedures before using them in a future large-scale validation study (full ICR to be submitted).
This proposed information collection meets the primary goals of ACF’s generic clearance for pre-testing (0970-0355): to develop and test information collection instruments and procedures.
During the initial development of the HBCC-NSAC Toolkit, the study team pre-tested the English version of the HBCC provider questionnaire4. As part of this pilot, we aim to pre-test the Spanish version of the provider questionnaire as well as the English and Spanish versions of the family questionnaire. Additional details about the completed pre-test of the English version of the HBCC provider questionnaire is described in Supporting Statement Part B Section B3. Specifically, this request is for approval to recruit HBCC providers and the families they care for (Instrument 1, Instrument 2, and Instrument 7), test the provider questionnaire (Instrument 3) and family questionnaire (Instrument 4) that make up the HBCC-NSAC Toolkit, and interview a small number of providers (Instrument 5) and families (Instrument 6) about their experiences completing the HBCC-NSAC Toolkit.
Guiding Questions
Phase one of this study will address the following guiding questions:
Are the Spanish version of the provider questionnaire instructions, statements, questions, and response options clearly written, easy to understand, and appropriate for Spanish-speaking providers?
Are the English and Spanish versions of the family questionnaire statements, questions, and response options clearly written, easy to understand, and appropriate for both English- and Spanish-speaking families?
How long does it take respondents to complete the Spanish version of the provider questionnaire?
How long does it take respondents to complete the English and Spanish versions of the family questionnaire?
Phase two of this study will address the following guiding questions:
Are the items in the provider questionnaire appropriate for a diverse population of HBCC providers, including both English and Spanish speakers? Do they capture intended information (proposed factor structure) and demonstrate meaningful variation in responses?
Are there differences in how the provider questionnaire items perform for HBCC providers with different characteristics (for example, for providers with licensed and unlicensed settings, by provider race and ethnicity)?
Are the English and Spanish versions of the family questionnaire statements, questions, and response options clearly written, easy to understand, and have similar meaning across diverse families? Do they capture variation?
Study Design
This pilot study will test the HBCC-NSAC Toolkit in two phases, (1) a pre-test of the Spanish version of the provider questionnaire and the Spanish and English versions of the family questionnaire and (2) a larger test of the full HBCC-NSAC Toolkit (in English and Spanish) and related recruitment and data collection procedures. Phase one is not intended to influence phase two; rather, examination of phase one data may point to potential patterns to look for in the phase two data and help to guide any revisions before the larger validation study (full ICR to be submitted). In both phases, the study team will work with community organizations that offer support and quality improvement opportunities to HBCC providers to recruit providers. Additional details about the respondent selection approach are described in Supporting Statement Part B Section B2.
Phase one. The study team will recruit up to 9 purposively selected Spanish speaking HBCC providers to complete the Spanish version of the provider questionnaire (Instrument 3). Up to 9 Spanish speaking families and up to 9 English speaking families will be purposively selected to complete the family questionnaire (Instrument 4). Additional details about the recruitment approach are described in Supporting Statement Part B Section B2 Respondent Recruitment and Site Selection. Providers will complete the provider questionnaire electronically in a Word document or on paper. Families will complete the family questionnaire on paper (Supporting Statement Part B Section B4). The study team then will conduct a cognitive interview with each provider and family respondent (Instrument 5 and Instrument 6). The recruitment materials for phase one of the pilot study are included in Appendix A: Community organization outreach materials, Appendix B: Phase one provider and family recruitment materials, and Appendix J: Family data collection instructions. Materials to remind respondents about data collection activities are included in Appendix D: Supplemental provider and family recruitment materials.
Phase two. In phase two, the study team will recruit 150 purposively selected providers from diverse backgrounds to complete the provider questionnaire (Instrument 3). Providers will complete the provider questionnaire on the web, on paper, or over the phone, according to their preference. The study team will ask providers to recruit one or more families to complete the family questionnaire on paper, for a total of up to 150 families (Instrument 7). The recruitment materials for phase two of the pilot study appear in Appendix A: Community organization outreach materials, Appendix C: Phase two provider and family recruitment materials, and Appendix J: Family data collection instructions. Materials to remind respondents about data collection activities appear in Appendix D: Supplemental provider and family recruitment materials.
To evaluate the large number of items currently in the provider questionnaire, the study team will use a matrix sampling of dimensions and assign HBCC providers to one of four versions of the provider questionnaire (Instrument 3.2). Supporting Statement Part B Section B1 and Appendix E: HBCC-NSAC Toolkit provider questionnaire matrix sampling provide additional details about this approach.
See Supporting Statement Part B for additional information about the study design, including appropriateness for intended use and the approach to recruitment and data collection.
The scope of this pilot study is limited to 159 HBCC providers who meet the study’s eligibility criteria (see Supporting Statement Part B Section B4) and 168 families of school-age children who receive care from participating HBCC providers. Participation is limited to providers and families who agree to participate and are available to complete the study instruments during winter/spring 2023. To achieve the pilot study purpose, the study team aims to recruit a diverse group of providers to participate in the study, including licensed home-based providers (i.e., FCC) and unlicensed home-based providers (i.e., FFN) with varying characteristics. Analysis of phase two data requires subsamples of sufficient size (N ≥ 30) (see Supporting Statement Part B Section B7). However, it may be difficult to identify and reach FFN providers from a range of backgrounds. So, the sample of FFN providers that are included in the study may be limited to those who engage with the partner organizations that agree to support study recruitment. These differences could, in turn, be associated with more limited or more similar responses (i.e., less variation in the construct of interest). Additionally, the characteristics of families that will be recruited to complete the family questionnaire will be limited to those cared for by providers who choose to participate in the study. Results from the pilot study will be used to provide initial evidence of the psychometric properties of the items and potentially refine the study instrument for use with a larger number of providers and families in a future validation study (full ICR to be submitted). The results are not intended to be representative of, or generalizable to, a given subpopulation.
Table A.1 summarizes the data collection activities including the instruments, their content and respondent types, and the language, mode, and duration of each data collection activity.
Table A.1. Data collection activities
Other Data Sources and Uses of Information
The data from this information collection will be used to improve the quality of the HBCC-NSAC Toolkit for use in the larger validation study. The study team will submit a full ICR requesting approval to complete data collection for the validation study to OMB. No other data sources or information will be used.
The study team will use information technology, including web and phone-based data collection methods to reduce the burden on respondents who agree to participate in the pilot study.
Phase one. Providers can complete the provider questionnaire electronically in a Word document or by paper. Families will complete the family questionnaire by paper due to its short length and a paper questionnaire may be easiest for families to receive from providers and return. Families can place their completed questionnaire in an envelope and seal it before returning it to providers. Cognitive interviews with providers and family members will be conducted and recorded with respondent permission using a secure virtual meeting platform.
Phase two. Providers can complete the provider questionnaire by paper, web, or telephone. Families will complete the family questionnaire on paper and can place it in an envelope and seal the envelope before returning it to providers. The provider questionnaire will be programmed using a secure web survey platform, and the web-based instruments will be optimized so providers can respond on smartphones. Providers may save their progress, allowing completion in more than one session if preferred. Online data collection enables efficient survey participation, as programming limits questions to relevant respondents and can notify respondents about potential data entry errors. For example, based on the provider’s responses to questions about the HBCC setting, the programmed provider questionnaire will be tailored to present only the statements that are most relevant for the provider’s context and setting. Web questionnaires also allow respondents to complete and submit data securely using unique, password-protected logins. Trained telephone interviewers will use the web questionnaire to administer the provider questionnaire to providers who prefer to submit their responses by phone.
None of the study instruments ask for information that can be obtained from alternative data sources. This HBCC-NSAC Toolkit will collect a unique set of data because it was developed by the HBCCSQ project to address gaps in existing measures used in HBCC settings as identified through the project’s measures review.5 Since one of the main goals of this project is measurement development, existing administrative data sets are also not appropriate. The design of the study instruments ensures minimal duplication of data collected across instruments.
Small businesses will likely be affected since a portion of the providers will be licensed and registered as a business. It is unlikely that any of the providers who are exempt from state licensing or regulation will be registered as a business.
The data collection has been designed to allow respondents to complete most instruments at times that are convenient for them. For example, in phase one and two, family members can complete the paper-based family questionnaire at their convenience during the data collection period. Likewise, providers can complete the Spanish-version of the provider questionnaire in phase one at their convenience electronically or on paper. In phase two, the provider questionnaire will be available for completion via computer or smartphone, and it will be programmed to allow completion in more than one session. If providers participating in phase two of the pilot study prefer to complete the provider questionnaire with a trained telephone interviewer, the data collection will be scheduled during hours that are convenient for providers (e.g., when children nap during the day, in the evening, or if needed, on the weekend).
This is a one-time data collection.
Federal Register Notice and Comments
In accordance with the Paperwork Reduction Act of 1995 (Pub. L. 104-13) and OMB regulations at 5 CFR Part 1320 (60 FR 44978, August 29, 1995), ACF published a notice in the Federal Register announcing the agency’s intention to submit a request to OMB for review of the overarching generic clearance for pre-testing activities. This notice was published on January 5, 2021; Volume 86, Number 2, page 308, and provided a sixty-day period for public comment. During the notice and comment period, no substantive comments were received. A second notice in the Federal Register announcing the agency’s submission of the overarching generic clearance for pre-testing activities for OMB’s review. This notice was published on May 21, 2021; Volume 86, Number 97, page 27624, and provided a thirty-day period for public comment. During the notice and comment period, no comments were received.
The study team consulted and will continue to consult with academic experts and experts with lived experience providing HBCC to complement the knowledge and experience of the study team (Table A.2).
The academic experts include researchers with expertise in measurement design and methods, home-based child care quality, and culturally responsive care. The other experts include individuals with lived experience providing home-based care to school-age children. The study team consulted with the academic experts and experts with lived experience providing HBCC listed below to review the draft provider questionnaire. The study team will consult with the listed experts with lived experience providing HBCC on recruitment strategies and communication, and analysis and interpretation of findings. The study team will consult with the academic experts on analysis and interpretation of findings.
Table A.2. The HBCC-NSAC Toolkit consultation with outside experts
Name |
Affiliation or role |
Academic experts |
|
Stephanie Curenton |
Boston University |
Iheoma U. Iruka |
Equity Research Action Coalition at FPG Child Development Institute |
Susan O’Conner |
National Center on Afterschool and Summer Enrichment |
Julie C. Rusbya |
Oregon Research Institute |
Susan Savage |
The Child Care Resource Center |
Eva Marie Shivers |
Indigo Cultural Center |
Experts with lived experience providing HBCC |
|
Decarla Burton |
Family child care provider |
Ruby Daniels |
Family child care provider |
Cynthia Burwell |
Family, friend, and neighbor provider |
aRetiring in January 2023 and will no longer be involved in the pilot study.
Consultation with community organizations that provide supports to HBCC providers
The study team also consulted with community organizations that provide supports (for example, training, technical assistance, or mentoring) to HBCC providers (Table A.3). Staff from the organizations reviewed the draft provider questionnaire and provided feedback. The study team will engage staff from these community organizations for future consultations regarding analysis and interpretation of pilot findings.
Table A.3. The HBCC-NSAC Toolkit consultation with community organizations that provide supports to HBCC providers
Community organization name |
All Our Kin |
Community Resources for Children |
Illinois Action for Children |
Wisconsin Early Childhood Association |
Providing monetary tokens of appreciation for research participants is a commonly used method across studies to show appreciation to research participants as well as to facilitate recruitment and to meet sample size goals.6 The pilot study’s design involves burden to participating providers and families as shown in Table A.5.
As noted in Supporting Statement Part B Section B7, analysis of phase two data requires the overall sample (N ≥ 100) and subsamples of sufficient size (N ≥ 30) per provider questionnaire dimension to obtain reliable estimates of item parameters and ascertain the reliability of the scales (Table A.4). Gaining cooperation from at least these numbers of respondents is needed for analyses to help the study team understand provider perspectives and address any concerns with the HBCC-NSAC Toolkit instruments (Instrument 3 Instrument 4) prior to the larger validation study (full ICR to be submitted). The study team will receive feedback on the HBCC-NSAC Toolkit instruments (Spanish versions of instrument 3 and Instrument 4) during phase one and be able to assess the psychometric properties of the instruments (Instrument 3 and Instrument 4) during phase two, if used with a diverse group of providers and the families they serve (see Supporting Statement Part B Section B2). If providers and families do not complete the instruments in the study at response rates indicated above, the team will not be able to answer the pilot study’s research questions that rely on their responses to the HBCC-NSAC Toolkit instruments.
Table A.4 Subsamples of sufficient size per provider questionnaire dimension
|
English |
Spanish |
Total |
Total |
|||
|
Urban |
Rural |
Urban |
Rural |
Urban |
Rural |
|
FCC sample per form |
8 |
4 |
5 |
2 |
13 |
6 |
19 |
Observations per dimension in FCCs |
16 |
8 |
10 |
4 |
26 |
12 |
38 |
FNN sample per form |
4 |
4 |
2 |
8 |
6 |
12 |
18 |
Observations per dimension in FFNs |
8 |
8 |
4 |
16 |
12 |
20 |
36 |
Spanish total per dimension |
|
|
|
|
14 |
20 |
34 |
English total per dimension |
|
|
|
|
24 |
16 |
40 |
Note: Four versions of the provider questionnaire each with 2 dimensions will be fielded. The dimensions overlap across versions with each dimension in at least 2 versions.
While the data will not be representative of, or generalizable to, any specific population of providers, it is important that the study team secure participation from providers from a range of diverse and under-resourced communities, including those with the most barriers to participation in the pilot study, such as those with low incomes, living in rural areas, or providing care during nontraditional hours, for example, evenings and weekends. Further, providers and families living in under-resourced communities may be hesitant to participate in a government-funded research study due to the inequitable ways that research and policy have often been conducted in communities of color.7 Monetary tokens of appreciation can increase response rates and reduce nonresponse bias,8 and could motivate respondents who are less likely to respond, such as those with increased barriers to participation (including financial barriers). The tokens of appreciation planned for providers and families are intended to respectfully acknowledge respondent’s involvement, particularly those who may be otherwise hesitant to participate in government-funded research. If the study team is not able to reach the providers with the most barriers to participation, the study team may not learn about whether and how these providers understand the items in the provider questionnaire and may not have sufficient sample sizes to conduct the psychometric analyses needed to understand the properties of the provider questionnaire (see Supporting Statement Part B Section B7).
It is also important that the providers secure participation from families. The study team will have a limited number of potential family respondents, making high response rates especially important to ensure different perspectives of families are represented in the pilot study results used to make any necessary revisions to the HBCC-NSAC Toolkit instruments.
Table A.5 lists the planned tokens of appreciation. Phase one tokens of appreciation proposed for providers reflect results from a nationally representative study on the effectiveness of tokens of appreciation for engaging participants in qualitative research, which found that any monetary token of appreciation was more likely to result in willingness to engage in a qualitative interview than a nonmonetary one or none.9 In the same study, among those participants who reported some willingness to engage in a 90-minute qualitative interview, those who were offered $50 or $75 were more willing to engage than those who were offered $25. These results suggest that $50 to $75 for a qualitative interview is a reasonable token of appreciation that is likely to engage participants. Based on these results, the study team plans to offer providers $75 to complete three data collection activities that are expected to take a total of 95 minutes across the three activities. This includes a cognitive interview and two questionnaires. In an effort to encourage providers to support the family data collection activities, we plan to offer an additional $10 token of appreciation to providers who complete the family data collection logistics call (Instrument 7) and agree to invite families to complete the family questionnaire. This additional $10 token of appreciation is intended to improve data quality by helping ensure collection of enough responses to the family questionnaire to conduct study analyses.
The proposed amounts for families are based on burden to families and evidence from previous studies with similar respondents and types of data collection requests. For example, the Head Start Family and Child Experiences Survey (FACES) conducted an experiment for the parent survey in FACES 2014-2018,10 lowering the $35 token of appreciation used in FACES 2006 and FACES 2009 to $15 as a base (with a bonus for responding early for a potential of $25). FACES 2014-2018 achieved lower response rates compared to the earlier studies (78 percent versus 93 to 96 percent), and the nonresponse analysis found significant differences in the demographic characteristics between respondents and nonrespondents.11 Respondents were more likely to have a child with a disability, be from non-English home language households, have limited cell phone minutes, and attend programs with a lower percentage of Black and higher percentage of white children compared to nonrespondents. Based on these results, the study team plans to offer families a $35 token of appreciation to complete the family questionnaire and a cognitive interview (total time to complete both is estimated to be 40 minutes).
As outlined above, acquiring a sufficient number of responses from a diverse group of providers and the families they serve in phase two is critical for data analysis and being able to answer the pilot study’s research questions. Phase two tokens of appreciation reflect amounts that are consistent with tokens of appreciation offered in phase one and provided in recent studies with HBCC providers.12 However, the planned tokens of appreciation for phase two are lower than those planned for phase one because providers and families are still asked to complete the provider and family questionnaires, but they are not asked to also complete cognitive interviews.
In addition to the proposed tokens of appreciation, the team is taking other steps to encourage participation, such as working with community organizations to recruit providers and working with providers to recruit families.
Table A.5. Planned tokens of appreciation
Respondent |
Activity |
Burden |
Token of appreciationa |
Phase one: Provider |
95 minutes |
$75 |
|
Phase one: Phase one providers, experts with lived experience providing HBCC and providers who participated in the summer 2022 pre-test (and are not pilot study respondents) |
|
15 minutes |
$10 |
Phase one: Family member b |
|
40 minutes |
$35 |
Phase two: Provider |
|
50 minutes |
$50 |
Phase two: Provider |
|
15 minutes |
$10 |
Phase two: Family member |
|
10 minutes |
aThe study team plans to provide gift cards because some participants might have difficulty cashing checks.
bSpanish-speaking providers from phase one will recruit Spanish-speaking families to complete the Spanish version of the family questionnaire. The project’s experts with lived experience providing HBCC and providers who participated in the summer 2022 pre-test will invite English-speaking families to complete the English version of the family questionnaire, and if needed, Spanish-speaking families to complete the Spanish version of the family questionnaire.
Personally Identifiable Information
This data collection effort will collect personally identifiable information (PII) such as names, email and mailing addresses, and telephone numbers to obtain consent to participate in data collection activities and arrange data collection (including scheduling and sending invitations, study materials, and reminders for the data collection activities such as the phase one cognitive interviews and delivering the token of appreciation). Staff from community organizations that provide supports to providers will be asked to use a secure website to share only provider names, contact information, and other characteristics to use to select providers. It is expected that some of these staff will need to ask providers for permission to share their contact information with the study team, while others may already have permission to share provider contact information.13 Providers will be asked to confirm or provide their contact information and other characteristics over the phone to arrange data collection activities. They may also be asked to share the names and contact information for other HBCC providers by phone to recruit more participants for the study.
The study team will ask providers (including providers who are phase one and phase two pilot study respondents, experts with lived experience providing HBCC and are not pilot study respondents, and providers who participated in the summer 2022 pre-test and are not pilot study respondents) to distribute and collect the family questionnaire (in a sealed envelope) from one or more family members of a school-age child in their care. Some providers may prefer that the study team reach out to families they recommend for the study. In this scenario, providers will share recommended family names and their contact information with the study team by phone or using a secure website.
Any PII collected by the study team, for example by phone or through a secure website, will be stored on a secure network drive at the Contractor’s offices. Information will not be maintained in a paper or electronic system which data are actually or directly retrieved by an individuals’ personal identifier. Participant’s responses to the HBCC-NSAC Toolkit and cognitive interviews will be stored separately from their PII.
Assurances of Privacy
Information collected will be kept private to the extent permitted by law. Respondents will be informed of all planned uses of data, that their participation is voluntary, and that their information will be kept private to the extent permitted by law. As specified in the contract, the Contractor will comply with all Federal and Departmental regulations for private information.
Respondents will complete a consent form (Appendix F) before completing the data collection instruments. The statement includes information so respondents can contact the study team with any questions or concerns. The statement also explains that the study team will not identify them or share any information that could identify them outside of the study.
In addition, the design of this study means that the team will collect data from people connected to each other, for example, providers and family members of the children the provider cares for. Providers will be asked to distribute and collect the family questionnaires from families of school-age children in their care. As described in Supporting Statement Part B Section B4, families can return the completed family questionnaire to providers in a self-sealed envelope provided by the study team. Although providers will know which families participate in the pilot, they will not know how families responded to the questionnaire if families choose to use the self-sealed envelope, and this will be explained clearly through an informed consent process.
All materials to be used with respondents as part of this information collection, including consent forms and instruments, will be submitted to the Contractor’s Institutional Review Board (IRB), Health Media Lab (HML), for approval. The study team plans to obtain a Certificate of Confidentiality, which helps to assure participants that their information will be kept private to the fullest extent permitted by law. The study team has applied for this Certificate and will provide it to OMB once it is received.
Data Security and Monitoring
As specified in the contract, the Contractor shall protect respondent privacy to the extent permitted by law and will comply with all Federal and Departmental regulations for private information. The Contractor has developed a Data Security and Monitoring Plan that assesses all protections of respondents’ PII. The Contractor shall ensure that all of its employees, consultants, subcontractors (at all tiers), and employees of each subcontractor, who perform work under this contract/subcontract, are trained on data privacy and security issues and comply with the above requirements.
All electronic data will be stored on a secure network drive at the Contractor’s offices and never in possession of ACF; data will be backed up on our secure servers for 60 days for disaster recovery purposes. Sixty days after the primary data files are securely deleted, the backed-up data will be automatically and securely overwritten, as required by our contract (i.e., “The Contractor shall dispose of the primary data and files created during the course of the study in accordance with specifications provided by ACF”). These plans are described in more detail in a data security plan, also required by the contract. Systems will be accessible only by staff working on the project through individual passwords and logins. For staff working remotely, access is granted through a secure, two- factor authentication VPN connection.
As specified in the contract, the Contractor shall use Federal Information Processing Standard compliant encryption (Security Requirements for Cryptographic Module, as amended) to protect all instances of sensitive information during storage and transmission. The Contractor shall securely generate and manage encryption keys to prevent unauthorized decryption of information, in accordance with the Federal Information Processing Standard. The Contractor shall: ensure that this standard is incorporated into the Contractor’s property management/control system; establish a procedure to account for all laptop computers, desktop computers, and other mobile devices and portable media that store or process sensitive information. Any data stored electronically will be secured in accordance with the National Institute of Standards and Technology (NIST) requirements and other applicable Federal and Departmental regulations. In addition, the Contractor must submit a plan for minimizing to the extent possible the inclusion of sensitive information on paper records and for the protection of any paper records, field notes, or other documents that contain sensitive or PII that ensures secure storage and limits on access.
No information will be given to anyone outside the study team and ACF. All PII, for example typed notes and audio recordings of cognitive interviews, will be stored on restricted, encrypted folders on the Contractor’s network, which is accessible only to the study team.
There are no sensitive questions or statements in the data collection instruments for HBCC providers (provider questionnaire). The family questionnaire includes one question that may be perceived as sensitive. The question asks the respondent whether they have discussed aspects of their child’s identity with their HBCC provider and how important that is to them. The question has examples that include aspects of racial, ethnic, and cultural identity as well as religion or spirituality broadly. To help families and HBCC providers collaborate on the aspects of their child’s care that are important to the families, it is necessary to ask families about their preferences. The family questionnaire does not ask respondents to appraise the HBCC provider or the care they provide in any way. It also does not ask the family to disclose their religious beliefs. The family questionnaire respondents will read the consent form (Appendix F), which states that respondents do not have to respond to any questions that make them uncomfortable, and that their participation in the family questionnaire is voluntary.
As described in Section A10, the data collection protocol and related materials, such as the consent forms and instruments, will be reviewed by the Contractor’s IRB in fall 2022. Data collection will not begin until IRB approval has been received.
Explanation of Burden Estimates
Table A.6 provides an estimate of time burden for the data collection, broken down by instrument. For the provider questionnaire, the time estimates are based on the pre-test conducted for the English version of the HBCC provider questionnaire during the initial development of the HBCC-NSAC Toolkit. For the family questionnaire, timing is based on internal testing with English-speaking parents of school-age children at the study team’s organization and Spanish-speaking individuals who are experienced in conducting observational measures in FCC settings with Spanish-speaking providers.
Estimated Annualized Cost to Respondents
The study team based average hourly wage estimates for deriving total annual costs on data from the Bureau of Labor Statistics, including the Occupational Employment Statistics (2021). For each instrument included in Table A.6, the team calculated the total annual cost by multiplying the annual burden hours by the average hourly wage, as follows:
The mean hourly wage of $13.31 for childcare workers (occupational code 39-9011) in May 2021 is used for home-based child care providers. Data from which these wages were drawn are available at https://www.bls.gov/oes/current/oes399011.htm.
The median usual weekly earnings in the first quarter of 2022 for full-time wage and salary workers age 25 and older with a high school diploma is used for family members. Dividing weekly earnings ($827) by 40 hours yields hourly wages of $20.68. Data from which these wages were drawn are available at https://www.bls.gov/charts/usual-weekly-earnings/usual-weekly-earnings-over-time-by-education.htm.
Table A.6. Estimated burden and annualized cost to respondents
Instrument |
No. of Respondents (total over request period) |
No. of Responses per Respondent (total over request period) |
Avg. Burden per Response (in hours) |
Total/Annual Burden (in hours) |
Average Hourly Wage Rate |
Total Annual Respondent Cost |
Instrument 1: Phase one provider telephone script and recruitment information collection |
18 |
1 |
0.333 |
6 |
$13.31 |
$79.86 |
Instrument 2: Phase two provider telephone script and recruitment information collection |
450 |
1 |
0.333 |
150 |
$13.31 |
$1,996.50 |
Instrument 3: HBCC-NSAC Toolkit provider questionnaire (Phase 1) |
9 |
1 |
0.750 |
7 |
$13.31 |
$93.17 |
Instrument 3: HBCC-NSAC Toolkit provider questionnaire (Phase 2) |
150 |
1 |
0.500 |
75 |
$13.31 |
$998.25 |
Instrument 4: HBCC-NSAC Toolkit family questionnaire (Phase 1) |
18 |
1 |
0.167 |
3 |
$20.68 |
$62.04 |
Instrument 4: HBCC-NSAC Toolkit family questionnaire (Phase 2) |
150 |
1 |
0.167 |
25 |
$20.68 |
$517.00 |
Instrument 5: Phase one provider cognitive interview guide |
9 |
1 |
0.500 |
5 |
$13.31 |
$66.55 |
Instrument 6: Phase one family cognitive interview guide |
18 |
1 |
0.500 |
9 |
$20.68 |
$186.12 |
Instrument 7: Family data collection logistics call (Phase 1) |
18 |
1 |
0.250 |
5 |
$13.31 |
$66.55 |
Instrument 7: Family data collection logistics call (Phase 2) |
150 |
1 |
0.250 |
38 |
$13.31 |
$505.78 |
TOTAL |
|
|
|
323 |
|
$4,571.82 |
The study team will work with community organizations that offer support and quality improvement opportunities to HBCC providers, to more efficiently support recruitment of potential respondents. Each community organization’s role is critical for recruiting HBCC providers to participate in the study and if needed, hosting field staff conducting on-site recruitment and data collection during site visits. We anticipate that the support of these organization, and particularly their involvement in informing providers about the pilot study, will play an important role in recruiting HBCC providers that might be hesitant to participate in the study (especially those from under-resourced communities as noted in Section A9). The study team will offer honoraria of $250 to organizations who identify a site coordinator to support pilot study activities. The study team will tailor recruitment goals for community organizations based on recruitment needs and the characteristics of providers in the partner’s network.
Table A.7 lists the estimated annualized costs to the federal government. Estimates are based on the study team’s budget for each task and include labor hours, other direct costs, subcontractor and consultant costs, indirect costs, and fee. The direct collection of information (field work) will take place within a one-year period.
Table A.7 Estimated annualized costs to the federal government
Cost Category |
Estimated Costs |
Field Work |
$513,806 |
Instrument Development, Analysis, and Reporting |
$218,833 |
Publications/Dissemination |
$0 |
Total/Annual costs |
$732,639 |
This is for an individual information collection under the umbrella clearance for pre-testing (0970-0355).
Table A.8 contains the timeline for the recruitment, data collection, analysis, and reporting activities.
Activity |
Timing |
Recruitmenta |
|
Train staff, identify and establish agreements with community organizations |
To begin immediately after OMB approval and extend for 3 months |
Phase one: Recruit HBCC providers and families |
To begin 2 weeks after OMB approval and extend for 1 month |
Phase two: Recruit HBCC providers and families |
To begin 2 weeks after OMB approval and extend for 3 months |
Data collectiona,b |
|
Phase one: HBCC-NSAC Toolkit provider questionnaire, phase one provider cognitive interview guide |
Phase one: To begin 1 month after OMB approval and extend for 1 month |
Phase one: HBCC-NSAC Toolkit family questionnaire, phase one family cognitive interview guide |
Phase one: To begin 1 month after OMB approval and extend for 1 month |
Phase two: HBCC-NSAC Toolkit provider questionnaire |
To begin 1 months after OMB approval and extend for 3 months |
Phase two: HBCC-NSAC Toolkit family questionnaire |
Phase two: To begin 1.5 months after OMB approval and extend for 2.5 months |
Analysis |
|
Data archive and data analysis plans |
To begin 3 months after OMB approval and extend for 1 month |
Conduct descriptive and Rash analysis, reduce and/or revise items |
To begin 4 months after OMB approval and extend for 4 months |
Obtain input from experts and community members |
To begin 4 months after OMB approval and extend for 1 month |
Reporting |
|
Brief internal progress memorandums to ACF describing the status of study recruitment, data collection, and initial pilot study findings |
To begin 2 weeks after OMB approval and extend for 3 months |
Final internal study memorandum to ACF with populated data tables |
To begin 3 months after OMB approval and extend for 4 months |
aOnce recruitment and data collection activities begin, they will be ongoing and will end either when targets are met or four months after OMB approval.
bThe data collection activities for phase one and phase two will begin at the same time. For phase one, the Spanish version of the HBCC-NSAC Toolkit provider questionnaire and the family questionnaire will be released one month after OMB approval. For phase two, the English version of the HBCC-NSAC Toolkit provider questionnaire will be released one month after OMB approval. The Spanish version of the HBCC-NSAC Toolkit provider questionnaire and the family questionnaire will be released after the majority of providers and families have completed the phase one data collection activities.
No exceptions are necessary for this information collection.
Instrument 1: Phase one provider telephone script and recruitment information collection
Instrument 2: Phase two provider telephone script and recruitment information collection
Instrument 3: HBCC-NSAC Toolkit provider questionnaire
Instrument 4: HBCC-NSAC Toolkit family questionnaire
Instrument 5: Phase one provider cognitive interview guide
Instrument 6: Phase one family cognitive interview guide
Instrument 7: Family data collection logistics call
Appendix A: Community organization outreach materials
Appendix B: Phase one provider and family recruitment materials
Appendix C: Phase two provider and family recruitment materials
Appendix D: Supplemental provider and family recruitment materials
Appendix E: HBCC-NSAC Toolkit provider questionnaire matrix sampling
Appendix F: Consent
Appendix G: Sources for HBCC-NSAC Toolkit development
Appendix H: HBCC-NSAC Toolkit English version pre-test summary
Appendix I: Provider and family invitations
Appendix J: Family data collection instructions
Appendix K: Frequently asked questions
Appendix L: Provider and family thank you letters
1Laughlin, L. “Who’s Minding the Kids? Child Care Arrangements: Spring 2011.” Current Populations Report, U.S. Census Bureau, April 2013, pp. 70-135. https://www.census.gov/library/publications/2013/demo/p70-135.html.
Liu, Meirong. “An Ecological Review of Literature on Factors Influencing Working Mothers’ Child Care Arrangements.” Journal of Child and Family Studies, vol. 24, no. 1, 2015, pp. 161–171.
NSECE Project Team. “Measuring Predictors of Quality in Early Care and Education Settings in the NSECE.” OPRE Report #2015-93, Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2015.
Liu, Meirong, and Steven G. Anderson. “Neighborhood Effects on Working Mothers’ Child Care Arrangements.” Children and Youth Services Review, vol. 34, no. 4, 2012, pp. 740–747.
Porter, Toni, Diane Paulsell, Patricia Del Grosso, Sarah Avellar, Rachel Hass, and Lee Vuong. “A Review of the Literature on Home-Based Child Care: Implications for Future Directions.” Washington, DC: Mathematica Policy Research, January 15, 2010.
2 A. R. Datta, C. Milesi, S. Srivastava, and C. Zapata-Gietl. “NSECE Chartbook – Home-Based Early Care and Education Providers in 2012 and 2019: Counts and Characteristics.” OPRE Report No. 2021-85, Washington DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2021.
3 Bromer, Juliet, Toni Porter, Chris Jones, Marina Ragonese-Barnes, and Jaimie Orland. “Quality in Home-Based Child Care: A Review of Selected Literature.” OPRE Report # 2021-136. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2021a.
Doran, Elizabeth, Ann Li, Sally Atkins-Burnett, Jasmine Forde, Jaimie Orland, Marina Ragonese-Barnes, Nathan Mix, Natalie Reid, and Ashley Kopack Klein (2022). Quality in Home-Based Child Care: Summary of Existing Measures and Indicators. OPRE Report #2022-27. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Doran, Elizabeth, Ann Li, Sally Atkins-Burnett, Jasmine Forde, Jaimie Orland, Marina Ragonese-Barnes, Nathan Mix, Natalie Reid, and Ashley Kopack Klein. “Compendium of Measures and Indicators of Home-Based Child Care Quality.” OPRE Report #2022-28. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2022.
4 Pretesting was completed with 9 respondents and therefore not subject to the Paperwork Reduction Act.
5 In addition to the measures review, the HBCCSQ project reviewed racial-ethnic socialization constructs that are commonly linked to positive child outcomes and racial-ethnic socialization measures. The project team used findings from this review to develop and refine items in the HBCC-NSAC Toolkit.
Doran, Elizabeth, Ann Li, Sally Atkins-Burnett, Jasmine Forde, Jaimie Orland, Marina Ragonese-Barnes, Nathan Mix, Natalie Reid, and Ashley Kopack Klein (2022). Quality in Home-Based Child Care: Summary of Existing Measures and Indicators. OPRE Report #2022-27. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
Doran, Elizabeth, Ann Li, Sally Atkins-Burnett, Jasmine Forde, Jaimie Orland, Marina Ragonese-Barnes, Nathan Mix, Natalie Reid, and Ashley Kopack Klein. “Compendium of Measures and Indicators of Home-Based Child Care Quality.” OPRE Report #2022-28. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, 2022.
6 Grady, Christine. “Payment of Clinical Research Subjects.” Journal of Clinical Investigation, vol. 115, no. 7, 2005, pp. 1681–1687. https://doi.org/10.1172/JCI25694.
7 Brown, K. Steven, Kilolo Kijakazi, Charmaine Runes, and Margery Austin Turner. “Confronting Structural Racism in Research and Policy Analysis: Charting a Course for Policy Research Institutions.” Washington, DC: Urban Institute, February 2019. Available at https://www.urban.org/sites/default/files/publication/99852/confronting_structural_racism_in_research_and_policy_analysis_0.pdf.
8 Singer, E., and Ye, C. (2013). The use and effects of incentives in surveys. Annals of the American Academy of Political and Social Science, 645(1): 112-141.
9 Kelly, Bridget, Marjorie Margolis, Lauren McCormack, Patricia A. LeBaron, and Dhuly Chowdhury. “What Affects People’s Willingness to Participate in Qualitative Research? An Experimental Comparison of Five Incentives.” Field Methods, vol. 29, no. 4, November 2017, pp. 333–350. https://doi.org/10.1177/1525822X17698958.
10 FACES 2006, FACES 2009, FACES 2014-2018, and FACES 2019 all have OMB Number 0970-0151. FACES 2006 expired on 06/30/2009, FACES 2009 expired on 06/30/2012, and FACES 2014-2018 expired on 02/28/2018. FACES 2019 expired on 04/30/2022.
11 FACES 2019 OMB Supporting Statement A for Data Collection (2021). Available at https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=202106-0970-002. Accessed October 7, 2021.
12 Bromer, Juliet, Toni Porter, Samantha Melvin, & Marina Ragonese-Barnes. “Findings from the Multi-State Study of Family Child Care Decline and Supply.” Herr Research Center: Erikson Institute, 2021b. Available at https://www.erikson.edu/wp-content/uploads/2021/11/FCD_DeclineStudy_2021.pdf.
Early Head Start–Child Care Partnerships Sustainability Study (EHS-CCP Sustainability). Early Head Start–Child Care Partnerships Sustainability Study OMB Information Collection Request 0970 – 0471: Supporting Statement Part A. July 2021. Available at https://www.reginfo.gov/public/do/PRAViewDocument?ref_nbr=202112-0970-012
13 The study team expects that some organizations will have documentation that their providers are willing to share their information. For example, their providers will have “opted in” to external projects by noting whether their email or phone, or both can be shared.
14 Examples of sensitive topics include (but not limited to): social security number; sex behavior and attitudes; illegal, anti-social, self-incriminating and demeaning behavior; critical appraisals of other individuals with whom respondents have close relationships, e.g., family, pupil-teacher, employee-supervisor; mental and psychological problems potentially embarrassing to respondents; religion and indicators of religion; community activities which indicate political affiliation and attitudes; legally recognized privileged and analogous relationships, such as those of lawyers, physicians and ministers; records describing how an individual exercises rights guaranteed by the First Amendment; receipt of economic assistance from the government (e.g., unemployment or WIC or SNAP); immigration/citizenship status.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Mathematica Report |
| Subject | report |
| Author | Sharon Clark |
| File Modified | 0000-00-00 |
| File Created | 2024-07-21 |