National Survey of Early Care and Education (NSECE)
Pre-testing of Evaluation Surveys
(OMB 0970‑0355)
Supporting Statement Part A
This statement covers a field test for the National Survey of Early Care and Education (NSECE), sponsored by the Office of Planning, Research and Evaluation, Administration for Children & Families (ACF), U S Department of Health and Human Services. The NSECE will be the first national survey of early child care and education in America in twenty years during which the use and funding of early care and education as well as (before and) after-school care has changed dramatically.1 NSECE will be able to provide a current picture of the supply and demand for child care and early education programs and fill a gap in our understanding of the factors influencing parents’ choice of care for their children. The NSECE will be a major effort, both in expenditure of public funds as well as response burden among households and individual and institutional providers of care to children under age 13. A multi-faceted field test that mirrors the design of the primary NSECE study is necessary to test data collection strategies, validate questionnaires and sampling procedures and review analysis plans to ensure these are most likely to function well in the main study.
The objective of the NSECE is to document the nation’s current utilization and availability of early care and education (including school-age care), and to deepen our understanding of the extent to which families’ needs and preferences coordinate well with providers’ offerings and constraints. The experiences of low-income families are of special interest as they are the focus of a significant component of ECE/SA public policy. The purpose of this field test to be conducted between January and May, 2011 is to test specific components of the NSECE design, including questionnaire items and implementation of data collection procedures in order to make any necessary revisions that would benefit the NSECE main study to be implemented in spring 2012. A significant advantage of the NSECE over recent studies is the dual emphasis on both the usage and availability of early care and education services. The two have not been studied together in a nationally-representative study in two decades. In fact, a comprehensive nationally-representative supply-side profile has not been constructed since that time. In addition to replicating the breadth on the supply-side from the 1989-1990 studies, the NSECE design includes the family, friend and neighbor sector of providers, which would be a significant expansion over the previous design. Also on the provider side, the previous studies included only providers that served pre-school (and possibly other) children. NSECE includes school-age only programs and other providers that serve the under-13 age group, whether or not they offer pre-school services. Another enhancement is the inclusion of a Workforce Provider Interview that will collect personal characteristics and instructional/caregiving practices from workforce members in home-based and center-based programs.
Questionnaire |
Sample Source |
Household Screener |
Address-based Sample |
Household Questionnaire |
Eligible households identified through Household Screener |
Home-based Provider Questionnaire |
Eligible households identified through Household Screener AND Administrative Lists |
Center-based Provider Questionnaire
|
Administrative Lists |
Workforce Provider Questionnaire |
Completed Center-based Provider interviews |
The NSECE field test design will mirror the main study and will include five inter-related questionnaires.
The Household Screener is used with all sampled households to determine household eligibility to complete the Household Questionnaire or the Home-Based Provider Questionnaire.
The Household Questionnaire is to be conducted with a parent or guardian of a child or children under age 13. Eligible respondents will be identified through the Household Screener.
The Home-Based Provider Questionnaire will be completed with two types of respondents. The first type is individuals who are identified on administrative lists as providing regulated or registered home-based care. The second type is individuals identified through the Household Screener as caring in a home-based setting for children under age 13 who are not their own (and who do not appear in the lists of the first type).
The Center-Based Provider Questionnaire is to be completed with directors of ECE/SA providers who can be identified from administrative lists such as state licensing lists, Head Start program records, or pre-K rolls.
The Workforce Provider Questionnaire will be completed with staff members of sampled formal providers. After each Center-based Provider interview is completed, one staff member from that organization will be sampled and administered the Workforce Provider interview.
For the Household Interview and Home-Based Provider Interview for individuals identified through the Household Screener, we developed a multi-mode address-based sampling (ABS) approach for the identification and interviewing of 1) households with children under the age of 13, and 2) individuals who provide home-based care for children under age 13 other than their own (and who would not appear on a state-level administrative list of ECE providers). Such a design is not compromised by low land-line usage rates among the population. The ABS sample will include an oversample of low-income families. The Home-Based Provider Interview with individuals providing regulated or registered home-based care and the Center-Based Provider Interview build a national sampling frame of all ‘listable’ providers of ECE/SA services and samples programs from that frame. Respondents for the related Workforce Provider Interview will include staff of providers who completed the Center-Based Provider Questionnaire.
Research questions from the NSECE have been developed to investigate factors influencing the availability and usage of the early care education programs and informal services and in turn uniquely inform current child care policy. For example, the Household Interview will collect data about families’ usage, need for and access to child care throughout the week, while the provider interviews will gather information about hours of operation of both center- and home-based ECE providers. The combination of these data will indicate service gaps and alternative care options selected by parents.
Specific questions to be answered by the NSECE field test include:
Questionnaire Performance. a) How effective the mail screener is in determining household eligibility? b) How well are the questionnaires functioning in terms of length of administration, respondent ability to answer questions, and rudimentary review of collected data? c) Application of less formal language for home-based provider respondents identified through the Household Screener; d) How effective are questionnaire revisions of proxy reporting in large providers, especially regarding financial data and/or staff characteristics/classroom practices?
Implementation of Sampling Design. a) How do the household sampling procedures work for setting a minimum size at the primary and secondary sampling unit levels? b) How do the actual yields of eligible children by geography and income level compare to estimates? c) How do the various household rates assumed in the sample size calculations, including oversampling of low-income households, work? d) What is the prevalence of eligible home-based providers identified through the household screener? e) What is the yield of providers in the current design, especially of Head Start and Pre-K programs that are thought to be rare?
Data collection strategies. a) What is the most appropriate incentive structure to recruit eligible households and to improve response rates? b) How does the Center-Based Provider Questionnaire perform in an on-line administration, particularly in regards to difficult revenue and enrollment items? c) Do the survey administration timings match assumptions? d) What privacy issues will be particularly important to tackle, for example, in collecting geographical and other identifying information from households, collecting provider identifying information from households, and possibly in collecting releases for administrative data? e) What staff positions in larger programs are likely to be associated with highest quality responses? What operational issues are associated with trying to pursue those respondent types exclusively or primarily?
NORC plans to use a multi-mode data collection approach that will give respondents the freedom to select the most convenient mode through which to complete the questionnaire. The Household Screener will be offered by mail, computer-assisted telephone interviewing (CATI), and computer-assisted personal interviewing (CAPI), while Household Questionnaire respondents will be offered CATI and CAPI options. Respondents to the Center-Based Provider Questionnaire, Home-Based Provider Questionnaire, and Workforce Provider Questionnaire may be offered CAPI, in-person or by telephone, and Web options. The Workforce Provider Questionnaire respondents will also be offered a self-administered questionnaire, if necessary. CATI, CAPI and Web surveys all reduce respondent burden and produce data that can be prepared for release and analysis faster and more accurately than is the case with pencil-and-paper interviews.
The NSECE design calls for several components which have not been found together in a single design, if found at all in recent work. These include: 1) simultaneous and integrated collection of provider and parent usage data of early care and education in a single study design; 2) coverage of care issues for all children up to age 13; 3) inclusion of informal home-based providers; and 4) nationally representative design, rather than focus on particular sub-sets such as low-income or selected geographic areas. Because the NSECE design would be unique among existing data collections, a field test of that design also has unique features that are not duplicative of existing efforts. Nonetheless, during the design phase of the NSECE great care was taken to consult extensively with representatives of surveys most similar to various components of the NSECE design, including individuals knowledgeable about the Early Childhood Longitudinal Study – Birth Cohort, the National Household Education Survey: Early Childhood Program Participation, the National Study of America’s Families, the 1990 National Child Care Survey and Profile of Child Care Settings, the National Study of Child Care in Low-Income Families, and Child Care Voucher Programs: Provider Experience in Five Counties.
Data collection for the NSECE field test may impact small organizations involved in the administration of center- and home-based provider surveys. All efforts will be made to minimize the burden of survey participation on these providers. Each organization will be asked to complete a single questionnaire and will be able to designate the appropriate respondent within the staff ranks. The only exception to this will be for a sub-sample of center-based providers who will be selected for the Workforce Provider Survey for which one staff member will be asked to complete a questionnaire. This will not be the same respondent who completes the Center-Based Provider Questionnaire.
6. Consequences of Less Frequent Data Collection
This study includes one field test round, which will inform the NSECE main study projected to be fielded in 2012. Since no repetition is proposed for the field test, there is no opportunity to collect data less frequently.
None of the listed special circumstances apply.
The first Federal Register notice for ACF’s generic clearance for pretesting was published in the Federal Register, volume 73, no. 39, p. 21957 on April 23, 2008. The second Federal Register notice was published in the Federal Register, volume 73, no. 137, p. 44271 on July 30, 2008.
The NSECE will convene one meeting of its expert panel (December 2010) prior to the proposed field test. Expert panel members come from research organizations and universities and include the following individuals:
Gina Adams
Urban Institute
Steve Barnett
Rutgers University
Doug Besharov
University of Maryland
Richard Brandon
RNB Consulting
Anne Collins
Abt Associates
Marcia Meyer
University of Washington
Christine Ross
Mathematica Policy Research
In addition to the expert panelists and consultants listed above, a number of federal representatives from ACF and ASPE/HHS, Department of Education, Office of Child Care, Office of Head Start, Bureau of the Census, USDA, and Bureau of Labor Statistics will also attend the expert panel meetings and/or provide related content expertise.
Federal employees providing consultation from outside of the Department of Health and Human Services include:
Alison Aughinbaugh
Bureau of Labor Statistics
Lynda Laughlin
Bureau of the Census
Chris Chapman
National Center for Education Statistics
9. Payment to Respondents
One of the motivating factors for conducting a field test is to determine the relevance, appropriateness and efficiency of numerous aspects of survey administration in order to inform the implementation of the NSECE main study in 2012. In order to achieve the goals of efficient survey administration, maximum response rates and minimum burden to respondents for the full study, ACF believes it is necessary to actually test the necessity of incentive payments of various levels as part of the field test in order to determine the optimal incentives for the main study.
Household Screener Incentive – Telephone. In an effort to increase the number of telephone screener completes, we propose offering an incentive to households after the first refusal, a hang-up during introduction, or the second unsuccessful call attempt. Upon sample selection, we will attempt to match all addresses to a telephone number; we expect approximately 46 percent to do so. Matched cases will first be worked by telephone interviewers. This sample of matched addresses will be split into two groups for the purpose of this experiment: non-incentive and incentive-eligible cases. All cases will be worked with the same initial calling approach; however when the incentive-eligible cases encounter a first refusal, hang-up during introduction, or two unsuccessful call attempts (e.g., non-contacts, callbacks), the household will be sent a $10 incentive via mail requesting that they complete the screener. Cases will then be recontacted by telephone interviewers to attempt to determine household eligibility through the screener. Cases that are non-incentive eligible will continue to follow the primary field test design, in which all household cases that experience a refusal in the telephone center are transferred to field interviewers to be worked in-person or over the telephone by a field interviewer. Used at the household level, this would apply to households being screened for eligibility for both the Household and Home-Based Provider (from Household Screener) Surveys.
Household Screener Incentive -- Mail. Since an increase in completion of mail screeners could lead to significant savings if field interviewers are not required to make contacts with cases that are ultimately ineligible, the second experiment is to be used on the portion of sample that are sent a mail screener. This mail screener incentive experiment includes two experimental groups. As shown in Exhibit 9.1, Households randomly assigned to experimental group 1 will receive $1 at the initial mailing and no incentive for the follow-up mailing. By contrast, sample households randomly assigned to experimental group 2 will receive no incentive at the initial mailing, but $5 at the follow-up mailing. Only addresses not matched to a telephone number will be eligible for this experiment.
Exhibit 9.1: |
Household Screener Mail Incentive Experiment |
Timing of Incentives |
Experimental Group 1 |
Experimental Group 2 |
1) Initial screener mailing |
$1 incentive |
No incentive |
3) Follow-up screener mailing |
No incentive |
$5 incentive |
Screener response rates from the two experimental groups will be compared to help determine which strategy will be the most cost effective for the main study. Furthermore, we will compare characteristics of households completed with and without incentives so that we can better target households that can be ‘converted to a mail screener complete’ by the use of an incentive for the main study. One possibility is that the effect of the incentive extends to the cooperativeness of the household with a field interviewer (after non-completion of the mail screener). We will test this hypothesis as well.
Household and Home-Based Provider Surveys Refusal Conversion Incentives. While the two above mentioned incentive experiments attempt to increase completion rates to the Household Screener, a third incentive is to be applied to households that have completed the screener and been found to have eligible respondents to the Household or Home-Based Provider Questionnaires. If those eligible respondents have refused the survey request and need additional conversion efforts, they would be included in the Household and Home-Based Provider Surveys Refusal Conversion Incentive Experiment. These efforts would occur only in the field since cases that receive either one refusal or five no-contacts in the telephone center will automatically be transferred to the field. (With the exception of those cases included in the Telephone Household Screener Incentive experiment described above.) Following demonstrated models from the National Immunization Survey, we propose implementing a refusal conversion incentive experiment that involves two experimental conditions. Field cases that have one refusal will be eligible for this experiment. As shown in Exhibit 9.2, households randomly assigned to experimental group 1 will receive a $5 prepaid incentive and households randomly assigned to experimental group 2 will receive a $5 prepaid incentive AND a $10 promised incentive upon completion of the interview. Used at the household level, this would apply to households eligible for both the Household and the Home-Based Provider Surveys.
Exhibit 9.2: |
Household and Home-Based Provider Surveys Refusal Conversion Incentive Experiment |
Timing of Incentives |
Experimental Group 1 |
Experimental Group 2 |
After 1 refusal in the field |
$5 Prepaid |
$5 prepaid + $10 promised |
Home-Based Providers. For informal home-based providers who are identified through the Household Screener either in-person or by telephone, or are in households that have multiple eligible respondents for the Household and/or Home-Based Provider Questionnaires, we will apply the refusal conversion incentive models described in the above household incentive section. By contrast, home-based providers identified through the Household Screener that are not eligible for the Household Questionnaire will also be eligible for the Home--Based Provider and Center-Based Provider Web Survey Incentive Experiment described below for completing the Web survey.
Home-Based Provider and Center-Based Provider Web Survey Incentive. In order to reduce field labor costs for formal provider data collection, providers identified and sampled from the administrative sampling frame (both center- and home-based) will be encouraged to complete the Web survey through the promise of a $50 incentive upon survey completion. Following the household mail screener format, prior to field data collection formal providers will be sent three mailings asking them to complete the Web survey. The first mailing will be an advance letter that explains the purpose of the study, the reason for their selection, and the survey URL. One week later a second mailing, in different packaging, would be sent, again requesting their participation. The third and final mailing would follow one week later, informing respondents that a field interviewer will be contacting them in the near future to complete an interview in a different mode. All providers who have not completed the Web survey by the third mailing will be randomly assigned to one of two experimental groups. Nonresponding providers assigned to experimental group 2 will be offered a $50 gift card upon completion of the Web survey and nonresponding providers assigned to the experimental group 1will not be offered an incentive.
Exhibit 9.3: |
Home-Based and Center-Based Provider Web Survey Incentive Experiment |
Timing of Incentives |
Experimental Group 1 |
Experimental Group 2 |
Respondent requires third mail invitation to web survey |
No incentive. |
$50 incentive upon completion of questionnaire |
Center-Based Provider Questionnaire Gatekeeper Incentive. The NSECE field test may also benefit from a gatekeeper incentive experiment that is specific to the common structure of formal providers. The program director of each sampled provider will be the survey respondent, but we expect that interviewers will often need to go through a gatekeeper to get appointments with and gain the cooperation of directors. The National Longitudinal Study of No Child Left Behind had success in giving $10 prepaid incentives to school gatekeepers in order to secure cooperation and maintain good relations to benefit future communications. This was particularly critical due to the multiple contacts required for scheduling interviews, which will also likely be required for this sample. We will randomly divide all sampled providers into two experimental groups after one refusal. For the experimental group 2, interviewers may offer gatekeeper incentives if deemed necessary; no incentive will be offered to experimental group 1.
Exhibit 9.4: |
Center-based Provider Questionnaire Gatekeeper Incentive Experiment |
Timing of Incentives |
Experimental Group 1 |
Experimental Group 2 |
If gatekeeper is hindering access to survey respondent |
No incentive. |
$10 incentive |
This comprehensive approach will allow for a comparison of incentive experiment outcomes on field versus telephone center completes, which will inform the levels of effort, data collection strategies, and staffing for the main data collection period.
Respondents will receive information about privacy protections when they consent to participate in the study. Information about privacy will be repeated in the introductory comments of interviewers. All interviewers will be knowledgeable about privacy procedures and will be prepared to describe them in detail or to answer any related questions raised by respondents. See Appendix A for contact materials.
We have crafted carefully worded consent language that explains in simple, direct language the steps we will take to protect the privacy of the information each sample member provides. Assurances of privacy related to the household and provider interviews will be given to each respondent as he or she is recruited for the study. Parents will be assured that their responses will not be shared with their childcare providers, providers will be assured that their responses will not be shared with other providers participating in the study. All respondents will be notified that their responses will be compiled only as part of aggregate statistics across all participating sample members.
NORC’s safeguards for the security of data include: storage of printed survey documents in locked space at NORC, and protection of computer files at NORC and its subcontractors against access by unauthorized individuals and groups. Protection of the privacy of individuals is accomplished through the following steps: oral permission for the interview is obtained from all respondents, after the interviewer ensures that the respondent has been provided with a copy of the appropriate NORC privacy information and understands that participation is voluntary, and information identifying respondents is separated from the questionnaire and placed into a separate database.
The field test data will be used for methodological purposes and not released for analysis outside of the project team.
At the close of the Household Questionnaire, respondents are requested to provide consent for the project to access administrative records from government subsidy programs. Parents who grant such consent are then requested to provide the full names, dates of birth, and the street address of their children under age 13. (Please see Section H of the Household Questionnaire for these items.) Such sensitive information is required in order to match administrative records to survey data. The availability and use of child-care subsidies is a key research topic of this study. These data require extensive questionnaire batteries for collection and are even then very difficult for parents to report accurately. Collection of administrative records would improve the quality of subsidy data and reduce respondent burden for subsidy recipients and non-recipients. Respondents are free to refuse consent for records access, and in this case will not be asked for personal identifying information.
The Home-Based Provider and Center-Based Provider Questionnaires ask for street address (or nearest street intersection) for the purposes of geographic mapping. Because households typically use early care and education providers who are located in close proximity to the home address, understanding the locations of households and their providers is essential to depicting the supply and demand for early care and education across the nation.
Estimated number of burden hours to complete the data collection of the field test task
Questionnaire
|
Annual Number of Respondents |
Number of Responses per Respondent |
Average Burden Hours per Response |
Estimated Annual Burden Hours |
Household Screener |
2,144 |
1 |
.1 |
215 |
Household Interview |
448 |
1 |
.75 |
336 |
Home-Based Provider Interview |
198 |
1 |
.3 |
60 |
Center-Based Provider Interview |
300 |
1 |
.67 |
201 |
Workforce Provider Interview |
240 |
1 |
.33 |
79 |
Estimated Total Annual Burden Hours |
|
|
|
891 |
13. Cost Burden to Respondents or Record Keepers
Respondents for this survey will not incur any capital, start-up, operation and maintenance, or purchase of service costs.
The total estimated cost of the NSECE field test is $800,000. This cost includes survey management, data collection, and other tasks involved in implementing the field test.
The field test proposes a new, one-time data collection from respondents.
As this effort is a field test in preparation for the main study data collection, no data findings will be published. Reporting on the field test will be methodological in nature, emphasizing implications for the NSECE main study.
The time schedule for data collection, tabulation and report delivery to DHHS is listed below.
Questionnaire Revisions October 2010 – November 2010
Sampling October 2010 – December 2010
Expert Panel Meeting December 2010
Field Test Data Collection January 2011 – May 2011
Data Processing June 2011 – August 2011
Data Analysis and Report Writing June 2011 – September 2011
Final Questionnaire Revisions (for main study) June 2011 – September 2011
Final Sampling Design (for main study) June 2011 – September 2011
Final Analysis Plan (for main study) June 2011 – September 2011
The nature of the questionnaires and sample design require that data collection be conducted with families while school-year activities are still in session. The January-May 2011 field test period is therefore essential to a valid implementation of the field test.
Does not apply.
We do not have any exceptions in Item 19, “Certification for Paperwork Reduction Act Submissions,” of OMB form 83-I.
1 A few main reports were issued from the 1989-90 studies. The main report from the provider survey component, the Profile of Child Care Settings, is available on-line at: http://www.eric.ed.gov/ERICWebPortal/custom/portlets/recordDetails/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED343702&ERICExtSearch_SearchType_0=no&accno=ED343702. The ERIC item number is ED343702.
The main report for the demand survey component, the National Child Care Survey, was published in a book that is now out of print. The citation is: Hofferth, Sandra L., April Brayfield, Sharon Deich and Pamela Holcomb. 1991. National Child Care Survey, 1990. Washington DC: Urban Institute Press.
| File Type | application/msword |
| Author | connelly-jill |
| Last Modified By | DSHOME |
| File Modified | 2010-11-15 |
| File Created | 2010-11-10 |