Volume 1 NHES 2019 Early Childhood Cognitive Interviews - Revised

Volume I

National Household Education Surveys Program (NHES)

2019 Early Childhood Cognitive Interviews - Revised

OMB# 1850-0803 v.215

(revised v.198)

May 2017

revised November 2017

Background

The National Household Education Survey (NHES) is a data collection program of the National Center for Education Statistics (NCES) aimed at providing descriptive data on the educational activities of the U.S. population, with an emphasis on topics that are appropriate for household surveys rather than institutional surveys. NHES topics have covered a wide range of issues, including early childhood care and education, children’s readiness for school, parents’ perceptions of school safety and discipline, before- and after-school activities of school-age children, participation in adult and career education, parents’ involvement in their children’s education, school choice, homeschooling, and civic involvement. NHES uses a two-stage design in which sampled households complete a screener questionnaire to enumerate household members and their key characteristics. Within-household sampling from the screener data determines which household member receives which topical survey. NHES typically fields 2 to 3 topical surveys at a time, although the number has varied across its administrations. Surveys are administered in English and in Spanish.

Beginning in 1991, NHES was administered roughly every other year as a landline random-digit-dial (RDD) survey. During a period of declining response rates in all RDD surveys, NCES decided to conduct a series of field tests to determine if a change to self-administered mailed questionnaires would improve response rates. After a 5-year hiatus in data collection for this developmental work, NCES conducted the first full-scale mail-out administration with NHES:2012, which included the Early Childhood Program Participation (ECPP) and the Parent and Family Involvement in Education (PFI) surveys. The same two surveys, along with the Adult Training and Education Survey (ATES), were fielded in NHES:2016. In 2019, the NHES will field the PFI and ECPP surveys along with the second administration of the ATES. This will be a two-stage study. In the first stage, households will be screened to determine if they contain eligible members. If eligible members are in the household, within- household sampling will be performed. Finally, topical surveys will be administered to the selected household members.

The ECPP, previously conducted in 1991, 1995, 2001, 2005, 2012 and 2016, surveys families of children ages 6 or younger who are not yet enrolled in kindergarten and provides estimates of children’s participation in care by relatives and non-relatives in private homes and in center-based daycare or preschool programs (including Head Start and Early Head Start). Additional topics addressed in ECPP interviews have included family learning activities; out-of-pocket expenses for nonparental care; continuity of care; factors related to parental selection of care; parents’ perceptions of care quality; child health and disability; and child, parent, and household characteristics.

The PFI, previously conducted in 1996, 2003, 2007, 2012, and 2016, surveys families of children and youth enrolled in kindergarten through 12^th grade or homeschooled for these grades, with an age limit of 20 years, and addresses specific ways that families are involved in their children’s school; school practices to involve and support families; involvement with children’s homework; and involvement in education activities outside of school. Parents of homeschoolers are asked about their reasons for choosing homeschooling and resources they used in homeschooling. Information about child, parent, and household characteristics is also collected. To minimize response burden and potential respondent confusion, both enrolled and homeschool versions of the PFI questionnaire were created for self-administration.

The ATES, previously conducted in 2016, surveys adults ages 16 to 65 who are out of high school and provides new measures of adults’ educational and occupational credentials. It identifies adults who have educational certificates, including the subject field of the certificate, its perceived labor market value, and its role in preparing for occupational credentialing; and counts adults who have an occupational certification or license, including the number of such credentials, type of work they are for, their perceived labor market value, and the role of education in preparing for these occupational credentials. To get a comprehensive picture of adult education and training, the survey also includes brief sections on adult participation in work experience programs (such as apprenticeships) and college classes.

2017 NHES Early Childhood Cognitive Interviews

Early learning is the bedrock for children’s later learning and success in school. Young children who have access to high quality early childhood education are more likely to be ready for school. However, not all children attend high quality early care arrangements, often attending unregulated child care that relatives, friends, and neighbors provide representing a range of quality. Collecting national data on the types of care young children attend and their level of school readiness provides information to stakeholders interested in improving the education outcomes of young children. Additionally, parents face many challenges to accessing affordable high quality early care and education arrangements. These challenges can make children vulnerable to attending early care and education arrangements that are suboptimal for their early learning. Understanding the challenges and the decision-making processes by which parents choose children’s care is of utmost importance so that stakeholders can support parents in the best possible way. The 2019 NHES-ECPP survey will expand content related to childcare decision-making and school readiness.

This request is to conduct cognitive interviews to assess the extent to which new NHES:ECPP survey items designed to capture data about childcare decision-making and school readiness can measure these early childhood phenomena accurately. Though no significant changes are planned to the structure of the questionnaire from the 2016 NHES-ECPP administration, the interviews will continue to evaluate respondents’ ability to navigate the survey instrument appropriately. Specifically, we want to ensure the items are clear, easily understood, and interpreted the way they were intended, and that respondents have the information needed to answer the questions. Additionally, cognitive interviews will ensure that the questions developed are applicable for racially, ethnically, and economically diverse populations and across care settings and age groups of children—including center-based, family-based, and Head Start settings. The results will guide potential revisions to the ECPP survey for NHES:2019.

Design

Each cognitive interview will explore parents’ reactions to and thoughts concerning the order and wording of new, reworded, and existing items on the proposed NHES:2019 ECPP paper instrument. The interviews will be conducted using a team of approximately 15 staff from Sanametrix, Activate Research, and Child Trends experienced with qualitative interviewing techniques, open-ended probes, and semi-structured interview protocols. Staff from NCES and the Census Bureau may observe selected interviews. The interviews will be conducted in-person in the metropolitan Washington, DC area in various locations (library, community center, coffee house, etc.) as is convenient for the participant. However, due to the hard-to-reach nature of the population being studied, some of the interviews may need to be conducted over the phone or through the use of video-conference technology. The protocol for the cognitive interviews is included in Attachment 2. The instruments that will be tested by the protocol are included in Attachment 3. If participants consent, the interviews will be audio recorded for the purposes of summarizing the results. Any materials that are generated during the interviews will be captured and incorporated into the results as well.

A total of 65 cognitive interviews will be conducted with parents of young children to determine whether there are systematic problems in the ECPP items by type of care and/or child’s age. Recruited parents will be grouped by their child’s age and participation in child care arrangements. At least 12 of the 65 interviews will be conducted in Spanish with Spanish-language dominant participants, making sure that at least one Spanish-speaking participant is represented in each grouping. Additionally, participants will be diverse with respect to race/ethnicity, education, and income. As such, tallies of race/ethnicity, education, and income of each recruited participant will be kept ensuring roughly equal representation of these demographic characteristics across the groupings.

Table 1. Grouping of Cognitive Interviews on New Measures of Early Childhood Phenomena

Type of Care	Age of child		Total
	0–23 months	2–3 years
Relative care	7	14	21
Nonrelative care	7	14	21
Center-based care	7	14	21
Total	21	42	63*

*Two interviews will be reserved to further explore items that are showing issues on any of the groups shown in the table. A total of 65 interviews will be completed.

NOTE: Care will be exercised to obtain a mix of race/ethnicity, education, and income groups; however, these are not selection criteria so equal distribution across these groups may not occur.

Recruiting and Paying Respondents

Participants will be recruited by Sanametrix, using multiple sources, including company databases, social media/Craig’s List, personal and professional contacts, community-based centers, early education and Head Start programs, fliers posted in the community, and on-the-ground recruitment (handing out fliers and describing the study to potential participants – Spanish language only). All recruitment materials (e-mail, social media, and fliers) are included in Attachment 1.

A brief screener interview will be conducted by phone with those who express an interest in participating in the cognitive interviews to determine their eligibility for the study. If the potential participant qualifies for the study, a cognitive interview will be scheduled at a time that is convenient for him or her. Recruited participants will receive a reminder call and/or text the day before or the day of the cognitive interview. The questions used to screen respondents for participation as well as the reminder call/text script are also included in Attachment 1.

People who have participated in usability testing, cognitive studies, or focus groups in the past 6 months and employees of the firms conducting the research will be excluded from participating.

To assure that we are able to recruit the select types of parents who are representative of those who would be taking part in the 2019 NHES and to thank them for their time and for completing the interview, during recruitment each parent will be offered a $75 incentive for participation. Participants will acknowledge receipt of the incentive (See Attachment 1).

Assurance of Confidentiality

The study will not retain any personally identifiable information. Prior to the start of the study, participants will be notified that their participation is voluntary. As part of the study, participants will be notified that the information they provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

For all participants, written consent will be obtained before interviews are administered. Participants will be assigned a unique identifier (ID), which will be created solely for data file management and used to keep all participant materials together. The ID will not be linked to the participant name in any way or form. The consent forms, which include the participant name, will be separated from the participant interview files, secured for the duration of the study, and will be destroyed after the final report is released. Interviews may be recorded using audio technology. The only identification included on the files will be the ID. The recorded files will be secured for the duration of the study and will be destroyed after the final report is completed.

Estimate of Hour Burden

We expect each cognitive interview to last approximately 60 minutes. Recruiting participants will require 5 minutes per potential participant. We anticipate it will require 20 recruitment attempts per eligible participant (thus an estimated 1,300 attempts to yield 65 participants). This will result in an estimated total of 174 hours of respondent burden for this study.

Table 2. Estimated response burden for 2017 NHES Early Childhood Cognitive Interviews

Respondents	Number of Respondents	Number of Responses	Burden Hours per Respondent	Total Burden Hours
Recruitment	1,300	1,300	0.0833	109
Cognitive Interviews	65	65	1	65
Total	1,300	1,365	-	174

Estimate of Cost Burden

There is no direct cost to respondents.

Project Schedule

The project schedule calls for recruitment to begin as soon as OMB approval is obtained. The interviews are expected to be completed by November 2017.

Cost to the Federal Government

The cost to the federal government for this cognitive interview study is approximately $144,569.