Justification

Volume I

National Household Education Surveys Program 2019 (NHES:2019) Low Response-Propensity Parents Focus Groups

OMB #1850-0803 v.188

March 2017

Attachments:

Attachment 1: Communication materials (Recruitment materials, screeners, and consent form)

Attachment 2: Study Materials (Moderator’s guide, NHES advance letters, and NHES screener)

Justification

The National Household Education Surveys Program (NHES) is conducted by the National Center for Education Statistics (NCES) and provides descriptive data on the educational activities of the U.S. population, with an emphasis on topics that are appropriate for household surveys rather than institutional surveys. NHES topics have covered a wide range of issues, including early childhood care and education, children’s readiness for school, parents’ perceptions of school safety and discipline, before- and after-school activities of school-age children, participation in adult and career education, parents’ involvement in their children’s education, school choice, homeschooling, and civic involvement. NHES uses a two-stage design in which sampled households complete a screener questionnaire to enumerate household members and their key characteristics. Within-household sampling from the screener data determines which household member receives which topical survey. NHES typically fields 2 to 3 topical surveys at a time, although the number has varied across its administrations. Surveys are administered in English and in Spanish.

Beginning in 1991, NHES was administered roughly every other year as a landline random-digit-dial (RDD) survey. During a period of declining response rates in all RDD surveys, NCES decided to conduct a series of field tests to determine if a change to self-administered mailed questionnaires would improve response rates. After a 5-year hiatus in data collection for this developmental work, NCES conducted the first full-scale mail-out administration with NHES:2012, which included the Early Childhood Program Participation (ECPP) and the Parent and Family Involvement in Education (PFI) surveys. The same two surveys, along with the Adult Training and Education Survey (ATES), were fielded in NHES:2016. In 2019, the NHES will field the PFI and ECPP surveys along with the second administration of the ATES. This will be a two-stage study. In the first stage, households will be screened to determine if they contain eligible members. If eligible members are in the household, within-household sampling will be performed. Finally, topical surveys will be administered to the selected household members.

The ECPP, previously conducted in 1991, 1995, 2001, 2005, 2012 and 2016, surveys families of children ages 6 or younger who are not yet enrolled in kindergarten and provides estimates of children’s participation in care by relatives and non-relatives in private homes and in center-based daycare or preschool programs (including Head Start and Early Head Start). Additional topics addressed in ECPP interviews have included family learning activities; out-of-pocket expenses for nonparental care; continuity of care; factors related to parental selection of care; parents’ perceptions of care quality; child health and disability; and child, parent, and household characteristics.

The PFI, previously conducted in 1996, 2003, 2007, 2012, and 2016, surveys families of children and youth enrolled in kindergarten through 12^th grade or homeschooled for these grades, with an age limit of 20 years, and addresses specific ways that families are involved in their children’s school; school practices to involve and support families; involvement with children’s homework; and involvement in education activities outside of school. Parents of homeschoolers are asked about their reasons for choosing homeschooling and resources they used in homeschooling. Information about child, parent, and household characteristics is also collected. To minimize response burden and potential respondent confusion, both enrolled and homeschool versions of the PFI questionnaire were created for self-administration.

The ATES, previously conducted in 2016, surveys adults ages 16 to 65 who are out of high school and provides new measures of adults’ educational and occupational credentials. It identifies adults who have educational certificates, including the subject field of the certificate, its perceived labor market value, and its role in preparing for occupational credentialing; and counts adults who have an occupational certification or license, including the number of such credentials, type of work they are for, their perceived labor market value, and the role of education in preparing for these occupational credentials. To get a comprehensive picture of adult education and training, the survey also includes brief sections on adult participation in work experience programs (such as apprenticeships) and college classes.

The NHES redesign phase tested various design features related to increasing response to the screener and to second-stage surveys. For example, the NHES is administered by the U.S. Census Bureau, and the Census Bureau “branding” increased screener response rates by five percent in the 2011 NHES Field Test, leading NHES to adopt U.S. Census Bureau branding for all of its materials in 2012 and in 2016. Additionally, NHES experimented with prepaid incentive amounts for the screener and the second “topical” stage of the survey to find amounts that encouraged response without escalating collection costs.

In 2016, NHES continued to experiment with prepaid incentive amounts. In previous NHES collections, random samples were assigned to receive $0, $2, or $5 during the screener phase and screener response rates were compared. In 2016, NHES began experimenting with the use of a response propensity model to predict sample members’ likelihood of response. Applying the model allowed NHES to use $0 and $2 incentives (compared to $5 prepaid incentives) for addresses likely to respond, thus saving collection costs. For those addresses modeled to be unlikely to respond, NHES experimented with $10 prepaid incentives (compared to $5 prepaid incentives) but found that the higher prepaid incentive amount did not increase response enough to make it cost-effective.

The 2019 collection will again field the three surveys fielded in 2016: ECPP, PFI, and ATES. The survey design will focus on attracting low-response propensity addresses to respond to the survey.

This request is to conduct focus groups with parents who share demographic characteristics with the occupants in addresses in previous NHES samples who responded at low rates. The focus groups will provide data to better understand both the barriers and benefits these parents tend to associate with participation in surveys like NHES and to identify communication strategies that will help overcome those barriers to participation. This information will guide recruitment strategies and materials development for NHES:2019.

Characteristics Associated with Low-Response Propensity in NHES

In addition to response rates, modeling response propensity suggested that addresses from which NHES is unlikely to get a response share similar demographic characteristics, suggesting that response bias is an issue that may affect data quality. Analyses of the demographics of low-response propensity addresses indicate that occupants of these addresses are more likely to fall into at least one of the following groups:

• Be age 35 or younger

• Rent rather than own

• Have less than a high school education

• Have an income below $15K

• Identify as Hispanic

Anyone matching any one of these characteristics is less likely to respond. The response propensity model was a regression model which showed that, holding all other variables constant, each of the above characteristics was significantly related to responding at lower rates to previous NHES administrations. Therefore, our recruitment criteria are designed to recruit parents that fall into any one of these categories. Not all conditions need to be fulfilled by individual focus group participants.

NCES hopes to improve NHES response rates and reduce response bias for the 2019 NHES by better understanding factors that motivate parents to complete (or decline to complete) the NHES questionnaire. NHES is placing special focus on understanding motivations to complete the survey among occupants in sampled addresses who are modeled to be unlikely to respond to the survey.

Design

We will conduct a total of eight in-person focus groups with parents from different states and school districts, to understand their perceptions of the study and how they would respond if selected for NHES. Although NHES topical surveys cover learning at all ages, our focus is on parents because fewer respondents complete the ECPP and PFI surveys than the ATES. We will work with recruiters at local focus group facilities to select a sample of parents and ensure diversity of geography and age of children. A recruitment screener (see attachment 1) will be used to determine eligibility of participants.

For parents agreeing to participate, we will address the following questions during the focus group:

• What value do parents perceive NHES has in education?

• Does the format of NHES influence parents’ motivations to participate?

• What information do parents use when deciding whether or not to participate in a survey like NHES? Who do they consult?

• What barriers prevent or discourage participation in NHES?

• What would minimize these barriers for parents?

• What factors might motivate parents to participate in a voluntary survey such as NHES?

• Does the U.S. Census Bureau “brand” help in motivating parents to complete the survey?

• Does the $5 bill included in the survey motivate parents to complete it?

• What are the perceived benefits of participation for schools?

• What current or potential NHES messages or materials do parents find relevant, useful, and informative?

• What would be the most effective channels, formats, and materials to use to communicate with parents?

• What other groups or people (e.g., teachers, community organizations) would be effective in influencing parents on this topic, and what would be the best way to reach those influencers?

Using the suggestions and information collected from these conversations, we will incorporate the parents’ feedback into the development of materials and strategies for NHES recruitment. The following materials, which are proposed for use in NHES, will be presented to parents during the focus groups to elicit feedback and suggestions (see attachment 2 for copies of all materials).

Materials for parents’ review (to be discussed during the focus group)

• NHES screener cover letter (attachment 2-B)

• NHES screener survey - English (attachment 2-C)

• NHES screener survey – Spanish (attachment 2-D)

A trained researcher working from a moderator’s guide (see attachment 2-A) will conduct the focus groups.

Each focus group session will include 8-10 parents—a number that allows for in-depth collection of information. Topics of the discussion will focus on identifying the benefits parents associate with NHES participation, the barriers they perceive to participating, which NHES advance materials they believe would be most and least useful, and their suggestions for factors that may increase response rates for NHES. The focus group sessions will last approximately 90 minutes.

During the focus groups, using the moderator’s guide (attachment 2-A), a moderator will lead the participating parents through a discussion, and one to two Hager Sharp team members will observe and take notes on the participants’ comments and suggestions, which they will later compile into a summary report. The session will also be video and audio recorded. The recordings will be destroyed as soon as the final report is approved, and no personally identifiable information will be included in the report.

Recruiting and Paying Respondents

Hager Sharp will work with local focus group facilities in the Richmond, VA area (Alan Newman Research), the suburban Philadelphia, PA area (Focus Pointe), and the Denver, CO area (Fieldwork Denver) to recruit for and host the in-person focus groups. Focus group facility staff will be contacting people from their databases to look for respondents who meet the eligibility criteria. The recruitment screener in Attachment 1 will be administered to ensure that those contacted through databases meet eligibility criteria. No advertisements or email notices will be utilized.

Eligibility criteria for focus group participants are based on the key regression model variables that were most salient for recruitment efforts, with Spanish-speakers being grouped together for language continuity. In order to meet realistic recruitment goals, some criteria were adapted from the regression model variables. For example, parents of children who are 35 or younger may be too difficult to recruit under cost and schedule constraints, so the criteria were relaxed to parents under the age of 40. Similarly, households with incomes below $15,000 may be too difficult to recruit under cost and schedule constraints, so the criteria were relaxed to households with incomes below $40,000. The recruitment criteria consist of:

• Parent of at least one school-aged child (PFI-eligible) or young child (ECPP-eligible) in household, AND age of parent is under 40

• Household income is below $40K

• Hispanic, Spanish-speaking parents among participants for two focus groups

We will also gather information on rent/home ownership status and education levels, and will attempt to recruit as many parents who rent and who have a high school education or less as possible.

The proposed configuration of eight focus groups is as follows:

Market	Language	HH Income	Age of Parent	Race/Ethnicity	Number of Focus Groups
Richmond, VA	English	Below $40K	Under 40	Mixed	3
Suburban Philadelphia, PA (Bala Cynwyd)	English	Below $40K	Under 40	Mixed	3
Denver, CO	Spanish	Below $40K	Under 40	Hispanic	2

To assure that we are able to recruit the select types of participants who are representative of those that are least likely to take part in NHES and to thank them for their time and for completing the interview, during recruitment each parent will be offered a $75 incentive for participation.

Assurance of Confidentiality

The statement below will be presented in all written materials (e.g., letters, emails) and read at the start of the focus group sessions. Participants will also be informed that they can leave the discussion at any time.

NCES is authorized to conduct this study by the Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C., § 9543). Your participation is voluntary and all of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151).

All contractor staff working on the NHES focus groups will sign the NCES Affidavit of Nondisclosure. Personal information (e.g., name, address) will be collected for recruitment purposes, but on the data file used for analyses, respondents will be identified only by a unique study ID number assigned to each participant. Within 48 hours of respondents’ participation in the focus groups, the discussion notes will be edited, organized, and cleaned, and all identifiers will be stripped from the data set. All computer files will be password-protected and hard copies will be locked in secure locations (e.g., data will be in locked file cabinets within locked offices). Only contract staff working directly on the data analysis portion of the project will have access to the data files. Once the final report is created, all personally identifiable information will be destroyed. All presentations of data in reports will be in aggregate form, with no links to individuals.

A consent form will be collected for all participants in the focus groups. The consent form is included in attachment 1.

Estimate of Hour Burden

Recruitment of parents is estimated to take about 10 minutes per parent, and we anticipate needing to contact up to 240 parents to form the desired focus groups of 80 diverse participants. The focus groups will take approximately 90 minutes to conduct. There is no cost to participants beyond the participation burden time. Table 1 provides the burden estimates for this study.

Table 1. Burden estimates for NHES:2019 focus groups with low-response propensity parents

Respondent group	Number of respondents	Number of responses	Burden hours per respondent	Total burden hours
Parents – Recruitment	240	240	0.167	40
Parents ‒ Focus Group Participation	80*	80	1.5	120
Total Burden	240	320		160

*Subset of initial contact group, not double counted in the total number of respondents.

Estimate of Cost Burden

There is no direct cost to respondents.

Project Schedule

The schedule of activities for the NHES focus groups is provided in table 2.

Table 2. Schedule for the NHES:2019 focus groups with low-response propensity parents

Activity	Tasks	Date ranges
Data collection	Recruit participants	Upon OMB clearance: March, 2017
	Conduct in-person focus groups	March-May, 2017
Analysis	Analyze data and produce report	by June, 2017

Cost to the Federal Government

The estimated cost to prepare for, administer, and report the results of the NHES focus groups is approximately $64,000. This cost includes salaried labor for contractor staff and other direct costs associated with organization of the sessions.