NHES 2014 Feasibility Study Memo - revised

Request for Clearance for the

2014 National Household Education Survey (NHES)

After-School Programs and Activities Survey (ASPA) and Adult Training and Education Survey (ATES)

Feasibility Test

approved under OMB# 1850-0803 v.85

burden revised under OMB# 1850-0803 v.93

original version: September 30, 2013

approved on: October 22, 2013

burden revised on: December 16, 2013

TABLE OF CONTENTS

Page

JUSTIFICATION 1

A.1 Circumstances Necessitating Collection of Information 4

A.2 Purposes and Uses of the Data 5

A.3 Use of Improved Information Technology 5

A.4 Efforts to Identify Duplication 5

A.5 Consultations Outside the Agency 6

A.6 Payments to Respondents 8

A.7 Assurance of Confidentiality 8

A.8 Sensitive Questions 8

A.9 Estimated Response Burden 10

A.10 Annualized Cost to Respondents 10

A.11 Annualized Cost to the Federal Government 10

A.12 Publication Plans and Project Schedule 10

A.13 Approval for Not Displaying the Expiration Date for OMB Approval 11

A.14 Exceptions to the Certification Statement 11

B.1 Statistical Design and Estimation 11

B.2 Survey Procedures 15

B.3 Methods for Maximizing Response Rates 17

B.4 Individuals Responsible for Study Design and Performance 18

References 18

List of Tables

Table Page

1 Estimated response burden for NHES Feasibility Study 10

2 Estimates of required sample size to detect a difference between two estimated proportions of 3 or 5 percentage points 12

3 Expected percentage of households with eligible children or adults, by sampling domain 13

4 Expected numbers sampled and expected numbers of completed screeners and topical surveys in the NHES-FS 13

List of Exhibits and Figures

Exhibit

1 Surveys conducted under the National Household Education Surveys Program, by years administered: 1991 through 2012 3

2 Number of questionnaire items by survey section and adult topical survey type 4

3 NHES Feasibility Test schedule of major activities 11

Figure

1 Screener Data Collection 19

2 Topical Mailing Plan – ASPA 20

3 Topical Mailing Plan – ATES 21

3 Topical Mailing Plan – ASPA/ATES 22

JUSTIFICATION

Request for Clearance

In 2008, the National Household Education Surveys program (NHES) began a redesign effort to convert from a system of landline random digit dial (RDD) surveys to a self-administered mail survey using an address-based sample (ABS). This redesign was prompted by declines in response rates to the telephone survey and concerns about population coverage using the landline telephone frame (due to increasing conversion to cellular-only coverage). The goals of the redesign effort were to develop and assess approaches to collecting important information on education topics from households while improving response rates and coverage from the previous design. A feasibility test of the new design was conducted in 2009 followed by a field test in 2011. The field test results helped to inform the final design of a full-scale NHES in 2012 (OMB# 1850-0768 v.9).

During the same period of time, NCES began supporting developmental work on new questionnaire items for federal household surveys on work-related education, training, and credentials for adults and out-of-school youth. The Interagency Working Group on Expanded Measures of Enrollment and Attainment (GEMEnA) is a collaboration among federal statistical agencies established by the OMB Office of Statistical and Science Policy, the Council of Economic Advisors, and the Under Secretary of Education to improve federal household statistics on the attainment of non-degree credentials such as industry-recognized certifications, occupational licenses, and educational certificates. In 2012, GEMEnA’s commission expanded to include the development of new and revised measures of enrollment or participation in education and training for work. One of GEMEnA’s roles is to guide NCES’s development of a new household survey on these topics to support research and policy analysis. NCES conducted two pilot studies (OMB# 1850-0803), first a two-stage telephone survey (42% response rate) and then a single-stage self-administered mail survey (69% response rate).

One of NCES’s goals from the beginning of the GEMEnA project was to see whether it would be feasible to eventually incorporate a revised survey on adult credentials back into the NHES. The NHES redesign work had resulted in a delay of the next full-scale NHES to 2015, beyond its typical two-year cycle, which meant that the GEMEnA work was able to proceed to the point where it will be possible to test instruments, operations, and processing for both an adult and a child survey in January 2014. This package, therefore, requests clearance for a Feasibility Study testing the integration of an adult topical survey into NHES mail operations and processing. The Feasibility Study will also include several experiments to inform the final design of a planned 2015 full-scale NHES containing the ATES and the two child-focused surveys fielded in 2012.

The NHES Feasibility Study (NHES-FS) will include two topical surveys, the Adult Training and Education Survey (ATES)¹ and the After-school Programs and Activities Survey (ASPA)². Adults ages 16 to 65 who are not enrolled in grade 12 or below will be eligible for the ATES topical survey and children in kindergarten through eighth grade who are ages 15 or younger will be eligible for the ASPA topical survey. The NHES Feasibility Study will screen 60,000 households. Of these households, it is expected that approximately 37,914 will return a screener survey. From these completed screeners, it is expected that approximately 24,451 households will contain an eligible adult but no eligible children; approximately 7,044 will contain an eligible adult and an eligible child; and approximately 156 will contain an eligible child but no eligible adults (for example, children who live with grandparents above age 65). Combining the ASPA and ATES provides greater efficiency in the data collection and reduces overall national burden by maximizing the use of a single household sample draw.

NHES Background

NCES developed NHES to complement its institutional surveys as the principal mechanism for addressing education topics appropriate for households rather than establishments. Such topics cover a wide range of issues, including early childhood care and education, children’s readiness for school, parent perceptions of school safety and discipline, before- and after-school activities of school-age children, participation in adult and continuing education, parent involvement in education, school choice, homeschooling, and civic involvement. The NHES consists of a series of rotating surveys using a two-stage design in which a household screener collects household membership and key characteristics for sampling and then appropriate topical survey(s) are mailed to sample members.

Data from the NHES are used to provide national cross-sectional estimates on populations of special interest to education researchers and policymakers. For surveys about children, the population of interest is defined by age or grade in school, or both, depending on the particular survey topic and research questions. For surveys of adults, the population of interest is those ages 16 to 65 who are not enrolled in grade 12 or below, excluding those on active duty military service and those who are institutionalized. The NHES targets these populations using specific screening and sampling procedures.

The NHES design also yields estimates for subgroups of interest for each child and adult survey, as defined by age (or grade for children), education level for adults, Hispanic origin, and racial background for all populations of interest³. In addition to providing cross-sectional estimates, the NHES is also designed to produce estimates from repeated cross sections to measure changes over time in key statistics.

The last NHES data collection prior to the redesign was conducted in 2007 and included the Parent and Family Involvement in Education Survey (PFI) and the School Readiness Survey (SR)⁴. The NHES:2012 was the first full-scale data collection using an addressed-based sample and a self-administered questionnaire and included the PFI survey and Early Childhood Program Participation Survey (ECPP). The overall screener plus topical response rate was approximately 58 percent for both the PFI and the ECPP in 2012, compared to the 2007 overall response rate of 39-41% (depending on the survey). The results suggest that the new methodology has the ability to address the response rate and coverage issues identified in the 2007 data collection.

The ATES will reintroduce an adult survey component to the NHES. ATES will provide a means to investigate issues related to education, training, and credentials for work that cannot be adequately studied through the Center’s institution-based data collection efforts. While the feasibility test is not being conducted to make survey estimates, the data gathered will allow for examination of the empirical properties of potential survey measures.

Survey data from the NHES have been used for a large number of descriptive and analytic reports and articles, including NCES publications, publications of other Federal agencies, policy analyses, theses and dissertations, conference papers, and journal articles. A list of NHES publications issued by NCES can be found on the NHES website, http://nces.ed.gov/nhes.

NHES Feasibility Study Surveys (NHES-FS)

As shown in exhibit 1, each administration of the NHES has included more than one topical survey. However, under the new mail data collection methodology, households have only been screened for and topical surveys have only focused on children. The NHES-FS will include two topical surveys: the ATES and the ASPA. To develop the ATES survey, NCES has been redeveloping and testing new survey content since 2009, including pilot testing, expert review, cognitive testing, and focus groups. A summary of the cognitive interviews conducted during August 2013 is provided in appendix D. Before adding an adult education survey back into the NHES it is important to test the feasibility of using a mail survey to screen households for both adults and children, and to test different approaches to collecting topical data from households (e.g., sampling either an adult or a child from the same household for topical follow up compared to sampling both and adult and a child for follow up).

Exhibit 1.  Surveys conducted under the National Household Education Surveys Program, by years administered: 1991 through 2012

Topical survey	NHES survey administration
	1991	1993	1995	1996	19991	2001	2003	2005	2007	2012
Early childhood education/ program participation	√		√		√	√		√		√
Adult education	√		√		√	√	√	√
School readiness		√			√				√
School safety and discipline		√
Parent and family involvement in education				√	√		√		√	√
Civic involvement				√	√
After-school programs and    activities			√2		√	√3		√
Household library use				√
Homeschooling					√		√		√	√

¹ The NHES:1999 was a special end-of-decade administration that measured key indicators from the surveys fielded during the 1990s.

² The After-School Programs and Activities Survey of the NHES:1995 only asked about children in first through third grades.

³ The After-School Programs and Activities Survey of the NHES:2001 also included items on before-school programs.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 1991–2012.

The ASPA is a repeated administration of a topic shown in the exhibit above but has been adapted for self-administration by mail and its content has been revised based on expert review and cognitive testing. A summary of the cognitive interview findings is provided in appendix D. Item justifications for questions included in the ASPA are provided in appendix E.

The full ATES questionnaire contains several sections asking respondents about credential attainment and others asking about work-related education and training. Because of concerns about respondent burden and to maximize response rates, GEMEnA has recommended that NCES test a separate Credentials for Work Survey (CWS), which would include a small number of core items about participation in education and training but eliminate most of the detail. NCES proposes to conduct a split panel experiment comparing unit and item level response rates for the full ATES versus the shorter CWS. If the shorter survey garners substantially higher response rates, NCES may choose, in the future, to test a separate Education and Training Survey. The following exhibit compares the number of questionnaire items in the full ATES and in each separate survey section.

Exhibit 2. Number of questionnaire items by survey section and adult topical survey type

Survey section	ATES	CWS	Education & Training Survey*
Educational attainment	2	2	2
Certification and licensures	17	17	2
Educational certificates	16	16	2
Apprenticeships	12	12	2
College and other classes	13	3	13
Training for work	16	2	16
Employment	21	17	21
Background	9	9	9
TOTAL	106	78	67

* Not part of the NHES-FS and this submission.

The instruments are provided in appendices B (screeners) and C (topical questionnaires).

• The Adult Training and Education Survey (ATES) provides a means to investigate issues related to education, training, and credentials for work that cannot be adequately studied through the Center’s institution-based data collection efforts. It targets non-institutionalized adults in the United States ages 16 to 65, who are not enrolled at grade 12 or below. The ATES will collect information on educational attainment, prevalence and characteristics of certifications and licenses and their holders, prevalence and characteristics of educational certificates and certificate holders, apprenticeships, and non-credit work-related training. It will also collect detailed employment information.

• The Credentials for Work Survey (CWS) consists of a subset of items from the ATES as shown in Exhibit 2 and focuses primarily on credential attainment. It will also include core education and training items from the ATES as well as the standard employment and background items.

• The After-school Programs and Activities Survey (ASPA), previously conducted in 1995, 2001, and 2005, surveys families of children and youth enrolled in kindergarten through 8th grade, with an age limit of 15 years, and addresses specific center- or school-based programs that children participate in during after-school hours and the characteristics of these programs. The ASPA also collects information on care that children receive after school from relatives and nonrelatives, and formal activities that children participate in after school. Additional topics addressed in the ASPA include factors related to the parental selection of programs and activities; characteristics of the child’s school; child health and disability; and child, parent, and household characteristics.

A.1 Circumstances Necessitating Collection of Information

The Education Sciences Reform Act of 2002 (ESRA 2002: 20 U.S. Code § 9543) defines the legislative mission of NCES to collect, report, analyze, and disseminate statistical data related to education in the United States and in other nations.

The NHES is specifically designed to support this mission by providing a means to investigate education issues that cannot be adequately studied through the Center’s institution-based data collection efforts. For example, school-age children participate in many types of informal or formal care settings and activities after school. There is no available sample frame that includes all of these types of care arrangements and activities. Likewise, although attaining a postsecondary credential has become increasingly important for securing opportunities to get high-return jobs in the United States in the 21st century, NCES has traditionally only collected data on postsecondary certificates and degrees awarded through credit-bearing instruction in institutions of higher education that participate in Title IV federal student aid programs. These comprise only a portion of subbaccalaureate education and training American adults seek and complete to learn the skills they need to find and keep good-paying jobs.

It is efficient and economical to interview parents about their children’s participation in after-school programs and activities through a household-based approach rather than incurring the cost and nonresponse involved in enlisting schools, obtaining lists of parents, and sampling parents from those lists. Similarly, it is also most efficient to interview adults through a household-based approach rather than trying to obtain lists from a myriad of private credential awarding bodies. Also, the household approach allows for capture of adults who do not participate in training or have a credential, providing a point of comparison.

There are also methodological reasons necessitating the Feasibility Study. Response rates for the combined study were low for the adult education component in past NHES administrations conducted by telephone. Therefore, it is necessary to test the feasibility of the combined mail data collection in order to examine trade-offs between efficiency, response rates, and data quality.

A.2 Purposes and Uses of the Data

The data collected in the NHES-FS will be used to evaluate whether or not adult and child topical surveys can be jointly administered using the new NHES self-administered mail survey methodology. The NHES-FS data will also be used to evaluate item performance in the topical surveys. They will not be used to generate official national estimates of the population. Information gathered from this feasibility study will be used to make recommendations for methodological approaches and survey measures.

A.3 Use of Improved Information Technology

The data for the NHES Feasibility Study will be collected for NCES by the United States Bureau of the Census using three complementary survey systems - (1) Amgraf One Form Plus, (2) Docuprint, and (3) integrated Computer Assisted Data Entry (iCADE), chosen for their efficiency and accuracy in the data collection process.

• Forms Design. Questionnaires will be created using Amgraf One Form Plus. Completed hardcopy forms can be processed by iCADE to capture responses through optical mark recognition (OMR) and keying from image (KFI). Questionnaires will be printed, trimmed, and stitched through an in-house print on-demand process using a Docuprint system which allows personalization, and the ability to tailor items to each specific respondent. The data from the questionnaires will be captured by the iCADE technology/software, which automatically extracts all check box entries (OMR) and captures and displays an image of all other entries to an operator for KFI.

• Image Preprocessing. iCADE applies image preprocessing to the forms in their image format in order to correct any skewing at the time of scanning, and the iCADE software performs registration to align the individual questionnaire page template with the appropriate scanned image. The scanner despeckles the image to remove unwanted pixels.

• Data Capture. iCADE reads the form image files, checks the presence of data, processes all check box fields through OMR, and presents an image of the handwritten fields to an operator for KFI.

• Verification. Extracted KFI data are subject to 100% field validation according to project specifications. If a data value violates validation rules, the value is flagged for review by verifiers who interactively review the images and the corresponding extracted data, and resolve validation errors.

• Archiving. Images will be scanned and archived to magnetic storage located on a secured server in case they are needed later. This eliminates the need to save paper copies of the completed questionnaires.

A.4 Efforts to Identify Duplication

ATES. Senior policy officials in the Departments of Education, Commerce, and Labor, foundations including the Gates Foundation and Lumina, and research organizations such as the Georgetown Center for Education and the Workforce have recognized that there is a lack of valid statistical information on prevalence of industry-recognized certifications and education certificates, and called for the development of new data sources. A series of meetings during the fall of 2009 launched a broad effort to begin to define and enumerate these credentials. NCES conducted a review of research literature and data collections since the work of a previous Interagency Committee in 2000, from which NCES developed a bank of existing survey items on certifications (completed 11/2009) and education certificates (completed 1/2010). This research found no surveys that adequately capture comprehensive data on adult training, education, and credentials.

ASPA. A panel of experts was convened in June 2013 to discuss and review the content and contributions of the ASPA survey. The experts identified a need for information about after-school activities that was not collected in a child-care framework. In particular, they felt there was a need for information about after-school activities that contribute to children’s learning. The ASPA survey was designed with these needs in mind. Limitations of other surveys are as follows: child care surveys such as the National Survey of Early Childhood Education (NSECE) tend to focus on low-income children and preschool-aged children; other surveys of school-aged children conducted by NCES such as the High School Longitudinal Study (HSLS), Early Childhood Longitudinal Study (ECLS); and Middle Grades Longitudinal Study (MGLS) are longitudinal in nature and therefore do not provide repeated, cross-sectional data across a wide range of ages; many studies of after-school programs are evaluations of particular types of programs, such as evaluations of the 21st Century Community Learning Center program, or focus on programs in a particular area, such as the evaluation LA’s Better Educated Students for Tomorrow (BEST) program.

A.5 Consultations Outside the Agency

A Technical Review Panel (TRP) comprising leading experts in survey methodology was established to provide input to the redesign of the NHES system. Most members of the panel met in February 2010 to discuss the proposed design for the field test, and their comments and suggestions led to changes reflected in this submission.

Technical Review Panel Participants and Their Affiliation at the Time of TRP Recruitment

Nancy Bates

U.S. Census Bureau

649 A. St. N.E.

Washington, DC 20002

Tel: 301-763-5248

E-mail: [email protected]

Paul Beatty

National Center for Health Statistics

Division of Health Care Statistics

3311 Toledo Road,

Hyattsville, MD 20782

Tel. 301-458-4090

E-mail: [email protected]

Johnny Blair

Survey Sampling and Methodology

Abt Associates Inc.

4550 Montgomery Avenue

Bethesda, MD 20814-3343

Tel: 301-634-1825

E-mail: [email protected]

Stephen Blumberg

National Center for Health Statistics

3311 Toledo Road

Hyattsville, MD 20782

Tel.301-458-4107

E-mail: [email protected]

Mick Couper

Survey Research Center

University of Michigan

ISR, 426 Thompson Street

Ann Arbor, MI 48104

Tel: 734-647-3577

E-mail: [email protected]

Don Dillman

Social and Economic Sciences Research Center, Professor

Washington State University

133 Wilson Hall

Pullman, WA 99164-4014

Tel: 509-335-1511

E-mail: [email protected]

Robert Groves

Survey Research Center, Institute for Social Research

University of Michigan

426 Thompson Street

Ann Arbor, MI 48106-1248

Tel: 734-764-8365

E-mail: [email protected]

Scott Keeter

Pew Research Center

1615 L. St. NW. Suite 700

Washington, DC 20036

Tel: 202-419-4362

E-mail: [email protected]

Kristen Olsen

Survey Research and Methodology

University of Nebraska-Lincoln

201 N. 13th St.

Lincoln, NE 68588-0241

Tel: 402-472-7737

E-mail: [email protected]

Roger Tourangeau

Joint Program in Survey Methodology

University of Maryland

1218 LeFrak Hall, University of Maryland

College Park, MD 20742

Tel: 240-595-0057

E-mail: [email protected]

Gordon Willis

Division of Cancer Control / Population Sciences

National Cancer Institute

6130 Executive Blvd, MSC 7344, EPN 4005

Bethesda, MD 20892-7344

Tel: 301-594-6652

E-mail: [email protected]

Content Area Experts

The content of the ASPA topical survey repeats, to a great extent, the content developed for previous NHES administrations. As a result, the ASPA survey reflects the cumulative input of many experts in the field and past NHES Technical Review Panels. In order to ensure that the ASPA addresses important issues in the topical areas of interest and incorporates important emerging issues, the design phase of the study included consultations with experts in the substantive areas addressed in the surveys. These experts included persons in academe and research organizations.

Substantive Experts: ASPA and Their Affiliation at the Time of TRP Recruitment

Nancy L. Deutsch - University of Virginia

Department of Leadership, Foundations, and Policy

Jean Grossman - Princeton University

Office of Population Research

Gwynn Hughes - Charles Stewart Mott Foundation

Terry Peterson - Afterschool and Community Learning Network

Jacquelynne Eccles - University of Michigan

Institute for Social Research

Georgia Hall - Wellesley College

National Institute on Out-of-School Time (NIOST)

Robert Halpern - Erikson Institute

Larry Suter - University of Michigan

Deborah Vandell - University of California, Irvine

As noted above, the ATES component of the feasibility study is part of ongoing work guided by GEMEnA, which has met monthly since October 2009 and consists of senior staff from the Bureau of the Census, the Bureau of Labor Statistics, the Council of Economic Advisors, the National Center for Education Statistics, the National Center for Science and Engineering Statistics, the Office of Statistical and Science Policy (OMB), and the Office of the Under Secretary of Education.

Substantive Experts: GEMEnA Member Agency Representatives

Census Bureau

Bob Kominski

Stephanie Ewert

Bureau of Labor Statistics

Dori Allard

Harley Frazis

National Center for Science and Engineering Statistics

Dan Foley

John Finamore

OMB Office of Statistical and Science Policy

Shelly Martinez

Department of Education – Office of the Under Secretary

Jon O’Bergh

National Center for Education Statistics

Sharon Boivin

Lisa Hudson

Kashka Kubzdela

Matthew Soldner

Sarah Crissey

Andy Zukerberg

A.6 Payments to Respondents

The NHES:2003 included an extensive experiment in the use of small cash incentives to improve unit response. The experiment demonstrated that gains in respondent cooperation could be realized with relatively modest cash incentives (Brick et al. 2006). Such incentives were used in NHES:2005 and NHES:2007. The NHES:2011 Field Test included an incentive experiment at the screener level testing the effect of including a $2 cash incentive on response rates compared to a $5 cash incentive in the initial screener mailing. The $5 screener was associated with higher response rates than the $2 incentive, so the $5 incentive was used in the NHES:2012. We will continue with this approach in the NHES Feasibility Test and use a $5 cash incentive in the first screener questionnaire mailing.

The NHES:2012 included an incentive experiment at the topical level to further refine an optimal strategy for the use of incentives in the NHES. For those households in which a child was selected as the subject of an ECPP or PFI questionnaire, cases that responded to the first or second mailing of the screener received a $5 cash incentive with the initial topical survey mailing. Evidence from the 2011 Field Test indicated that topical response rates could benefit significantly by providing later screener respondents with a larger topical incentive. To confirm this finding, we subsampled late screener respondents (those responding to the 3^rd or 4^th questionnaire mailing) to receive either a $5 or $15 cash incentive with their first topical survey mailing. The results from the NHES:2012 indicate that among later screener responders, the $15 incentive was associated with higher response rates compared to the $5 incentive. Based on these findings, we will use the same strategy in the NHES Feasibility Test and send a $5 cash incentive in the initial topical mailing to cases that responded to the first or second screener mailing and a $15 cash incentive in the initial topical mailing to any cases that responded later than three days after the third screener mailing.

A.7 Assurance of Confidentiality

Respondents will be informed of the voluntary nature of the survey and of the confidentiality provision in the initial cover letter and on the questionnaires, stating that their responses may be used for statistical purposes only and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002), 20 U.S. Code § 9573].

Additionally, all staff members and subcontractors working on the NHES and having access to the data are required to sign the NCES Affidavit of Nondisclosure. Notarized affidavits are kept on file by the contractor and submitted to NCES quarterly. In addition, all contractor staff members who have access to confidential data and work on the project more than 30 days are required to have a federal background check.

A.8 Sensitive Questions

The NHES is a voluntary survey, and no persons are required to respond to it. In addition, respondents may decline to answer any question in the survey. Respondents are informed of the voluntary nature of the survey in the cover letter that accompanies the questionnaire, as well as on the actual questionnaire. At the same time, some items in the surveys may be considered sensitive by some respondents:

ATES: The ATES survey currently asks a question about income that may be considered sensitive:

• Personal earnings in the past year.

A measure of income is important because education attainment is statistically associated with income, and the empirical properties of the survey measures may differ for people with different income levels. The American Community Survey (ACS) was the source for most of the ATES employment and background items. Item response rates for income questions were reasonably high in the 2013 pilot test. The item response rate for personal earnings was 96.4.

ASPA: Child development and education experts consider economic disadvantage and children’s disabilities to be important factors in children’s school experiences and their activities outside of school. As a result, the ASPA survey contains measures of these characteristics, including:

• Household income;

• Receipt of public assistance in the form of Temporary Assistance to Needy Families (TANF), food stamps, and the Women, Infants, and Children program (WIC); and

• Children’s disability conditions.

Measures of household income and government assistance are important because access to after-school programs and activities by children at risk and children from families of different socioeconomic backgrounds is of interest to policymakers, child development specialists, and educators. These items are important to identifying children at risk and have been administered successfully in previous NHES studies. Respondents are also asked the age at which they first became a parent to a child. This may be sensitive for parents in some situations.

The 2012 response rates for these items were very high. For total household income, the 2012 PFI survey had an item response rate of 95.4 percent. Item response rates for receipt of public assistance were also high: for Temporary Assistance to Needy Families, 97.9 percent; for the Women, Infants, and Children Program, 97.7 percent; and for Food Stamps, 98.4 percent. In the 2012 mail survey, it is not possible to examine item missing data for child disability because of the multiple response, list format of the question. Missing data may indicate either unreported data or that the child does not have a disability. However, in prior NHES collections, response to this item was high; in the 2007 PFI, the item response rates were over 99 percent. In the 2012 PFI, the item response rate for age at which the child’s parent first became a parent to any child was 96.2 for the first parent reported and 96.0 for the second parent reported.

In order to understand families’ and children’s access to after-school programs and activities, the ASPA survey includes questions about assistance that the family receives to pay for:

• Formal after-school programs; and

• Organized after-school activities.

Similar questions were asked in the NHES:2012 ECPP survey. The response rates for a similar question asking families about the receipt of assistance paying for center-based care in the ECPP was 97.5 percent.

The ASPA survey also includes items concerning children’s school performance and difficulties in school. Among these are:

• Children’s school performance and difficulties, including school grades, suspensions, and expulsions; and

• Identification of children’s schools.

Items concerning school performance and difficulty are important to the ASPA survey as correlates of children’s participation in various types of after-school programs and activities. These items were asked in the NHES:2012 PFI and item response rates for these items were high: 99.0 percent for children’s grades, 97.6 percent for out-of-school suspension, and 97.5 percent for expulsion.

Another element of the surveys that may be sensitive to some parents is the identification of children’s schools. This feature allows analysts to link the NHES data to other NCES datasets containing additional information about schools, greatly enhancing the ability to examine the relationships between students’ and families’ experiences and the characteristics of schools. The item response rate for the identification of the child’s school was 97.0 percent in 2012.

A.9 Estimated Response Burden

The response burden per instrument and the total response burden are shown in table 1. The administration times for the screener, ATES, and ASPA questionnaires are based on practice administrations and past experience. The expected number of respondents and number of responses are based on the expected numbers of completed surveys of each type, discussed in section B.1.3. The hourly rate of $21.50 is based on the average for all civilian workers from the March 2013 National Compensation Survey (http://www.bls.gov/news.release/ecec.t02.htm). There are no other costs to respondents. There are also no recordkeeping requirements associated with NHES. For the NHES-FS, a total of 9,453 burden hours are anticipated, resulting in a burden cost to respondents of approximately $203,240.

Revised burden table for NHES Feasibility Study

Interview forms	Estimated time (minutes)	Number sampled	Anticipated Response Rate	Estimated Number of respondents	Estimated Number of responses	Total time (hours)
Screener (average)	8	60,000* (53,400)	71%	37,914	37,914	5,055
ATES questionnaire – national sample	15	7,761	70%	5,433	5,433	1,358
CWS questionnaire – national sample	10	7, 761	70%	5,433	5,433	906
ATES questionnaire – seeded sample	15	2,150	70%	1,505	1,505	376
CWS questionnaire – seeded sample	10	2,150	70%	1,505	1,505	251
ASPA questionnaire	20	6,026	75%	4,520	4,520	1,507
Study Total		60,000		40,924	56,310	9,453

* Approximately 11% of addresses will be returned by USPS as invalid, reducing the final sample size to 53,400 addresses.

A.10 Annualized Cost to Respondents

There are no costs beyond those presented in section A.11.

A.11 Annualized Cost to the Federal Government

The total cost of NHES Feasibility Study to the government is approximately 3.2 million dollars over a period of 20 months. This includes all direct and indirect costs of the design, data collection, analysis, and reporting phases of the study, as well as the delivery of data sets to NCES.

A.12 Publication Plans and Project Schedule

The primary objectives of the NHES-FS are to evaluate whether and how the ATES can be incorporated into the NHES data collection operations, and to evaluate item performance on the ATES and ASPA topical surveys. Future NHES data collections will implement these findings and produce datasets, statistics, and reports. The following are the planned outcomes of the NHES-FS:

• Operational issues: Any events or issues that result in a delay to the implementation of each event within the Feasibility Test schedule. Since many of these operational tasks will be evaluated, each process will need to be documented so procedures are in place for full scale data collection in 2015.

• Respondent feedback: Any feedback received from a respondent will be evaluated for reaction to methods (e.g., reaction to the use of government funds for incentives, difficulty of mode, etc.).

• Response rates and components: Overall, and for each experimental manipulation/stage, response rates (both unit and item response rates, return rates, and any appropriate refusal conversion rates) will be tracked.

• Response timing: For each manipulation, time to response/questionnaire receipt will be assessed to determine the effects on the data collection schedule.

Exhibit 3 presents the schedule of project activities for NHES Feasibility Study.

Exhibit 3.  NHES Feasibility Test schedule of major activities

Task	Date of Scheduled Conduct/Completion
Survey Instruments Formatting and Printing	October-December, 2013
Data Collection Begins (advance letter mailing)	January 2, 2014
Data Collection Ends	August 20, 2014

A.13 Approval for Not Displaying the Expiration Date for OMB Approval

The OMB authorization number and expiration date will be displayed on the hard copy questionnaire.

A.14 Exceptions to the Certification Statement

There are no exceptions to the certification statement.

B.1 Statistical Design and Estimation

The NHES Feasibility Study administered in 2014 (NHES-FS) will be an address-based sample covering the 50 states and the District of Columbia and will be conducted from January through August 2014. Households will be randomly sampled as described in section B.1.1, and a screener questionnaire will be sent to each sampled household. Demographic information about household members provided on the screener will be used to determine whether anyone is eligible for the After-School Programs and Activities (ASPA) survey, the Adult Training and Education Survey (ATES), or both. In order to limit respondent burden, regardless of the number of eligible people in a household, no more than one child and one adult per household will be sampled for the topical surveys and no more than two topical surveys will be administered in a household.

In addition, the study will include a seeded sample of adults with certain types of occupational credentials. Because the names and addresses of potential seeded sample members were solicited from credentialing organizations, they will receive a personally-addressed copy of the topical questionnaire. They are not part of the household sample and are not included in the screener operation for NHES-FS.

The target population for the ASPA survey consists of children ages 3-15 (as of December 31, 2013) who are enrolled in kindergarten through eighth grade. The target population for the ATES survey includes adults ages 16-65 who are not enrolled in grade 12 or below.

B.1.1 Sampling Households

A nationally representative sample of 60,000 addresses will be used. The sample will be drawn in a single stage from a file of residential addresses maintained by a vendor, Marketing Systems Group (MSG), updated with the United States Postal Service (USPS) Computerized Delivery Sequence File (CDSF).

The NHES-FS is designed to meet precision requirements that allow for comparison among various experiments included in the study. The precision requirements are the ability to detect a 3 percentage point change in estimates between 60 and 80 percent. Table 2 shows the sample sizes needed to meet these precision requirements (alpha=.05, 1-B=.80).

Table 2. Estimates of required sample size to detect a difference between two estimated proportions of 3 or 5 percentage points

True value of P1 	3 percentage point difference		5 percentage point difference
	Value of P2 (1)	Sample size (2)	Value of P2 (1)	Sample size (2)
80 percent	77 or 83 percent	1,550	75 or 85 percent	560
60 percent	57 or 63 percent	2,310	55 or 65 percent	840

(1): the value of P₂ when the true difference is 3 or 5 percentage points.

(2): the number of completed interviews assuming one or two eligible people per household are selected for either ASPA or ATES. The sample size applies to overall estimates and for any particular subgroup of interest, e.g. Hispanic.

B.1.2 Within-Household Sampling

Eligible children and adults within households that have a completed screener will be sampled for the ASPA or ATES topical surveys. One aspect in the development of the sampling scheme for the NHES-FS is to determine the impact on response rates of sampling an adult and a child from the same household. To facilitate being able to make this comparison, one-third of the sample households with both an eligible adult and an eligible child will be designated to receive both ASPA and ATES, with the other two-thirds of the sample households with both an eligible adult and an eligible child receiving only one or the other. With a mail survey, the potential adverse impact of response burden is more of a concern than with a CATI instrument, since the customization that is possible with CATI is impossible or impractical with a hard-copy mail instrument. As a result, the decision for 2014 is to measure the change in response rate associated with administering two topical survey interviews per household rather than one.

Each household will be randomly pre-designated as “ASPA household,” “ATES household,” or “BOTH household”. This pre-designation will be used only when a household has members in both domains. In any household with adults and children in the eligible population for only one survey (either ASPA or ATES, but not both), one child or adult will be randomly selected in that domain. Because ASPA eligible children comprise a smaller portion of the population compared to ATES eligible adults, differential sampling will be applied to ensure a sufficient sample size for the ASPA survey. Among households with children and adults eligible for both surveys and not designated for both, approximately 75 percent will be designated to the ASPA only domain and 25 percent will be designated to the ATES only domain. If a sampled household has more than one person eligible for ASPA or ATES, the sampled person will be determined randomly.

B.1.3 Expected Yield

As described above, the initial sample will consist of approximately 60,000 addresses. An expected screener response rate of 71 percent and an address ineligibility⁵ rate of 11 percent are assumed, based on results from the NHES:2012. Under these assumptions, the expected number of completed screeners is 37,914.

The ASPA and ATES topical surveys will be administered to households with completed screeners that have eligible children or adults. For the NHES Feasibility Study, we expect to achieve a percentage of households with eligible children of about 19 percent, and for eligible adults of about 83 percent. Expected estimates of the percentage of households with eligible children or adults overall and in each sampling domain are given in table 3, as well as the expected number of screened households in the nationally representative sample, based on the distribution of household composition and assuming a total of 37,914 completed screeners.

Table 3.  Expected percentage of households with eligible children or adults, by sampling domain

Household composition	Percent of households	Expected number of screened households
Households with at least one ASPA eligible child and no ATES eligible Adults	0.4	156
Households with no ASPA eligible child and at least one ATES eligible adult	64.5	24,451
Households with at least one ASPA eligible child and at least one ATES eligible adult	18.6	7,044
Total households with eligible children or adults	83.5	31,653

NOTE: The distribution in this table assumes 37,914 screened households. Detail may not sum to totals because of rounding.

SOURCE: Annual Social and Economic Supplement to the Current Population Survey, Bureau of Labor Statistics, U.S. Department of Labor, 2011.

Table 4 summarizes the expected numbers of completed interviews for the NHES-FS. These numbers take into account within-household sampling. A topical response rate of 75 percent is assumed for ASPA based on results from the NHES:2012, and 70 percent is assumed for ATES based on unpublished test results. Based on an initial sample of 60,000 addresses, the expected number of completed screener questionnaires is 37,914. Of these, we expect to have approximately 6,000 children selected for the ASPA questionnaire and approximately 28,000 adults selected for the ATES questionnaire. In addition, a seeded sample of 4,300 adults believed to have industry-recognized certifications, state licenses, or educational certificates will be selected for the ATES questionnaire.

Table 4.  Expected numbers sampled and expected numbers of completed screeners and topical surveys in the NHES-FS

Survey	Expected number sampled	Expected number of completed interviews
Household screeners	60,000	37,914
ASPA	6,026	4,520
ATES	27,974	19,582
ATES Seeded Sample	4,300	3,010

B.1.4 Estimation Procedures

The data sets from the NHES-FS will have weights assigned to facilitate estimation of nationally representative statistics. All households responding to the screener will be assigned weights based on their probability of selection and a non-response adjustment, making them representative of the household population. All individuals responding to the topical questionnaires will have a record with a person weight designed such that the complete data set represents the target population. The seeded sample cases from the ATES questionnaire will be excluded from weighting process since they were not selected using a probabilistic sampling procedure.

The estimation weights for the NHES-FS surveys will be formed in stages. The first stage is the creation of a base weight for the household, which is the inverse of the probability of selection of the address. The second stage is a screener nonresponse adjustment to be performed based on characteristics available on the frame and discussed below. These weights may be used to produce national household-level estimates to aid in evaluating the performance of questionnaire items.

The household-level weights are the base weights for the person-level weights. For each completed topical questionnaire, the person-level weights also undergo a series of adjustments. The first stage is the adjustment of these weights for the probability of selecting the person within the household. The second stage is the adjustment of the weights for topical survey nonresponse to be performed based on characteristics available on the frame and discussed below. The third stage is the raking adjustment of the weights to Census Bureau estimates of the target population. The variables that may be used for raking at the person level include race and ethnicity of the sampled person, household income, home tenure (own/rent/other), region, age, grade of enrollment, gender, family structure (one parent or two parent), and highest educational attainment in household. These variables (e.g., family structure) have been shown to be associated with response rates. The final ranked person-level weights include undercoverage adjustments as well as adjustments for nonresponse.

Standard errors of the estimates will be computed using a jackknife replication method. The replication process repeats each stage of estimation separately for each replicate. The replication method is especially useful for obtaining standard errors for statistics such as quantiles. The standard errors may be computed using the complex survey data analysis package WesVar Complex Samples Software or other software packages that use replication methods such as Stata, SAS, SUDAAN, or the AM software package. Also, PSU and STRATUM variables will be available for use in Taylor series linearization or to compute standard errors for internal analysis.

B.1.5 Nonresponse Bias Analysis

To the extent that those who respond to surveys and those who do not differ in important ways, there is a potential for nonresponse biases in estimates from survey data. The estimates from NHES-FS are subject to bias because of unit nonresponse to both the screener and the extended topical surveys, as well as nonresponse to specific items. Per NCES statistical standards, a unit-level nonresponse bias analysis will be conducted if the NHES-FS overall unit response rate (the screener response rate multiplied by the topical response rate) falls below 85 percent. Additionally, any item with an item-level response rate below 85 percent will be subject to an examination of bias due to item nonresponse.

Unit nonresponse

To identify characteristics associated with unit nonresponse, a multivariate analysis will be conducted using a categorical search algorithm called Chi-Square Automatic Interaction Detection (CHAID). CHAID begins by identifying the characteristic of the data that is the best predictor of response. Then, within the levels of that characteristic, CHAID identifies the next best predictor(s) of response, and so forth, until a tree is formed with all of the response predictors that were identified at each step. The final result is a division of the entire data set into cells by attempting to determine sequentially the cells that have the greatest discrimination with respect to the unit response rates. In other words, it divides the data set into groups so that the unit response rate within cells is as constant as possible, and the unit response rate between cells is as different as possible. Since the variables considered for use as predictors of response must be available for both respondents and nonrespondents, demographic variables from the sampling frame provided by the vendor (including household education level, household race/ethnicity, household income, number of children in the household, number of adults in the household, age of head of household, whether the household owns or rents the dwelling, and whether there is a surname and/or phone number present on the sampling frame) will be included in the CHAID analysis.

In addition to the above, the magnitude of unit nonresponse bias and the likely effectiveness of statistical adjustments in reducing that bias will be examined by comparing estimates computed using adjusted weights to those computed using unadjusted weights. The unadjusted weight is the reciprocal of the probability of selection, reflecting all stages of selection. The adjusted weight is the extended interview weight adjusted for unit nonresponse (without the raking adjustment). In this analysis, the statistical significance of differences in estimates will be investigated only for key survey estimates including, but not limited to, the following:

All surveys

• Age/grade of child or age of adult

• Census region

• Race/ethnicity

• Mother’s or adult’s employment status

• Mother’s or adult’s home language

• Educational attainment of mother/adult

• Family type

• Household income (ASPA only)

• Home ownership

Adult Training and Education Survey (ATES)

• Highest degree or level of school

• Certification or license

• Certificate

• Completed work-related training last 12 months

After School Programs and Activities (ASPA)

• Type of care arrangement

• Mean hours in care arrangement

• Whether or not there is a charge or fee for care arrangement

• Participation in learning enrichment activities

• School type

The final component of the bias analysis will include comparisons between respondent characteristics known population characteristics from extant sources including the Current Population Survey (CPS) and the American Community Survey (ACS). Additionally, for substantive variables, weighted estimates will be compared to prior NHES administrations if available. While differences between NHES-FS estimates and those from external sources as well as prior NHES administrations could be attributable to factors other than bias, differences will be examined in order to confirm the reasonableness of the 2014 estimates.

Item nonresponse

In order to examine item nonresponse, all items with response rates below 85 percent will be listed. Alternative sets of imputed values will be generated by imposing extreme assumptions on the item nonrespondents. For most items, two new sets of imputed values—one based on a “low” assumption and one based on a “high” assumption—will be created. For most continuous variables, a “low” imputed value variable will be created by resetting imputed values to the value at the 5^th percentile of the original distribution; a “high” imputed value variable will be created by resetting imputed values to the value at the 95^th percentile of the original distribution. For dichotomous and most polytomous variables, a “low” imputed value variable will be created by resetting imputed values to the lowest value in the original distribution, and a “high” imputed value variable will be created by resetting imputed values to the highest value in the original distribution. Both the “low” imputed value variable distributions and the “high” imputed value variable distributions will be compared to the unimputed distributions. This analysis helps to place bounds on the potential for item nonresponse bias through the use of “worst case” scenarios.

B.2 Survey Procedures

This section describes the data collection procedures to be used in the NHES-FS:2014. These procedures represent a combination of best practices to maximize response rates based on findings from the NHES:2012 and NATES within the Agency’s budget constraints. The NHES is a two-phase self-administered survey. In the first phase, households are screened to determine if they have eligible adults or children. In the second phase, a detailed topical questionnaire is sent to households with the eligible adults and/or children. The NHES employs multiple contacts with households to maximize response. These include an advance letter and up to four questionnaire mailings for both the screener and the topical surveys. In addition, households will receive one reminder postcard after the initial mailing of a screener or topical and an automated phone call reminder for nonrespondents in conjunction with the third mailing. To further improve the design, a small number of experiments are proposed. These experiments are outlined below. Respondent contact materials are provided in appendix A and questionnaires are provided in appendices B (screeners) and C (topical questionnaires).

Screener Procedures

Figure 1 presents a flow chart for the NHES-FS data collection which will begin with the mailing of an advance notification letter in early January 2014. Both NHES:2012 and NATES showed that an advance letter was effective at raising response rates to the first questionnaire mailing, which reduces the amount of follow-up required. A questionnaire package will be mailed to households one week after the advance letter. The packages will contain a cover letter, household screener, and business reply envelope. In some cases, the initial questionnaire mailing package will contain a $5 cash incentive and/or a magnet as parts of two experiments which are described below. The same version of the questionnaire sent in the first mailing will be sent in all subsequent nonresponse follow-up mailings. A thank you/reminder postcard will be sent to all sampled addresses approximately one week after the first mailing. All nonresponse follow-up questionnaire mailings will contain a cover letter, replacement screener questionnaire, and business reply envelope. A second questionnaire mailing will be sent to nonresponding households approximately two weeks after the postcard. Approximately three weeks after the second mailing, a third questionnaire mailing will be sent to non-responding households using rush delivery (FedEx or UPS). For addresses for which the frame includes a telephone number, an automated reminder phone call will be made on the same day as the third questionnaire mailing to encourage households to complete the study as soon as possible. Results from the 2009 and 2011 tests indicated that households with eligible children tend to respond to later mailings, for this reason we added an additional mailing to our process and extended the data collection period as part of the NHES:2012 data collection and we will continue that protocol in the NHES-FS. The fourth questionnaire package will be sent to nonresponding households approximately three weeks after the third questionnaire mailing.

The NHES-FS includes several experiments to test approaches aimed toward decreasing unit and item nonresponse. These experiments are illustrated in Figure 1. Experiment 1 will test unit and item response rate differences between a screener that asks for age measured in years versus a screener that asks for age measured as year and month of birth. NCES would prefer to ask age as year and month of birth in order to implement more accurate age-eligibility criteria for topical surveys. However, there is a concern that this type of age measure is more personal and therefore more sensitive which can potentially lead to greater unit or item nonresponse.

Experiment 2 will test response rate differences between households that do not receive a cash incentive with their initial screener questionnaire mailing and households that receive the NHES standard $5 incentive. A small proportion of households (around 1,800) will be randomly assigned to not receive the $5 cash incentive. The purpose of this experiment is to provide up-to-date information about the effectiveness of the household-level incentive compared to no incentive.

Experiment 3 will test the effectiveness of mailing a large credit-card sized magnet with the screener to serve as a reminder to complete the survey. NCES previously tested the use of a coin-sized magnet during the NHES:2012 development. This small magnet did not increase screener response, but there was some evidence it boosted topical response. We will examine screener and topical unit response in the magnet and no magnet treatment groups.

Topical Procedures

Topical survey mailings will follow procedures similar to the screener procedures. Households with at least one sample person will be assigned to one of three topical survey mailing groups:

1. Households that have been selected to receive an ASPA questionnaire only;

2. Households that have been selected to receive an ATES questionnaire only; and

3. Households that have been selected to receive both an ASPA and ATES questionnaire.

Within each group, questionnaires will be mailed in waves to minimize the time between the receipt of the screener and the mailing of the topical.

Regardless of treatment group or wave, households with a selected sample member will have the same topical contact strategy. The initial topical mailing will include a cover letter, questionnaire, business reply envelope, and a cash incentive. All nonresponse follow-up mailings will contain a cover letter, replacement questionnaire, and business reply envelope. The NHES:2011 Field Test as well as the NHES:2012 showed that a $5 incentive was effective with most respondents at the topical level. However, late screener respondents (those that responded to the third or fourth mailing) were more likely to respond to the $15 incentive amount. As a result, we will offer these late screener responders a $15 prepaid incentive with their first topical questionnaire mailing. Late screener responders receiving both the ASPA and ATES questionnaires will receive two $15 incentives with their first topical questionnaire mailing. All other respondents will receive $5 with their initial mailing, with the dual households receiving two $5 incentives. One week after the initial topical mailing, all households will receive a thank you/reminder postcard.

Nonresponding households will be mailed a second topical questionnaire approximately two weeks after the reminder postcard. If households that have been mailed a second topical questionnaire do not respond, a third package will be mailed by rush delivery (FedEx or UPS) approximately three weeks after the second mailing. An automated reminder phone call will also be made to households where the frame vendor has a phone number. If the household does not respond to the third mailing, a fourth mailing will be sent. Figure 2 shows the topical mailing plans.

In addition to the screener experiments described earlier, the NHES-FS also contains topical experiments designed to test methods of decreasing unit response to the topical survey and to help refine questionnaire content. Experiment 4 will compare unit response rates when a household is mailed both the ATES and ASPA questionnaires versus a single ATES or ASPA questionnaire. We hypothesize that households mailed two surveys will have lower unit response than households mailed one survey. However, collecting information from multiple members of the same household could allow us to improve the efficiency of the data collection and generate cost savings in the future if it does not depress unit response rates.

Experiment 5 will include a split-ballot test of the ASPA topical instrument to test the efficiency of different question wordings and formats for selected items. Households will be randomly assigned to each version. Response patterns will be compared to previous NHES administrations and external surveys to identify the item wording that is yielding the best response. Item response rates and navigation error rates will also be compared across the two versions of the ASPA instruments to determine which format is yielding higher quality data. The highest performing items and format will be used in the NHES:2015 where applicable and in the NHES:2017 when the ASPA is planned for administration.

Experiment 6 will include a split-ballot test of the full ATES instrument including sections on work-related training and non-credit college classes versus a shorter instrument including only key items from those sections. Households will be randomly assigned to either the ATES or the Credentials for Work Survey (CWS). Because of historical concerns about response rates from adults on household surveys about their own education and training, this experiment will determine whether the longer length of the ATES instrument discourages respondents from completing and mailing the form. Unit and item nonresponse rates will be compared across the two instruments. If the nonresponse rates are substantially lower for the ATES than for the CWS, NCES will field the CWS in NHES:2015 with plans to field the complementary Education and Training Survey in NHES:2017.

Survey Monitoring

Mail survey returns will be processed upon receipt, and reports from the survey management system will be prepared at least weekly. The reports will be used to continually assess the progress of data collection.

B.3 Methods for Maximizing Response Rates

The NHES-FS design incorporates a number of features to maximize response rates. This section discusses those features.

Total Design Method/Respondent-Friendly Design. Surveys that take advantage of respondent-friendly design have demonstrated increases in survey response (Dillman, Smyth, and Christian 2008; Dillman, Sinclair, and Clark 1993). We have honed the design of the NHES forms through multiple iterations of cognitive interviewing and field testing. These efforts have focused on the design and content of all respondent contact materials. As noted previously, we will include a respondent incentive in the initial screener mailing for some households. Respondent incentives will also be used in the initial topical mailing. Many years of testing in the NHES have shown the effectiveness of a small cash incentive on increasing response. The Census Bureau will maintain an email address and a toll-free questionnaire assistance (TQA) line to answer respondent questions or concerns. Additionally, if a respondent chooses to provide their information to the TQA staff, staff will be able to collect the respondent’s information in an Internet-based screener questionnaire. If successful in the feasibility study, this online questionnaire could be used for future NHES self-response. Additionally, the questionnaires contain contact information for the Project Officer.

Engaging Respondent Interest and Cooperation. The content of respondent letters and frequently asked questions (FAQs) is focused on communicating the legitimacy and importance of the study. Past experience has shown that the NHES survey topics are salient to most parents. In the NHES:2012, Census Bureau “branding” was experimentally tested against Department of Education branding. Response rates to Census Bureau branded questionnaires were associated with higher response rates compared to Department of Education branded questionnaires. Based on this finding, we will highlight the Census Bureau’s participation in the data collection.

Nonresponse Follow-up. The data collection protocol includes several stages of nonresponse follow-up at each phase. In addition to the number of contacts, changes in method (mail, FedEx, and alternate envelopes) are designed to capture the attention of potential respondents.

B.4 Individuals Responsible for Study Design and Performance

From NCES, the following persons participated in the study design and are responsible for the collection and analysis of the data: Sarah Carroll, Sharon Boivin, Lisa Hudson, and Chris Chapman, and from the Census Bureau Sharon Stern.

References

Blumberg, S.J. and Luke, J.V. (2009). Wireless substitution: Early release of estimates from the National Health Interview Survey, July-December 2009. National Center for Health Statistics. Available from: http://www.cdc.gov/nchs/nhis.htm. May 18, 2010.

Brick, J.M., Hagedorn, M.C., Montaquila, J., Brock Roth, S., and Chapman, C. (2006). Monetary Incentives and Mailing Procedures in a Federally Sponsored Telephone Survey: Methodology Report. U.S. Department of Education. Washington, DC: National Center for Education Statistics.

Dillman, D.A., Sinclair, M.D., and Clark, J.R. (1993). Effects of questionnaire length, respondent-friendly design, and difficult questions on response rates for occupant-addressed Census mail surveys. Public Opinion Quarterly, 57, 289-304.

Dillman, D.A., Smyth, J.D., and Christian, L.M. (2008). Internet, mail, and mixed mode surveys: The Tailored Design Method. New York: Wiley.

Hagedorn, M., Roth, S.B., Carver, P., Van de Kerckhove, W., Smith, S., and Mulligan, G. (2009). National Household Education Surveys Program of 2007: Methodology Report. National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC.

U.S. Department of Labor. (2013). Employer Costs for Employee Compensation – March 2013. Washington, DC: Bureau of Labor Statistics. Available online at http://www.bls.gov/news.release/pdf/ecec.pdf

Figure 1: Screener Data Collection

Exp. 6

Exp. 2 & 3

Exp. 1

Exp. 5

Exp. 4

Experiment 1 – Version A (age) or B (month and year of birth) of the screener

Experiment 2 - $0 vs. $5 cash incentive in initial screener mailing

Experiment 3 – magnet or no magnet in initial screener mailing

Experiment 4 – Joint ASPA and ATES topical group

Experiment 5 – ASPA split panel

Experiment 6 – ATES split panel

Figure 2. Topical Mailing Plan – ASPA

Figure 3. Topical Mailing Plan – ATES

Figure 4. Topical Mailing Plan – ASPA/ATES

1 For convenience, this package refers to the questionnaire that will be sent to adults as ATES. As described later in this package, NCES actually plans to field two versions of this questionnaire, one named ATES and the other—a subset of ATES items—named the Credentials for Work Survey.

2 The APSA is not planned for administration until 2017. In 2015, NCES plans to re-field the Parent and Family Involvement in Education Survey (PFI) and Early Childhood Program Participation Survey (ECPP), which were originally planned for the full-scale collection in 2014 that was cancelled due the 2013 sequestration budget cuts. The PFI and ECPP surveys have already been through extensive development work via cognitive interviews, the NHES pilot test, the NHES field test, and the full-scale NHES:2012, so further testing of the full instruments is not needed. The ASPA is being tested because it has not yet been administered in the new mail data collection mode and was last administered in 2005. There are a few items in the ASPA that are shared with the PFI and ECPP. We will use the results of the ASPA split panel tests to select which items to include in the 2015 PFI and ECPP surveys.

3 NCES collects information on the following racial/ethnic groups in the NHES child topical surveys: American Indian or Alaska Native, Asian, Black or African American, Hispanic, Native Hawaiian or other Pacific Islander, and White. Reported estimates by race typically use the following categories: White, non-Hispanic; Black, non-Hispanic; Hispanic; Asian or other Pacific Islander, non-Hispanic; and other, non-Hispanic. Areas with higher concentrations of Blacks and Hispanics are typically oversampled in order to ensure sufficient samples sizes to generate reliable estimates for these subgroups. In the Feasibility Study, these groups will not be oversampled because no estimates will be published, instead comparisons will primarily be made across item distributions for the two split panel forms.

4 NHES:2007 also included an adult education module, but data collection was stopped due to low response rates.

5 Ineligible addresses are those which are undeliverable. Once a screener mailing for an address is returned as undeliverable as addressed (UAA), the address will be coded ineligible.