Part A Supporting Statement NHES 2011-2012 Field Test

NHES:2011/2012 Field Test

Request for OMB Review

OMB# 1850-0768 v.7

Part A

June 30, 2010

TABLE OF CONTENTS

Page

JUSTIFICATION 1

A.1 Circumstances Necessitating Collection of Information 7

A.2 Purposes and Uses of the Data 7

A.3 Use of Improved Information Technology 8

A.4 Efforts to Identify Duplication 9

A.4.1 Topical Surveys 9

A.5 Collection of Data from Small Businesses 11

A.6 Consequences of Less Frequent Data Collection 11

A.7 Special Circumstances 11

A.8 Public Comment and Consultations Outside the Agency 11

A.9 Payments to Respondents 15

A.10 Assurance of Confidentiality 16

A.11 Sensitive Questions 18

A.12 Estimated Response Burden 19

A.13 Annualized Cost to Respondents 20

A.14 Annualized Cost to the Federal Government 20

A.15 Reasons for Program Changes 21

A.16 Publication Plans and Project Schedule 21

A.17 Approval for Not Displaying the Expiration Date for OMB
Approval 22

A.18 Exceptions to the Certification Statement 22

References 23

List of Tables

Table Page

1 Weighted unit response rates and percentage distribution of type of unit nonresponse for the NHES screener: 1991–2007 4

2 Weighted unit response rates and percentage distribution of type of unit nonresponse for the NHES screener: 1991–2007 4

2a Estimated response burden for NHES:2011 field test 20

2b Estimated response burden for NHES:2012 20

TABLE OF CONTENTS (continued)

List of Exhibits

Exhibit

1 Surveys conducted under the National Household Education Surveys Program, by years administered: 1991 through 2007 6

2 NCES Affidavit of Nondisclosure 17

3 NHES:2011 schedule of major activities 22

4 NHES:2012 schedule of major activities 22

PART A. JUSTIFICATION

Request for Clearance

The National Household Education Surveys Program (NHES) has been a system of landline random-digit-dial (RDD) surveys sponsored by the National Center for Education Statistics (NCES). Surveys have been conducted approximately every other year from 1991 through 2007. Like virtually all RDD surveys, the NHES has experienced declining response rates. In addition, the increased percentage of households without landline telephones (mostly due to conversion to cellular-only coverage) raised issues about population coverage. While studies examining possible biases in the NHES survey estimates have not identified nonresponse bias, some indications of possible coverage bias were detected in a special bias study conducted in 2007.

The combination of the continuing declines in RDD survey response and population coverage issues prompted NCES to undertake a redesign of the NHES program. The goals of the redesign effort are to develop and assess approaches to collecting important information on education topics from households while improving response rates and coverage from the previous design. The redesign effort is also expected to provide information that will be useful to other government survey programs and the survey methodology field in general.

This request is for clearance of an operational field test to be conducted in 2011 and for a 60 day Federal Register notice waiver for the submission of the full-scale data collection package. The field test will provide an opportunity to examine proposed methods on a smaller and more economical scale prior to the full-scale data collection planned for 2012. The waiver is requested because no substantive changes in full-scale survey instruments or methodology are expected from those described in this package, to be tested in 2011. Also, the 2012 data collection will need to begin a few months after draft results of the 2011 field test will become available.

The NHES:2011 field test will involve the screening of approximately 60,000 households to identify those with eligible children and youths. The NHES:2012 will screen approximately 198,000 households. It is expected that less than 11,500 parents or guardians of sampled children will be asked to respond to one of two topical surveys: the Early Childhood Program Participation survey (ECPP) or the Parent and Family Involvement in Education survey (PFI) in 2011 and less than 37,500 in 2012.

Clearance is requested by September 30, 2010, in order to complete the final formatting of scannable questionnaires in time for printing and of final instrument programming for telephone follow-up.

NHES Background

The NHES was developed by NCES to complement its institutional surveys; it is the principal mechanism for addressing topics that cannot be addressed in institutional data collections. By collecting data directly from households, the NHES allows NCES to gather data on a wide range of issues, such as early childhood care and education, children’s readiness for school, parent perceptions of school safety and discipline, before- and after-school activities of school-age children, participation in adult and continuing education, parent involvement in education, school choice, homeschooling, and civic involvement. These topics are addressed through a series of rotating surveys. The study has been conducted in the winter and spring of approximately every other year from 1991 through 2007, and each of these prior administrations used random digit dial (RDD) sampling and telephone data collection from landline telephones only. Each study collection has involved the administration of household screening questions (screener) and two or three topical surveys. The 2007 data collection included the Parent and Family Involvement in Education survey (PFI) and the School Readiness survey (SR).

Data from the NHES are used to provide national cross-sectional estimates on populations of special interest to education researchers and policymakers. For surveys about children, the population of interest is defined by age or grade in school, or both, depending on the particular survey topic and research questions. For surveys of adults, the population of interest is persons ages 16 and older who are not enrolled in grade 12 or below, excluding those on active duty military service and those who are institutionalized. The NHES targets these populations using specific screening and sampling procedures.

The NHES design also yields estimates for subgroups of interest for each survey, as defined by age or grade for children, education participation status for adults, Hispanic origin, and Black or African American racial background for all populations of interest. In addition to providing cross-sectional estimates, the NHES is also designed to provide estimates of change over time in key statistics.

Survey data from the NHES have been used for a large number of descriptive and analytic reports and articles, including NCES publications, publications of other Federal agencies, policy analyses, theses and dissertations, conference papers, and journal articles. A list of NHES publications issued by NCES can be found on the NHES website, http://nces.ed.gov/nhes.

Motivation for Redesign

Like many other telephone surveys and ongoing periodic survey programs, the NHES has been experiencing declining response rates. Screener response rates for the NHES have declined from above 80 percent in the early 1990s to 53 percent in 2007. (See table 1) Final response rates for the NHES are dependent on both the screener response rate and topical response rate. The final response rate for the 2007 PFI was 39.1 percent. The SR survey achieved a 40.7 percent final response rate in 2007. Since its inception, the NHES has used a variety of procedures to boost response rates, such as:

• advance letters;

• incentives;

• one or more refusal conversion attempts;

• special mailings such as Federal Express prior to refusal conversion;

• increased numbers of call attempts;

• leaving messages on answering machines;

• special training of interviewers to avoid refusals;

• assigning refusal cases to specially trained refusal conversion interviewers; and

• changes in the protocol for working the cases, such as increasing total time in the field.

Publications presenting some of these studies can be found at http://nced.ed.gov/nhes. Despite these interventions, the NHES response rates have continued to fall and, although response to the topical surveys was relatively high among screener respondents, the final overall response rates for NHES 2007 were low.

Meanwhile, with the increasing proportion of households having only cellular telephone service, landline telephone coverage rates have declined from about 93 percent of households in early 2004 to about 74 percent of households in the second half of 2009 (Blumberg and Luke 2009).

As a result of the precipitous declines in response and possible coverage issues, the NHES is undergoing a redesign and examining alternatives to an RDD telephone survey methodology. The proposed alternative methodology uses an address-based sample and a contact strategy that relies primarily on mail data collection. The redesign is being conducted in two phases: a small-scale feasibility test and a large-scale response maximization field test.

Table 1.  Weighted unit response rates and percentage distribution of type of unit nonresponse for the NHES screener: 1991–2007

Year of survey	Number of completed screeners	Unit response rate (percent)	Type of unit nonresponse (percentage distribution)
			Refusals	Maximum calls	Other nonresponse
1991	60,322	81.0	84	7	9
1993	63,844	82.1	68	15	181
1995	45,465	73.3	84	9	7
1996	55,838	69.9	83	10	7
1999	55,929	74.1	76	17	7
2001	48,385	67.5	74	18	8
2003	32,049	61.7	76	16	8
2005	58,140	64.2	77	15	8
2007	54,034	52.5	86	10	4

¹ The NHES:1993 percentage of other nonresponse cases is higher than that in other survey years. The lower rate of refusals and the generally higher response rate in NHES:1993 are indicative of the fact that less refielding of other nonresponse cases was needed prior to ending data collection with an acceptable screener response rate.

NOTE: To avoid any differences in rates that might be attributable to the calculation method, all unit response rates given here were calculated using the business office method. Therefore, response rates given here are somewhat different than the official response rates cited in survey reports and documentation. The official rates for 2001, 2003, and 2005 use the survival method. The official rate for 2007 uses the vendor-assisted method. See chapter 4 of the National Household Education Surveys Program of 2007: Methodology Report (Hagedorn et al. 2009) for details on the methods for computing response rates. The number of household members enumerated in each data collection differed according to the sample requirements of the topical surveys conducted in the specific year. Maximum call cases are those that received at least eight call attempts during which contact was made with a person on at least one occasion, yet the screener was not completed. Other nonresponse includes cases with language problems, no-answer and answering machine calls (downweighted to reflect the appropriate proportion assumed to be residential), and other forms of nonresponse. Detail may not sum to totals because of rounding.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), selected years, 1991-2007.

Table 2.  Weighted unit response rates and percentage distribution of type of unit nonresponse for the NHES screener: 1991–2007

Interview type	Number of completed interviews	Unit response rate (percent)1	Overall response rate2
SR	2,633	77.0	40.7
PFI	10,681	74.1	39.1

¹ The unit response rate is the percentage of completed interviews for a specific stage of the survey (i.e., the Screener, SR, or PFI interview). It is a ratio of the number of completed interviews to the number of units (e.g., households and household members) sampled for the interviews. For many telephone numbers sampled for the Screener interview, no contact was ever made. Based on results of the vendor-assisted method calculations, 37.6 percent of these numbers were assumed to be residential and were added to the denominator for the calculation of the Screener unit response and overall unit response rates. Additionally, the Screener unit response rate accounts for the subsampling of cases for nonresponse followup, which is discussed further in section 4.1.1.

² The overall unit response rate indicates the percentage of possible interviews that have been completed, taking all sampling stages into account. The overall unit response rate and the unit response rate are identical for the first stage of sampling and interviewing (i.e., the Screener). For the SR or PFI surveys, the overall unit response rate is the product of the Screener unit response rate and the interview unit response rate (e.g., for the SR survey, the calculation for the overall unit response rate is 100 x (0.528 x 0.770) = 40.7).

SOURCE: U.S. Department of Education, National Center for Education Statistics, School Readiness (SR) Survey of the National Household Education Surveys Program (NHES), 2007; and Parent and Family Involvement in Education (PFI) Survey of the NHES, 2007.

The feasibility test conducted in 2009 achieved an overall response rate of approximately 52%, as compared to the 2007 overall response rate of 39-41% (depending on the module). The results suggest that the new methodology has the ability to address the response rate and coverage issues identified in the 2007 data collection. The results from the feasibility test were presented at the 65th Annual Conference of The American Association for Public Opinion Research in May 2010 and can be found in Appendix A.

Based largely on the results of the feasibility study, a large-scale field test is planned in an effort to test whether response rates can be further increased and to refine operational elements of the new design. The data collection is scheduled to begin in early January 2011 and finish by June 2011 (see section A.16).

The NHES has been conducted approximately every two years since the first data collection in 1991. Following this schedule, the next NHES would have been conducted in 2009. However, due to the redesign, this collection was canceled, leaving a gap in the data time series. Full-scale NHES data collection is planned for early 2012.

NHES:2011/2012 Surveys

As shown in exhibit 1, each administration of the NHES has included more than one topical survey. The NHES:2011 field test and NHES:2012 will include two topical surveys: the Parent and Family Involvement in Education survey (PFI) and the Early Childhood Program Participation survey (ECPP). These two surveys are repeated administrations of topics shown in the exhibit below, but have been adapted for administration under a new methodological design. The instruments are described in detail in Part C of this document and appear in Appendix B.

Exhibit 1.  Surveys conducted under the National Household Education Surveys Program, by years administered: 1991 through 2007

Survey topics	NHES survey administration
	1991	1993	1995	1996	19991	2001	2003	2005	2007
Early childhood education/participation								
Adult education								
School readiness									
School safety and discipline		
Parent and family involvement in education									
Civic involvement					
After-school programs and activities			2			3		
Household and library use				

¹ NHES:1999 was a special end-of-decade administration that measured key indicators from the surveys fielded during the 1990s.

² These items were only asked about children in first through third grades.

³ The NHES:2001 survey about after-school programs and activities (ASPA) also included before-school programs.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 1991, 1993, 1995, 1996, 1999, 2001, 2003, 2005, and 2007.

The surveys that will be administered in the NHES:2011 field test are not intended to produce survey estimates for analytical purposes. The goal of the field test is methodological. The evaluation of the revised NHES methodology requires a realistic administration of proposed methods and the use of instruments that reflect the topics, length, and complexity of a regular NHES survey. The field test will also permit the examination of alternate question wording and response patterns (e.g. skip instruction errors) that may suggest instrument changes for the 2012 administration.

The topical surveys planned for the 2011 field test are:

• The Early Childhood Program Participation Survey (ECPP), previously conducted in 1991, 1995, 2001, and 2005, surveys families of children ages 6 or younger who are not yet enrolled in kindergarten and provides estimates of children’s participation in care by relatives and non-relatives in private homes and in center-based daycare or preschool programs (including Head Start and Early Head Start). Additional topics addressed in ECPP interviews have included family learning activities; out-of-pocket expenses for nonparental care; continuity of care; factors related to parental selection of care; parents’ perceptions of care quality; child health and disability; and child, parent and household characteristics.

• The Parent and Family Involvement in Education Survey (PFI), previously conducted in 1996, 2003, and 2007, surveys families of children and youth enrolled in kindergarten through 12th grade or homeschooled for these grades, with an age limit of 20 years, and addresses specific ways that families are involved in their children’s school; school practices to involve and support families; involvement with children’s homework; and involvement in education activities outside of school. Parents of homeschoolers are asked about their reasons for choosing homeschooling and resources they used in homeschooling. Information about child, parent, and household characteristics is also collected. To minimize response burden and potential respondent confusion, an enrolled and homeschool version of the PFI questionnaire was created for self administration. The homeschool version will be administered in 2012, but will not be included in the 2011 field test due to sample and cost implications.

A.1 Circumstances Necessitating Collection of Information

NCES has as its legislative mission the collection and publication of data on the condition of education in the Nation (the National Education Statistics Act of 1994, P.L. 103-382, October 20, 1994 (20 USC 9001)): “The duties of the Center are to collect, analyze, and disseminate statistics and other information related to education in the United States and in other nations.”

The NHES is specifically designed to support this mission by providing a means to investigate education issues that cannot be adequately studied through the Center’s institution-based data collection efforts. For example, young children are cared for in many types of informal or formal settings and some children are cared for only in their own homes. There is no available sample frame that includes all of these types of care arrangements. The ECLS-K collects data on past care arrangements among students enrolled in a kindergarten program, which does not include homeschooled students and is only retrospective. It is efficient and economical to interview parents about their involvement in children’s education through a household-based approach rather than incurring the cost and nonresponse involved in enlisting schools, obtaining lists of parents, and sampling parents from those lists. The NHES surveys conducted from 1991 through 2007 afford the opportunity to track change over time in several important education domains that are of interest to policymakers and researchers such early childhood experiences and parent involvement in education.

Many issues that are central to assessing the condition of education in the United States can be measured adequately only by a household-based survey conducted at regular intervals. Other studies dealing with similar topics differ in crucial ways from the NHES (see section A.4.1 for summary information about other studies). In particular, none of them measure the topics of interest at specific, planned intervals; as a result, changes over time cannot be studied effectively with other existing data sources. While the NHES:2011 field test is not being conducted to make survey estimates, the topical surveys represent key repeated topics and measures for the NHES program, and support a realistic test of the data collection methodology during the field test.

The continuation of the NHES program to fill this important need requires that the methodology be revised to reflect the current survey environment. The NHES:2011 field test is a vital part of that process. The 2011 field test will yield a final design for the 2012 NHES data collection.

A.2 Purposes and Uses of the Data

The data collected in the NHES:2011 field test will be used to evaluate the revised survey methodology and its ability to produce improvements in population coverage and response rates, while also testing the impact of respondent error on data quality and the efficiency of the proposed methodologies.

The resulting 2012 data collection will provide policymakers and researchers with data on early childhood education, parent and family involvement in education, and homeschooling that are not available elsewhere. The data contribute directly to NCES education indicators, reports, and statistical abstracts, and researchers nationwide rely on NHES data for important policy analyses.

A.3 Use of Improved Information Technology

The NHES:2011 field test will be conducted for NCES by Westat using two complementary survey systems that will improve the efficiency and accuracy of the data collection process.

The self-administered questionnaires will be implemented in TeleForm, which will be utilized for forms design and electronic data capture and archiving capabilities.

• Forms Design. Questionnaires will be created using the Designer module and selected information, such as name and address, can be pre-filled prior to distribution. Form templates are used to classify each data field as a text entry, choice, signature, or image zone. Completed hardcopy forms can be processed by TeleForm to capture responses without manual data entry.

• Image Preprocessing. TeleForm applies image preprocessing to the forms in their image format in order to correct any skewing that may have occurred during scanning or faxing, and to remove other unwanted marks from the form according to project specifications.

• Data Capture. TeleForm reads the form image files and extracts data according to rules established for each questionnaire template. TeleForm can recognize handwritten (ICR), printed (OCR), check box, and ‘bubble’ (OMR) data types.

• Verification. Extracted data are subject to field validation according to project specifications. If a data value violates validation rules, the data may be flagged for review by verifiers who interactively review the images and the corresponding extracted data, and resolve validation errors.

• Archiving. Images will be scanned and archived to disk in case they are needed later. This eliminates the need to save paper copies of the completed questionnaires.

Telephone non response follow-up interviews will be conducted using the computer-assisted telephone interviewing (CATI) system. The most important features of the system for the NHES are the following:

• Sampling: CATI will be programmed to identify eligible household members and sample respondents for interviews. The use of online sampling eliminates the need for separate screening and interviewing calls, reducing survey cost and respondent burden.

• Scheduling: The CATI scheduler will be used to route telephone numbers to interviewers, maintain a schedule of callback appointments, and reschedule unsuccessful contact attempts to an appropriate day and time.

• Skip Patterns: The CATI system will automatically guide interviewers through the skip patterns in the questionnaire, reducing the potential for interviewer error and shortening the questionnaire administration time.

• Receipt Control: The CATI system will provide for automatic receipt control in a flexible manner that will be used to produce status reports that allow ongoing monitoring of the telephone non response operation.

A contractor has not been selected for the NHES:2012 data collection. However, it is anticipated that the collection will utilize technologies similar to those described above to reduce respondent burden and increase efficiency in the data collection.

A.4 Efforts to Identify Duplication

In the course of developing the pilot study approach, we drew upon recent literature in survey methodology and knowledge drawn from methodological research presented in venues such as professional meetings and seminars of the American Association for Public Opinion Research. In addition, a Technical Review Panel of experts in survey methodology was convened to provide input to the approach, and these experts were queried about other similar efforts. At that time, no other national surveys examining a similar alternative to an RDD design were identified.

Appendix C contains a review of other surveys that cover topics similar to those in the NHES. The review shows that there is little overlap between the NHES and these other surveys.

A.4.1 Topical Surveys

As noted in the introduction, the ECPP and PFI forms will be fielded in the NHES:2011 but no national estimates, topical reports, or public use datasets will result from the study. Rather, the NHES:2011 field test is being conducted for methodological purposes. The NHES:2012 data collection will produce national estimates, reports, public-use datasets, and restricted datasets.

Past and current efforts have been made to avoid duplication in the NHES topical surveys. No other surveys have been identified that duplicate the ECPP and PFI Surveys. The limitations of other existing surveys in relation to the data collected by NHES fall into three general categories.

Population: Most other surveys do not address these survey topics for the populations of interest in the NHES. For example, the Head Start Family and Child Experiences Survey (FACES) focus on children in Head Start, whereas all children who have not yet started kindergarten are of interest in the ECPP Survey. The National Survey of Parents of Public School Students and Survey of Family and School Partnerships in Public Schools focus on parents of children in public schools; those whose children attend private schools or are homeschooled are not represented. Some studies, such as the Early Childhood Longitudinal Study-Birth Cohort (ECLS-B) and the Early Childhood Longitudinal Study-Kindergarten Class of 1998-1999, focus on single-year cohorts that are followed over time and therefore do not provide nationally representative data on different age groups. The NHES surveys are designed to complement these longitudinal collections with more frequent and more inclusive cross-sectional data.

Survey Content: Extant studies are limited in the content that they include relative to the goals of the NHES surveys. Studies such as the National Survey of America’s Families and the National Study of the Changing Workforce collect some information on child care or program participation, but their primary emphasis is on other topics, and the depth of information on early care and education experiences is limited. The Head Start FACES project collects information on Head Start program participation and some family measures, but does not account for all nonparental care and programs. The Current Population Survey October Education Supplement is limited to a relatively small number of items on education participation and does not address the roles that parents play in their children’s school, schoolwork, and home activities. Also, no nationally representative study other than the NHES collects detailed data on homeschooling.

Current Estimates and Measuring Change Over Time. Many of the extant surveys follow one cohort or periodic cohorts (e.g., the Early Childhood Longitudinal Studies, Head Start FACES) or are no longer conducted (e.g., the National Survey of America’s Families, Family Involvement in Education: A National Portrait). As a result, they cannot meet the NHES goal of providing up-to-date cross-sectional estimates and measures of change over time for all children who have not started kindergarten or for children in kindergarten through 12th grade, as is provided by the NHES.

Due to these limitations in extant studies and the importance of data collected by the NHES, NCES plans to continue to conduct the ECPP and PFI surveys under the NHES program.

A.5 Collection of Data from Small Businesses

Not applicable.

A.6 Consequences of Less Frequent Data Collection

This request is for clearance of the NHES:2011 field test and a waiver from the 60 day review for the NHES:2012. Topics covered in this NHES collection have been addressed in previous NHES administrations; repeating the surveys allows for analysis of trends over time. Less frequent collection would result in incomplete tracking of these trends. The last full NHES study was conducted in 2007

A.7 Special Circumstances

None of the special circumstances listed in the instructions for completing the supporting statement apply to the NHES:2011 field test or to the NHES:2012.

A.8 Public Comment and Consultations Outside the Agency

The 60-day Federal Register notice was published on July 8, 2010 (75 FR, No. 130, p. 39215). No public comments were received in response to this notice.

Methodological Experts

A Technical Review Panel (TRP) comprising leading experts in survey methodology was established to provide input to the redesign of the NHES system. Most members of the panel met in February 2010 to discuss the proposed design for the field test, and their comments and suggestions led to changes reflected in this submission.

Technical Review Panel Participants and Their Affiliation at the Time of TRP Recruitment

Nancy Bates

U.S. Census Bureau

649 A. St. N.E.

Washington, DC 20002

Tel: 301-763-5248

E-mail: [email protected]

Paul Beatty

National Center for Health Statistics

Division of Health Care Statistics

3311 Toledo Road,

Hyattsville, MD 20782

Tel. 301-458-4090

E-mail: [email protected]

Johnny Blair

Survey Sampling and Methodology

Abt Associates Inc.

4550 Montgomery Avenue

Bethesda, MD 20814-3343

Tel: 301-634-1825

E-mail: [email protected]

Stephen Blumberg

National Center for Health Statistics

3311 Toledo Road

Hyattsville, MD 20782

Tel.301-458-4107

E-mail: [email protected]

Mick Couper

Survey Research Center

University of Michigan

ISR, 426 Thompson Street

Ann Arbor, MI 48104

Tel: 734-647-3577

E-mail: [email protected]

Don Dillman

Social and Economic Sciences Research Center, Professor

Washington State University

133 Wilson Hall

Pullman, WA 99164-4014

Tel: 509-335-1511

E-mail: [email protected]

Robert Groves

Survey Research Center, Institute for Social Research

University of Michigan

426 Thompson Street

Ann Arbor, MI 48106-1248

Tel: 734-764-8365

E-mail: [email protected]

Scott Keeter

Pew Research Center

1615 L. St. NW. Suite 700

Washington, DC 20036

Tel: 202-419-4362

E-mail: [email protected]

Kristen Olsen

Survey Research and Methodology

University of Nebraska-Lincoln

201 N. 13th St.

Lincoln, NE 68588-0241

Tel: 402-472-7737

E-mail: [email protected]

Roger Tourangeau

Joint Program in Survey Methodology

University of Maryland

1218 LeFrak Hall, University of Maryland

College Park, MD 20742

Tel: 240-595-0057

E-mail: [email protected]

Gordon Willis

Division of Cancer Control / Population Sciences

National Cancer Institute

6130 Executive Blvd, MSC 7344, EPN 4005

Bethesda, MD 20892-7344

Tel: 301-594-6652

E-mail: [email protected]

Content Area Experts

The content of the NHES:2011 and NHES:2012 topical surveys repeats, to a great extent, the content developed for previous NHES administrations. As a result, the two surveys reflect the cumulative input of many experts in the field and past NHES Technical Review Panels. In order to ensure that the ECPP and PFI Surveys address important issues in the topical areas of interest and incorporate important emerging issues, the design phase of the study included consultations with experts in the substantive areas addressed in the surveys. These experts included persons in government agencies, academe, and research organizations.

Substantive Experts: ECPP and Their Affiliation at the Time of TRP Recruitment

Jerry West - Mathematica

Mathematica Policy Research, Inc.

600 Maryland Ave., SW, Suite 550

Washington, DC 20024-2512

(202) 484-9220

[email protected]

Ann Collins – Abt Assoc. Cambridge, MA

Abt Associates Inc.

55 Wheeler Street

Cambridge, MA 02138-1168

(617) 492-7100

[email protected]

Ron Haskins – Brookings Institution and Casey Foundation

The Brookings Institution

1775 Massachusetts Ave., NW

Washington, DC 20036

(202) 797-6004

[email protected]

Ivelisse Martinez-Beck – HHS Division of Child and Family Development

Administration for Children and Families

370 L’Enfant Promenade, S.W.

7th Floor West, Room 7A011

Washington, D.C. 20447

(202) 690-7885

[email protected]

Lynda Laughlin – Census

U.S. Census Bureau

4600 Silver Hill Road

Suitland, MD 20746

(301) 763-6314

[email protected]

Substantive Experts: PFI and Their Affiliation at the Time of TRP Recruitment

Richard Brandon – Univ. of Washington

Human Services Policy Center, Evans School of Public Affairs

University of Washington

1107 NE 45th St.

Seattle, WA 98105

(206) 685-3135

[email protected]

Annette Lareau – Univ. of Pennsylvania

Department of Sociology

University of Pennsylvania

McNeil Hall

Philadelphia, PA 19104

(215) 898-3515

[email protected]

Joyce Epstein – The Johns Hopkins University

Center for Social Organization of Schools

3003 N. Charles St., Suite 200

Baltimore, MD 21218

410-516-8807

[email protected]

Lawrence Aber - NYU

Steinhardt School of Culture, Education, and Human Development

New York University

82 Washington Square East

New York, NY 10003

(212) 998-5410

[email protected]

A.9 Payments to Respondents

The NHES:2003 included an extensive experiment in the use of small cash incentives to improve unit response. The experiment demonstrated that gains in respondent cooperation could be realized with relatively modest cash incentives (Brick et al. 2006). Such incentives were used in NHES:2005 and NHES:2007. The NHES:2009 feasibility test showed that an increase in the modest respondent incentives could significantly boost response rates (results presented in AAPOR presentation, 3 AAPOR2010.pdf, part of Appendix A). One of the goals of NHES:2011 is to evaluate the impact of different incentive levels, at the screener and the topical survey stages, and of other strategies on increasing response rates while minimizing cost to the government (see Part B.2 for more details).

As a result, an advance cash incentive of $2 or $5 will be sent with the first screener mailing. A $2 incentive was used in past rounds of the NHES. The NHES:2009 incorporated a $2 incentive and achieved a response rate of approximately 60%. NHES:2011 will test whether $5 prepaid incentive will boost the screener response rate and, if so, by how much. For those households in which someone is selected as the subject of an ECPP or PFI questionnaire, cases will be subsampled to receive $0, $5 $10, $15, or $20 with the topical surveys to test effects on response of five different levels of monetary incentive in the mail survey. These levels are based on an experiment conducted in 2009 in which $15 yielded a significantly higher response rate than $0 or $5 at the topical level (see 3 AAPOR2010.pdf and Part B.2 for more details).

Incentive levels for the NHES:2012 will be based on the outcome of these NHES:2011 field test experiments and project budget constraints.

A.10 Assurance of Confidentiality

Respondents will be informed of the voluntary nature of the survey and of the confidentiality provision in the initial cover letter and on the questionnaires, stating that their responses may be used for statistical purposes only and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) Section 9573, 20 US Code]. Additionally, all staff members and subcontractors working on the NHES and having access to the data (including monitoring of interviews) are required to sign the NCES Affidavit of Nondisclosure (exhibit 2). Notarized affidavits are kept on file by the contractor and submitted to NCES quarterly. In addition, all contractor staff members who have access to confidential data and work on the project more than 30 days are required to have a federal background check.

Exhibit 2.  NCES Affidavit of Nondisclosure

Affidavit of Nondisclosure

(Job Title)

(Date Assigned to Work with NCES Data) 

(Organization, State or Local Agency Name) 

I, __________________________________ , do solemnly swear (or affirm) that when given access to the subject NCES database or file, I will not -

(Organization or Agency Address)

(NCES Database or File Containing Individually Identifiable Information*) 

(i) use or reveal any individually identifiable information furnished, acquired, retrieved or assembled by me or others, under the provisions of Section 9573, 20 US Code and Title V, subtitle A of the E-Government Act of 2002 (P.L. 107-347) for any purpose other than statistical purposes specified in the NCES survey, project or contract;

(ii) make any disclosure or publication whereby a sample unit or survey respondent (including students and schools) could be identified or the data furnished by or related to any particular person or school under these sections could be identified; or

(iii) permit anyone other than the individuals authorized by the Commissioner of the National Center for Education Statistics to examine the individual reports.

___________________________________ (Signature)

(Notary Public/Seal) My commission expires__________________ .

[The penalty for unlawful disclosure is a fine of not more than $250,000 (under 18 U.S.C. 3571) or imprisonment for not more than five years (under 18 U.S.C. 3559), or both. The word "swear" should be stricken out when a person elects to affirm the affidavit rather than to swear to it.] 

* Request all subsequent follow-up data that may be needed. This form cannot be amended by NCES, so access to databases not listed will require submitting additional notarized Affidavits.

Sworn to and subscribed before me this _______________ day of 

_______________, 20________ . Witness my hand and official Seal. 

City/County of _________________ Commonwealth/State of ________________ . 

A.11 Sensitive Questions

The NHES is a voluntary survey, and no persons are required to respond to the interviews. In addition, respondents may decline to answer any question in the survey. Respondents are informed of the voluntary nature of the survey in the cover letter that accompanies the questionnaire, as well as on the actual questionnaire.

Some items in the surveys may be considered sensitive by some respondents:

ECPP and PFI Surveys. Child development and education experts consider economic disadvantage and children’s disabilities to be important factors in children’s preschool and school experiences and in family involvement in their education. As a result, the ECPP and PFI surveys contain measures of these characteristics, including:

• Household income;

• Receipt of public assistance in the form of Temporary Assistance to Needy Families (TANF), food stamps, and the Women, Infants, and Children program (WIC); and

• Children’s disability conditions.

Measures of household income and government assistance are important because access to early childhood programs by children at risk and the education involvement of families of different socioeconomic backgrounds are of interest to policymakers, child development specialists, and educators. These items are important to identifying children at risk and have been administered successfully in previous NHES studies. Respondents are also asked the age at which they first became a parent to a child. This may be sensitive for parents in some situations.

The 2007 response rates for these items were very high. For total household income, the 2007 PFI survey had an item response rate of 90.45 percent. Those for receipt of public assistance were also high: for Temporary Assistance to Needy Families, 98.86 percent; for the Women, Infants, and Children Program, 99.05 percent; and for Food Stamps, 99.03 percent. The 2007 item response rates for questions about children’s disabilities were all over 99 percent. In the 2009 pilot, the item response rate for age at which mother first became a parent to any child was 98.4 percent among mothers completing the survey.

ECPP Survey: In addition to the items above, the ECPP survey also includes questions about assistance to pay for child care. This measure is important to understand families’ and children’s access to early childhood programs.

PFI Survey: The PFI survey includes items concerning children’s school performance and difficulties in school. Among these are:

• Children’s school performance and difficulties, including school grades, suspensions, and expulsions; and

• Identification of children’s schools.

Items concerning school performance and difficulty are important to the PFI Survey as indicators of school readiness for young children, and as correlates of parent and family involvement for children of all ages and grades. Item response rates for these PFI items in 2007 were also high: 98.22 percent for children’s grades, 99.57 percent for out-of-school suspension, and 99.64 percent for expulsion.

Another element of the surveys that may be sensitive to some parents is the identification of children’s schools. This feature allows analysts to link the NHES data to other NCES datasets containing additional data about schools, greatly enhancing the ability to examine the relationships between students’ and families’ experiences and the characteristics of schools. This was done for the first time in the PFI survey of NHES:2007; the item response rate for the identification of the child’s school was 97.1 percent.

A.12 Estimated Response Burden

The response burden per instrument and the total response burden are shown in table 2a for the NHES:2011 field test and table 2b for the NHES:2012. The administration times for the screener, ECPP, and PFI questionnaires are based on practice administrations.

The expected number of respondents and number of responses are based on the expected numbers of completed surveys of each type, discussed in section B.1.4. For the NHES: 2011 field test, the cost to respondents for the total hour burden is estimated to be $110,036, that is, $19.88 per hour for 5,535 burden hours. The hourly rate is based on the average for all civilian workers from the 2007 National Compensation Survey (http://www.bls.gov/ncs/ocs/sp/nctb0298.pdf). There are no other costs to respondents. There are also no recordkeeping requirements associated with NHES. For the NHES:2012, a total of 18,266 burden hours are anticipated, resulting in a cost to respondents of approximately $363,128.

Table 2a.  Estimated response burden for NHES:2011 field test

Interview forms	Estimated time (minutes)	Number sampled	Estimated Number of respondents	Estimated Number of responses	Estimated Total time (hours)
Study Total		60,000	32,400	40,905	5,535
Screener (average)	5	60,000	32,400	32,400	2,700
ECPP questionnaire	20	3,251	2,438	2,438	813
PFI questionnaire	20	8,089	6,067	6,067	2,022

NOTE: Details may not sum to totals due to rounding of partial hours.

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2011 National Household Education Survey Field Test.

Table 2b.  Estimated response burden for NHES:2012

Interview forms	Estimated time (minutes)	Number sampled	Estimated Number of respondents	Estimated Number of responses	Total time (hours)
Study Total		198,000	106,920	134,987	18,266
Screener (average)	5	198,000	106,920	106,920	8,910
ECPP questionnaire	20	10,728	8,046	8,046	2,682
PFI questionnaire	20	26,694	20,021	20,021	6,674

NOTE: Details may not sum to totals due to rounding of partial hours.

SOURCE: U.S. Department of Education, National Center for Education Statistics, 2012 National Household Education Survey.

A.13 Annualized Cost to Respondents

There are no costs beyond those presented in section A.12.

A.14 Annualized Cost to the Federal Government

The total cost of NHES:2011 to the government is approximately $2.7M over a period of 20 months. This includes all direct and indirect costs of the design, data collection, analysis, and reporting phases of the study, as well as the delivery of data sets to NCES. The contract for the NHES:2012 has not been awarded and, therefore, the expected cost to the government is currently not available.

A.15 Reasons for Program Changes

The increase in the burden this collection reinstatement is due to the fact that the last clearance was for a smaller scale pilot test, and this request is for NHES:2011 field test, which is necessary to obtain the methodological data that will inform the redesign of the full-scale data collection of the NHES:2012.

A.16 Publication Plans and Project Schedule

The primary objectives of the NHES:2011 are to identify the best methods to boost response rates in future NHES data collections. The NHES:2012 will implement these findings and produce datasets, statistics, and reports. The following are the planned outcomes of the NHES:2011:

• Operational issues: Any events or issues that result in a delay to the implementation of each event within the field test schedule. Since many of these operational tasks will be evaluated, each process will need to be documented so procedures are available at implementation of the large-scale data collection in 2012.

• Respondent feedback: Any feedback received from a respondent will be evaluated for reaction to methods (e.g., reaction to the use of government funds for incentives, difficulty of mode, etc.).

• Response rates and components: Overall, and for each experimental manipulation/stage, response rates (both unit and item response rates, return rates, and any appropriate refusal conversion rates) will be tracked.

• Response timing: For each manipulation, time to response/questionnaire receipt will be assessed to determine the effects on the data collection schedule.

Exhibits 3 and 4 present schedules of project activities for NHES:2011 and NHES:2012, respectively.

Exhibit 3.  NHES:2011 schedule of major activities

Task	Date of Scheduled Conduct/Completion
Cognitive Research	April 13 to June 30, 2010
Cognitive Report and Recommendations	July 15, 2010
Revised TeleForm Questionnaires to NCES	August 15, 2010
CATI Questionnaires to NCES	August 15, 2010
OMB Clearance Completed	October 1, 2010
Interviewer Training Materials	November 15, 2010
Field Test Begins (screener mailing)	January 13, 2011
Outline for Field Test Report	April 15, 2011
Field Test Ends	June 1, 2011
Draft Field Test Report	August 15, 2011
Revised Field Test Report	November 1, 2011
Draft data file delivery	October 1, 2011
Final data file delivery	December 1, 2011

Exhibit 4.  NHES:2012 schedule of data collection

Task	Date of Scheduled Conduct/Completion
Data Collection Begins (screener mailing)	January 13, 2012
Data Collection Ends	June 1, 2012

A.17 Approval for Not Displaying the Expiration Date for OMB Approval

The OMB authorization number and expiration date will be displayed on the hard copy questionnaire. Telephone interviewers will have the OMB authorization number in their study materials and will provide this information to respondents upon request.

A.18 Exceptions to the Certification Statement

There are no exceptions to the certification statement.

References

Blumberg, S.J. and Luke, J.V. (2009). Wireless substitution: Early release of estimates from the National Health Interview Survey, July-December 2009. National Center for Health Statistics. Available from: http://www.cdc.gov/nchs/nhis.htm. May 18, 2010.

Brick, J.M., Hagedorn, M.C., Montaquila, J., Brock Roth, S., and Chapman, C. (2006). Monetary Incentives and Mailing Procedures in a Federally Sponsored Telephone Survey: Methodology Report. U.S. Department of Education. Washington, DC: National Center for Education Statistics.

Hagedorn, M., Roth, S.B., Carver, P., Van de Kerckhove, W., Smith, S., and Mulligan, G. (2009). National Household Education Surveys Program of 2007: Methodology Report. National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC.

U.S. Department of Labor. (2008). National Compensation Survey: Occupational Wages in the United States: July 2007. Washington, DC: Bureau of Labor Statistics. Available online at http://www.bls.gov/ncs/ocs/sp/nctb0298.pdf