Part A NHES2009 Pilot 2009-05-22 Supporting Statement

NHES:2009 Pilot Test

Request for

IMT/OMB Review

Part A

February 20, 2009

TABLE OF CONTENTS

Section Page

PART A: JUSTIFICATION 1

Request for Clearance 1

A.1. Circumstances Necessitating Collection of Information 7

A.2. Purposes and Uses of the Data 8

A.3. Use of Improved Information Technology 8

A.4. Efforts to Identify Duplication 9

A.5. Collection of Data from Small Businesses 11

A.6. Consequences of Less Frequent Data Collection 11

A.7. Special Circumstances 11

A.8. Public Comment and Consultations Outside the Agency 11

A.9. Payments to Respondents 15

A.10. Assurance of Confidentiality 16

A.11. Sensitive Questions 19

A.12. Estimated Response Burden 20

A.13. Annualized Cost to Respondents 21

A.14. Annualized Cost to the Federal Government 21

A.15. Reasons for Program Changes 21

A.16. Publication Plans and Project Schedule 22

A.17. Approval for Not Displaying the Expiration Date for OMB Approval 23

A.18. Exceptions to the Certification Statement 23

LIST OF TABLES

Table Page

1 Weighted unit response rates and percentage distribution of type of unit nonresponse for the NHES Screener: 1991–2007 5

2 Estimated response burden for NHES:2009 21

LIST OF EXHIBITS

Exhibit Page

1 Surveys conducted under the National Household Education Surveys Program, by years administered: 1991 through 2007 6

2 NCES Affidavit of Nondisclosure 17

3 Westat Confidentiality Pledge 18

4 NHES:2009 schedule of major activities 23

PART A: JUSTIFICATION

Request for Clearance

The National Household Education Surveys Program (NHES) has been a system of landline random-digit-dial (RDD) surveys sponsored by the National Center for Education Statistics (NCES). Surveys have been conducted approximately every other year from 1991 through 2007.  Like virtually all RDD surveys, NHES has experienced declining response rates. In addition, the increase in the percentage of households without landline telephones (mostly due to conversion to cellular-only coverage) raises issues about population coverage. While studies examining possible biases in the NHES survey estimates have not identified nonresponse bias, some indications of possible coverage bias were detected in a special bias study conducted in 2007.

The combination of the continuing declines in RDD survey response and population coverage prompted NCES to undertake a redesign of the NHES program. The goals of the redesign effort are to develop and assess approaches to collecting important information on educational topics from households with improved response and population coverage. The redesign effort is also expected to provide information that will be useful to other government survey programs and the survey methodology field generally.

This request is for clearance of an operational Pilot Test to be conducted in the fall of 2009. The Pilot Test will provide an opportunity to examine proposed methods on a smaller and more economical scale prior to a large-scale Field Test planned for 2011. A separate request will be submitted for approval of the 2011 Field Test.

The NHES:2009 Pilot Test will involve the screening of approximately 11,800 households to identify those with eligible children and youth. Parents or guardians of sampled children will be asked to respond to one of two topical surveys: The Early Childhood Program Participation Survey (ECPP), and the Parent and Family Involvement in Education Survey (PFI). The PFI Survey has been divided into two questionnaire forms for ease of self-administration: one focuses on children enrolled in school for kindergarten through 12th grade and one focuses on children who are homeschooled.

Clearance is requested by August 1, 2009, in order to complete the final formatting of scannable questionnaires and final instrument programming for telephone followup.

NHES Background

NHES was developed by the NCES to complement its institutional surveys; it is the principal mechanism for addressing topics that cannot be addressed in institutional data collections. By collecting data directly from households, NHES allows NCES to gather data on a wide range of issues, such as early childhood care and education, children’s readiness for school, parent perceptions of school safety and discipline, before- and after-school activities of school-age children, participation in adult and continuing education, parent involvement in education, school choice, homeschooling, and civic involvement. The survey has been conducted by Westat in the winter and spring approximately every other year from 1991 through 2007, and each of these prior administrations used random digit dial (RDD) sampling and telephone data collection from landline telephones only. Each survey collection has involved the administration of household screening questions (Screener) and two or three topical surveys.

NHES provides data for national cross-sectional estimates on populations of special interest to NCES and education researchers. For surveys about children, the population of interest is defined by age or grade in school, or both, for the particular survey topic and research questions. For surveys of adults, the population of interest is persons ages 16 and older who are not enrolled in grade 12 or below, excluding those on active duty military service and those who are institutionalized. NHES targets these populations using specific screening and sampling procedures. The NHES design also yields estimates for subgroups of interest for each survey, as defined by age or grade for children, educational participation status for adults, and Black and Hispanic origin for all populations of interest. In addition to providing cross-sectional estimates, NHES is also designed to provide estimates of change over time in key statistics.

The NHES survey data have been used for a large number of descriptive and analytical reports and articles, including NCES publications, those of other federal agencies, theses and dissertations, conference papers, and journal articles. These reports and articles have addressed the wide range of topics addressed in the NHES surveys, including child care, early childhood education, school readiness, parent involvement, homeschooling, adult education, and so on. Some are briefs addressing one or two key indicators, whereas others present multivariate analyses and in-depth discussion. A list of NHES publications issued by NCES can be found at the NHES web page, http://nces.ed.gov/nhes.

Because of the complexity of the instruments and the complex within-household sampling techniques used, the previous NHES surveys were conducted using computer-assisted telephone interviewing (CATI) technology. Benefits of the CATI administration of the NHES surveys have included improved project administration, online sampling and eligibility checks, scheduling of interviews according to a priority scheme, managing data quality by controlling skip patterns and checking responses during the interview for range and consistency, and a “help” function for a limited number of items to assist interviewers in answering respondents’ questions during the interview.

Procedures that have been used in an effort to attain high response rates in past NHES surveys include:

• advance letters;

• incentives;

• one or more refusal conversion attempts;

• special mailings such as Federal Express prior to refusal conversion;

• increased numbers of call attempts;

• leaving messages on answering machines;

• special training of interviewers to avoid refusals;

• assigning refusal cases to specially trained refusal conversion interviewers; and

• changes in the protocol for working the cases, such as increasing total time in the field.

The data collection protocol used in the NHES program evolved over time to meet the challenges of maintaining response rates. Most of the changes were based upon the results of methodological experiments conducted under the NHES program, including those focusing on household screening strategies, mailing by USPS and FedEx, and the use of incentives. Publications presenting some of these studies can be found at the NCES web page for NHES, http://nced.ed.gov/nhes.

Bias in the NHES

Concerns about declining response and landline telephone coverage rates led to the development and administration of an independent study, conducted in conjunction with the 2007 National Household Education Surveys Program (NHES:2007).  For this independent bias study, a clustered sample of addresses was selected and matched with telephone numbers where possible; cases that could not be completed by telephone using the standard NHES protocol were sent into the field for in-person attempts to gain cooperation. 

 

Through this study, we were able to examine both nonresponse bias and noncoverage bias in the three surveys comprising NHES:2007.  To evaluate nonresponse bias, estimates of demographics and key statistics were compared for the reduced data collection effort (by telephone) and the full effort (by telephone and in-person).  Additionally, neighborhood observations obtained by the interviewer and ZIP code-level characteristics obtained from Census 2000 were examined for households that did not respond to either the telephone or in-person effort.  To evaluate noncoverage bias, estimates for households with a landline phone were compared to those for the entire sample.   There was no bias of substantive importance in the NHES:2007 estimates due to nonresponse. Although the study demonstrated the potential for noncoverage bias, the raking adjustments used in weighting might mitigate that bias.  The complete report can be found at: http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2009029.

Motivation for Redesign

Like many other telephone surveys and ongoing periodic survey programs, NHES has been experiencing declining response rates. NHES Screener response rates have declined from above 80 percent in the early 1990s to 53 percent in 2007. (See table 1.)

Table 1.  Weighted unit response rates and percentage distribution of type of unit nonresponse for the NHES Screener: 1991–2007

Year of survey	Number of completed Screeners	Unit response rate (percent)	Type of unit nonresponse (percentage distribution)
			Refusals	Maximum calls	Other nonresponse
1991	60,322	81.0	84	7	9
1993	63,844	82.1	68	15	181
1995	45,465	73.3	84	9	7
1996	55,838	69.9	83	10	7
1999	55,929	74.1	76	17	7
2001	48,385	67.5	74	18	8
2003	32,049	61.7	76	16	8
2005	58,140	64.2	77	15	8
2007	54,034	52.5	86	10	4

¹ The NHES:1993 percentage of other nonresponse cases is higher than that in other survey years. The lower rate of refusals and the generally higher response rate in NHES:1993 are indicative of the fact that less refielding of other nonresponse cases was needed prior to ending data collection with an acceptable Screener response rate.

NOTE: To avoid any differences in rates that might be attributable to the calculation method, all unit response rates given here were calculated using the business office method. Therefore, response rates given here are somewhat different than the official response rates cited in survey reports and documentation. The official rates for 2001, 2003, and 2005 use the survival method. The official rate for 2007 uses the vendor-assisted method. See chapter 4 of Hagedorn et al. (forthcoming) for details on the methods for computing response rates. The number of household members enumerated in each data collection differed according to the sample requirements of the topical surveys conducted in the specific year. Maximum call cases are those that received at least eight call attempts during which contact was made with a person on at least one occasion, yet the Screener was not completed. Most maximum call cases in NHES had 20 attempts. Other nonresponse includes cases with language problems, no-answer and answering machine calls (downweighted to reflect the appropriate proportion assumed to be residential), and other forms of nonresponse. Detail may not sum to totals because of rounding.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), selected years, 1991-2007.

Meanwhile, with the increasing prevalence of households having only cellular telephone service, landline telephone coverage rates have declined from about 93 percent of households in early 2004 to about 80 percent of households in the first half of 2008 (Blumberg and Luke 2008).

As a result of the precipitous declines in response and coverage rates, NHES is undergoing a redesign, examining alternatives to a RDD telephone survey methodology. The proposed alternative methodology uses a multi-mode approach with an address-based sample, a contact strategy that relies primarily on mail data collection, and includes several experiments that may affect the cost of and response to NHES. The original plan called for a large-scale Field Test of the new methodology in winter/spring 2010; however, due to a moratorium on Federal household data collection during the decennial census, the data collection for the large-scale Field Test has been delayed to 2011.

In an effort to test operational and experimental elements of the new design, a smaller-scale Pilot Test is planned, to be conducted in the fall of 2009 with a reduced sample (approximately 10 percent of the anticipated 2011 Field Test sample size). The data collection schedule for the Pilot Test is given in section A.17. It is anticipated that the field period for the Pilot Test will start in early September 2009 and finish by mid-December 2009. Under this schedule, it is anticipated that the PFI questionnaire will need to be modified slightly for the Pilot Test (relative to the questionnaire that is planned for the NHES:2011 Field Test) as a result of the data collection occurring very early in the school year.

NHES:2009 Pilot Test Surveys

As shown in exhibit 1, each administration of NHES has included more than one topical survey. NHES:2009 includes two surveys: the Parent and Family Involvement in Education Survey (PFI), and the Early Childhood Program Participation Survey (ECPP). These two surveys are repeated administrations of topics shown in the exhibit below, but have been adapted for administration under a new methodological design. The instruments are described in detail in Part C of this document and appear in appendix A.

Exhibit 1.  Surveys conducted under the National Household Education Surveys Program,
by years administered: 1991 through 2007

Survey topics	NHES survey administration
	1991	1993	1995	1996	19991	2001	2003	2005	2007
Early childhood education/program    participation	√		√		√	√		√
Adult education	√		√		√	√	√	√
School readiness		√			√				√
School safety and discipline		√
Parent and family involvement in    education				√	√		√		√
Civic involvement				√	√
After-school programs and    activities			√2		√	√3		√
Household and library use				√

¹ NHES:1999 was a special end-of-decade administration that measured key indicators from the surveys fielded during the 1990s.

² These items were only asked about children in first through third grades.

³ The NHES:2001 survey about after-school programs and activities (ASPA) also included before-school programs.

SOURCE: U.S. Department of Education, National Center for Education Statistics, National Household Education Surveys Program (NHES), 1991, 1993, 1995, 1996, 1999, 2001, 2003, 2005, and 2007.

The surveys that will be fielded in the NHES:2009 Pilot Test are not intended to produce survey estimates for analytical purposes. The goal of the Pilot Test is methodological. However, the assessment of the NHES methodology requires a realistic administration of proposed methods and the use of instruments that reflect topics, length, and complexity that would normally be fielded in an NHES survey. The Pilot Test will also permit the assessment of response patterns (e.g. skip errors) that may suggest instrument changes for the 2011 Field Test.

The topical surveys planned for the 2009 Pilot Test and 2011 Field Test are:

• The Early Childhood Program Participation Survey (ECPP), previously conducted in 1991, 1995, 2001, and 2005, provides estimates of children’s participation in care by relatives and non-relatives in private homes and in center-based daycare or preschool programs (including Head Start and Early Head Start). Additional topics addressed in ECPP interviews have included family learning activities, out-of-pocket expenses for nonparental care, continuity of care, factors related to parental selection of care, parents’ perceptions of care quality, child health and disability, and child, parent, and household characteristics. The ECPP Survey population includes children ages 6 or younger who are not yet enrolled in kindergarten.

• The Parent and Family Involvement in Education Survey (PFI), previously conducted in 1996, 2003, and 2007, addresses specific ways that families are involved in their children’s school, school practices to involve and support families, involvement with children’s homework, and involvement in educational activities outside of school. Parents of homeschoolers are asked about their reasons for choosing homeschooling and resources they used in homeschooling. The interviews also include questions about child, parent, and household characteristics. The PFI Survey population includes children and youth enrolled in kindergarten through 12th grade or homeschooled for these grades, with an age limit of 20 years.

A.1. Circumstances Necessitating Collection of Information

NCES has as its legislative mission the collection and publication of data on the condition of education in the Nation (the National Education Statistics Act of 1994, P.L. 103-382, October 20, 1994 (20 USC 9001)):

The duties of the Center are to collect, analyze, and disseminate statistics and other information related to education in the United States and in other nations.

NHES is specifically designed to support this mission by providing a means to investigate educational issues that cannot be adequately studied through the Center’s institution-based data collection efforts. For example, young children are cared for in many types of informal or formal settings and some children are cared for only in their own homes. As a result, no institutional sample frame is available for young children. It is efficient and economical to interview parents about their involvement in children’s education through a household-based approach rather than incurring the cost and nonresponse involved in enlisting schools, obtaining lists of parents, and sampling parents from those lists. The NHES surveys conducted from 1991 through 2007 afford the opportunity to track change over time in several important educational domains that are of interest to policymakers and researchers. For example, information about children’s emerging literacy and numeracy collected in NHES surveys is germane to the No Child Left Behind Act.

Many issues that are central to assessing the condition of education in the United States can be measured adequately only by a household-based survey conducted at regular intervals. Other studies dealing with similar topics differ in crucial ways from NHES (see section A.4.1 for summary information about other studies). In particular, none of them measure the topics of interest at specific, planned intervals; as a result changes over time cannot be studied effectively with other data sources. While the NHES:2009 Pilot Test is not being conducted to make survey estimates, the topical surveys represent key repeated topics and measures for the NHES program, to support a realistic test of the methodology.

The continuation of the NHES program to fill this important need requires that the methodology be revised to reflect the current survey environment. The NHES:2009 Pilot Test is a vital part of that process.

A.2. Purposes and Uses of the Data

The data collected in the NHES:2009 Pilot Test will be used to evaluate the revised survey methodology and its ability to produce improvements in population coverage and response rates, while also testing the impact of respondent error on data quality and the efficiency of the proposed methodologies. Information gathered from this pilot will be used to finalize the methodologies that will be used in the 2011 Field Test and to make the necessary additions and modifications to the instruments.

A.3. Use of Improved Information Technology

The NHES:2009 Pilot Test will be conducted using two complementary survey systems that will improve the efficiency and accuracy of the data collection process.

The self-administered questionnaires will be implemented in Teleform, which offers a wide range of capabilities to projects. The following features are important for NHES:2009:

• Forms Design. Questionnaires can be created using the Designer module and selected information such as name and address can be pre-filled prior to distribution. Form templates are used to classify each data field as a text entry, choice, signature or image zone. Completed hardcopy forms can be processed by TeleForm to capture responses without manual data entry.

• Image Preprocessing. TeleForm applies image preprocessing to the forms in their image format in order to correct any skewing that may have occurred during scanning or faxing, and to remove other unwanted marks from the form according to project specifications.

• Data Capture. TeleForm reads the form image files and extracts data according to rules established for each questionnaire template. TeleForm can recognize handwritten (ICR), printed (OCR), check box, and ‘bubble’ (OMR) data types.

• Verification. Extracted data are subject to field validation according to project specifications. If a data value violates validation rules, the data may be flagged for review by verifiers who interactively review the images and the corresponding extracted data, and resolve validation errors.

Telephone followup interviews will be conducted using the Voxco computer-assisted telephone interviewing (CATI) system. The most important features of the Voxco system for NHES are the following:

• Sampling: The CATI will be programmed to identify eligible household members and sample respondents for interviews. The use of online sampling eliminates the need for separate screening and interviewing calls, reducing survey cost and respondent burden.

• Scheduling: The CATI scheduler will be used to route telephone numbers to interviewers, maintain a schedule of callback appointments, and reschedule unsuccessful contact attempts to an appropriate day and time.

• Skip Patterns: The CATI system will automatically guide interviewers through the skip patterns in the questionnaire, reducing the potential for interviewer error and shortening the questionnaire administration time.

• Receipt Control: The CATI system will provide for automatic receipt control in a flexible manner that will be used to produce status reports that allow ongoing monitoring of the survey’s progress.

A.4. Efforts to Identify Duplication

The principal goals of the NHES:2009 Pilot Test are methodological: the assessment of alternative survey designs. In the course of developing the Pilot Test approach, Westat drew upon recent literature in survey methodology and knowledge drawn from methodological research presented in venues such as professional meetings and seminars of the American Association for Public Opinion Research. In addition, a Technical Review Panel of experts in survey methodology was convened to provide input to the approach, and these experts were queried about other similar efforts. No other national surveys examining a similar alternative to an RDD design were identified.

A.4.1. Topical Surveys

As noted in the introduction and section A.1., the surveys to be fielded in the Pilot Test represent measures and topics of the types planned for the redesigned survey system. This is an important feature of the Pilot Test and Field Test processes: that they include real NHES surveys. However, the surveys are not being conducted for the purpose of making national estimates or producing reports or data sets. Rather, the NHES:2009 Pilot Test is being conducted for methodological purposes.

Past and current efforts have been made to avoid duplication in NHES topical surveys. No other surveys have been identified that duplicate the ECPP and PFI Surveys. The limitations of existing surveys fall into three general categories.

Population: Most other surveys do not address these survey topics for the populations of interest in NHES. For example, the Head Start Family and Child Experiences Survey (FACES) focuses on children in Head Start, whereas all children who have not yet started kindergarten are of interest in the ECPP Survey. The National Survey of Parents of Public School Students and Survey of Family and School Partnerships in Public Schools focus on parents of children in public schools; those whose children attend private schools or are homeschooled are not represented. Some studies of both early childhood and school-age populations focus on single-year cohorts that are followed over time, and therefore do not provide data for the entire population of interest in an NHES survey; the Early Childhood Longitudinal Study-Birth Cohort (ECLS-B) and the Early Childhood Longitudinal Study-Kindergarten Class of 1998-1999 illustrate this limitation.

Survey Content: Some extant studies are limited in the content that they include relative to the goals of the NHES surveys. Studies such as the National Survey of America’s Families and the National Study of the Changing Workforce collect some information on child care or program participation, but their primary emphasis is on other topics, and the depth of information on early care and educational experiences is limited. The Head Start FACES project collects information on Head Start program participation and some family measures, but does not account for all nonparental care and programs. The Current Population Survey October Education Supplement is limited to a relatively small number of items on educational participation, and does not address the roles that parents play in their children’s school, schoolwork, and home activities. No nationally representative study other than NHES collects detailed data on homeschooling.

Current Estimates and Measuring Change Over Time. Many of the extant surveys follow one cohort or periodic cohorts (e.g., the Early Childhood Longitudinal Studies, Head Start FACES) or are no longer conducted (e.g., the National Survey of America’s Families, Family Involvement in Education: A National Portrait). As a result, they cannot meet the NHES goal of providing up-to-date cross-sectional estimates and measures of change over time for all children who have not started kindergarten or for children in kindergarten through 12th grade.

Due to these limitations in extant studies, NCES plans to continue to conduct the ECPP and PFI Surveys under the NHES program. Therefore, these surveys are appropriate models for testing the revised study methodology.

A.5. Collection of Data from Small Businesses

Not applicable.

A.6. Consequences of Less Frequent Data Collection

This request is for clearance of the NHES:2009 Pilot Test only. A separate request will be submitted for the 2011 Field Test and for future NHES collections.

A.7. Special Circumstances

None of the special circumstances listed in the instructions for completing the supporting statement apply to the NHES:2009 Pilot Test.

A.8. Public Comment and Consultations Outside the Agency

Methodological Experts

A Technical Review Panel comprising leading experts in survey methodology was established to provide input to the redesign of the NHES system. Most members of the panel met in January 2009 to discuss the proposed design, and their comments and suggestions led to changes reflected in this submission.

Technical Review Panel Participants

Nancy Bates

U.S. Census Bureau

649 A. St. N.E.

Washington, DC 20002

Tel: 301-763-5248

E-mail: [email protected]

Paul Beatty
National Center for Health Statistics

Division of Health Care Statistics
3311 Toledo Road,
Hyattsville, MD 20782

Tel. 301-458-4090

E-mail: [email protected]

Johnny Blair

Survey Sampling and Methodology

Abt Associates Inc.

4550 Montgomery Avenue

Bethesda, MD 20814-3343

Tel: 301-634-1825

E-mail: [email protected]

Stephen Blumberg

National Center for Health Statistics

3311 Toledo Road

Hyattsville, MD 20782

Tel.301-458-4107

E-mail: [email protected]

Mick Couper

Survey Research Center

University of Michigan

ISR, 426 Thompson Street

Ann Arbor, MI 48104

Tel: 734-647-3577

E-mail: [email protected]

Don Dillman

Social and Economic Sciences Research Center, Professor

Washington State University
133 Wilson Hall
Pullman, WA 99164-4014

Tel: 509-335-1511

E-mail: [email protected]

Robert Groves

Survey Research Center, Institute for Social Research

University of Michigan

426 Thompson Street

Ann Arbor, MI 48106-1248

Tel: 734-764-8365

E-mail: [email protected]

Scott Keeter

Pew Research Center

1615 L. St. NW. Suite 700

Washington, DC 20036

Tel: 202-419-4362

E-mail: [email protected]

Kristen Olsen

Survey Research and Methodology

University of Nebraska-Lincoln

201 N. 13th St.
Lincoln, NE 68588-0241
Tel: 402-472-7737

E-mail: [email protected]

Roger Tourangeau

Joint Program in Survey Methodology

University of Maryland

1218 LeFrak Hall, University of Maryland

College Park, MD 20742

Tel: 240-595-0057

E-mail: [email protected]

Gordon Willis

Division of Cancer Control / Population Sciences

National Cancer Institute

6130 Executive Blvd, MSC 7344, EPN 4005

Bethesda, MD 20892-7344

Tel: 301-594-6652

E-mail: [email protected]

Content Area Experts

The NHES:2009 topical surveys repeat, to a great extent, designs developed for previous NHES administrations. As a result, they reflect the cumulative input of many experts in the field and past NHES Technical Review Panels. In order to ensure that the ECPP and PFI Surveys address important issues in the topical areas of interest and incorporate important emerging issues, the design phase of the study included consultations with experts in the substantive areas addressed in the surveys. These experts included persons in government agencies, academe, and research organizations.

Substantive Experts: ECPP

Jerry West - Mathematica

Mathematica Policy Research, Inc.

600 Maryland Ave., SW, Suite 550
Washington, DC 20024-2512
(202) 484-9220

[email protected]

Ann Collins – Abt Assoc. Cambridge, MA

Abt Associates Inc.
55 Wheeler Street
Cambridge, MA 02138-1168

(617) 492-7100

[email protected]

Ron Haskins – Brookings Instit. And Casey Foundation

The Brookings Institution

1775 Massachusetts Ave., NW

Washington, DC 20036

(202) 797-6004

[email protected]

Ivelisse Martinez-Beck – HHS Division of Child and Family Development

Administration for Children and Families
370 L'Enfant Promenade, S.W.

7th Floor West, Room 7A011

Washington, D.C.  20447

(202) 690-7885

[email protected]

Lynda Laughlin – Census

U.S. Census Bureau

4600 Silver Hill Road

Suitland, MD 20746

(301) 763-6314

[email protected]

Substantive Experts: PFI

Richard Brandon – Univ. of Washington

Human Services Policy Center, Evans School of Public Affairs

University of Washington

1107 NE 45^th St.

Seattle, WA 98105

(206) 685-3135

[email protected]

Annette Lareau – Univ. of Pennsylvania

Department of Sociology

University of Pennsylvania

McNeil Hall

Philadelphia, PA 19104

(215) 898-3515

[email protected]

Joyce Epstein – The Johns Hopkins University

Center for Social Organization of Schools

3003 N. Charles St., Suite 200

Baltimore, MD 21218

410-516-8807

[email protected]

Lawrence Aber - NYU

Steinhardt School of Culture, Education, and Human Development

New York University

82 Washington Square East

New York, NY 10003

(212) 998-5410

[email protected]

A.9. Payments to Respondents

NHES:2003 included an extensive experiment in the use of small cash incentives to improve unit response. The experiment demonstrated that gains in respondent cooperation could be realized with relatively modest cash incentives (Brick et al. 2006). Such incentives were used in NHES:2005 and NHES:2007 and are planned for the NHES:2009 Pilot Test.

An advance cash incentive of $2 will be sent with the first screener mailing. For those households in which someone is selected as the subject of an ECPP or PFI questionnaire, cases will be subsampled to receive a prepaid $0, $5 or $15 incentive with the topical surveys to experimentally test effects on response of three different levels of monetary incentive in the mail survey.

As discussed in section B.1.1, an attempt will be made to match telephone numbers to the sampled addresses. Telephone completion of a screener will be attempted with a subsample of cases with telephone number matches if the household does not respond to the screener mailings. If a screener is completed by telephone and someone in the household is sampled for a topical survey, a random sample of those taking the topical interview will be offered $5 to complete the survey, to be mailed following the interview.

A.10. Assurance of Confidentiality

Education Sciences Reform Act of 2002 (ESRA 2002) - Under this law all individually identifiable information about students, their families, and their schools shall remain confidential. To this end, this law requires that no person may:

a. Use any individually identifiable information furnished under the provisions of this section for any purpose other than statistical purposes for which it is supplied, except in the case of terrorism (see discussion of the Patriot Act);

b. Make any publication whereby the data furnished by any particular person under this section can be identified; or

c. Permit anyone other than the individuals authorized by the Director to examine the individual reports.

All Westat staff members working on NHES and having access to the data (including monitoring of interviews) are required to sign the NCES Affidavit of Nondisclosure (exhibit 2) and a similar Westat confidentiality pledge (exhibit 3). Employees of Child Trends, a subcontractor to Westat, will also be required to sign the Affidavit. Notarized affidavits are kept on file at Westat and documentation is submitted to NCES quarterly. In addition, all staff members who have access to confidential data are required to have a federal background check.

The following confidentiality pledge accompanies all surveys in this pilot:

Your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law [Education Sciences Reform Act of 2002 (ESRA 2002) Public Law 107-279, Section 183].

Exhibit 2.  NCES Affidavit of Nondisclosure

______________________________________ ___________________________________

(Job Title) (Date of Assignment to NCES Project)

______________________________________ ___________________________________

(Organizations, State or local (NCES Data Base or File Containing

agency or instrumentality) Individually Identifiable Information)

_______________________________________

(Address)

I, __________________________________, do solemnly swear (or affirm) that when given access to the

subject NCES data base or file, I will not

(i) use or reveal any individually identifiable information furnished, acquired, retrieved or assembled by me or others, under the provisions of Section 406 of the General Education Provisions Act (20 U.S.C. 1221e-1) for any purpose other than statistical purposes specified in the NCES survey, project or contract;

(ii) make any disclosure or publication whereby a sample unit or survey respondent could be identified or the date furnished by or related to any particular person under this section can be identified; or

(iii) permit anyone other than the individuals authorized by the Commissioner of the National Center for Education Statistics to examine the individual reports.

________________________________________

(Signature)

(The penalty for unlawful disclosure is a fine of not more than $250,000 (under 18 U.S.C. 3559 and 3571) or imprisonment for not more than 5 years, or both. The word “swear” should be stricken out wherever it appears when a person elects to affirm the affidavit rather than to swear to it.)

State of Maryland

County of ______________________________

Sworn and subscribed to me before a Notary Public in and for the aforementioned County and State this ___________________ day of _________________ 2009.

________________________________________

(Notary Public)

Exhibit 3.  Westat Confidentiality Pledge

A.11. Sensitive Questions

NHES is a voluntary survey, and no persons are required to respond to the interviews. In addition, respondents may decline to answer any question in the survey.

Some items in the surveys may be considered sensitive by some respondents.

ECPP and PFI Surveys. Child development and education experts consider economic disadvantage and children’s disabilities to be important factors in children’s preschool and school experiences and family involvement in their education. As a result, the ECPP and PFI Surveys contain measures of these characteristics. These include:

• Household income;

• Receipt of public assistance in the form of Temporary Assistance to Needy Families (TANF), food stamps, and the Women, Infants, and Children program (WIC); and

• Children’s disabilities.

Measures of household income and government assistance are important because access to early childhood programs of children at risk and the educational involvement of families of different socioeconomic backgrounds are of interest to policy makers, child development specialists, and educators. These items are important to identifying children at risk and have been administered successfully in previous NHES studies.

The 2007 response rates for these items were very high. For total household income, the 2007 PFI survey had an item response rate of 90.45 percent. Those for receipt of public assistance were also high: for Temporary Assistance to Needy Families, 98.86 percent; for Women, Infants, and Children Program, 99.05 percent; and for Food Stamps, 99.03 percent. The 2007 item response rates for questions about children’s disabilities were all over 99 percent.

ECPP Survey: In addition to the items above, the ECPP Survey also includes questions about assistance to pay for child care. This measure is important to understand families’ and children’s access to early childhood programs.

PFI Survey: The PFI Survey includes items concerning children’s school performance and difficulties in school. Among these are:

• Children’s school performance and difficulties, including school grades, suspensions and expulsions; and

• Identification of children’s schools.

Items concerning school performance and difficulty are important to the PFI Survey as indicators of school readiness for young children, and as correlates of parent and family involvement for children of all ages and grades. Item response rates for these PFI items in 2007 were high: 98.22 for children’s grades, 99.57 for out-of-school suspension, and 99.64 for expulsion.

Another element of the surveys that may be sensitive to some parents is the identification of children’s schools. This feature allows analysts to link NHES data to other NCES datasets containing additional data about schools, greatly enhancing the ability to examine the relationships between students’ and families’ experiences and the characteristics of schools. This was done for the first time the PFI Survey of NHES:2007; the item response rate for the identification of the child’s school was 92.53 percent.

A.12. Estimated Response Burden

The response burden per instrument and the total response burden for NHES:2009 are shown in table 2. The administration times for the Screener, ECPP, and PFI questionnaires are based on practice administrations.

The expected number of respondents and number of responses are based on the expected numbers of completed surveys of each type, discussed in section B.1.4. The cost to respondents for the total hour burden is estimated to be $27,036, that is, $19.88 per hour for 1,360 burden hours. The hourly rate is based on the average for all civilian workers from the 2007 National Compensation Survey (http://www.bls.gov/ncs/ocs/sp/nctb0298.pdf). There are no other costs to respondents. There are also no recordkeeping requirements associated with NHES:2009.

Table 2.  Estimated response burden for NHES:2009

Instrument	Avg. Completion time	Sample size	Expected number of completed questionnaires	Expected response rate (%)	Total burden hours (completes * avg. completion time)
Screener Questionnaires
Engaging screener (NHES 2009 Screener Engaging.pdf)	8	3,934	2,384	60.6	318
Core Item Screener (NHES 2009 Screener Core.pdf)	5	3,133	1,899	60.6	158
Bilingual (swim lane) Core (NHES 2009 Screener Bilingual.pdf)	5	800	485	60.6	40
Screen out Screener (NHES 2009 Screener Screen-out.pdf)	3	3,934	2,384	60.5	119
Nonresponse Screener (NHES 2009 Screener Nonresponse.pdf)	1	4,670	700	15.0	12
Topical Surveys
Early Childhood Program Participation (NHES2009 ECPP)	20	812	653	80.4	217
Parent Family Involvement– Enrolled Student (NHES2009 PFI-Enrolled)	20	1,650	1,326	80.4	442
Parent Family Involvement -Homeschooled Student (NHES2009 PFI-Home-schooled)	20	203	163	80.4	54
Total	NA	NA	9,994	NA	1,360

A.13. Annualized Cost to Respondents

There are no costs beyond those presented in section A.12.

A.14. Annualized Cost to the Federal Government

The total cost of NHES:2009 to the government is approximately $600,000 over a period of 18 months. This includes all direct and indirect costs of the design, data collection, analysis, and reporting phases of the study, as well as the delivery of data sets to NCES.

A.15. Reasons for Program Changes

This collection is a reinstatement of hours for the NHES:2009 Pilot Test, which will provide methodological data to inform the redesign and full-scale Field Test of the NHES program in 2011.

A.16. Publication Plans and Project Schedule

The primary objectives of the Pilot Test are to evaluate the operational success of the proposed methods and to get a preliminary assessment of the various methods with respect to response and cooperation rates.

The following are outcomes that will be included in this evaluation:

• Operational issues: Any events or issues that result in a delay to the implementation of each event within the Pilot Test schedule. Since many of these operational tasks are new, each process will need to be documented so procedures are available at implementation of Field Test data collection in 2011.

• Respondent feedback: Any feedback received from a respondent will be evaluated for reaction to methods (e.g., reaction to the use of government funds for incentives, difficulty of mode, etc.).

• Response rates and components: Overall and for each experimental manipulation/stage, response rates (both unit and item response rates, return rates, and any appropriate refusal conversion rates) will be tracked.

• Response timing: For each manipulation, time to response/questionnaire receipt will be assessed to determine the effects on the data collection schedule.

The NHES:2009 Pilot Test includes survey questionnaires that continue the content of the NHES program and reflect the types of instruments that NCES expects to field in future NHES collections. The two topical surveys designated for the NHES:2009 Pilot Test are based on a set of research questions and priorities developed over several survey cycles and based on reviews of literature and consultations with experts in the respective fields. The Pilot Test, however, is not intended to produce survey estimates; rather, the topical surveys are included in order to reflect types of surveys anticipated for future NHES collections.

Project Schedule

Exhibit 4 presents a schedule of major project activities.

Exhibit 4.  NHES:2009 schedule of major activities

Task	Date of Scheduled Conduct/Completion
IMT/OMB Submission	March 13, 2009
Cognitive Research	March 16 to April 30, 2009
Cognitive Report and Recommendations	May 15, 2009
Revised Teleform Questionnaires to NCES	June 1, 2009
CATI Questionnaires to NCES	July 1, 2009
OMB Clearance Completed	August 1, 2009
Interviewer Training Materials	August 1, 2009
Spanish Hard Copy Instruments	August 1, 2009
Pilot Test Begins (Screener mailing)	September 1, 2009
Outline for Field Test Report	October 1, 2009
Pilot Test Ends	December 18, 2009
Draft Pilot Test Report	March 1, 2010
Revised Field Test Report	April 16, 2010

A.17. Approval for Not Displaying the Expiration Date for OMB Approval

The OMB authorization number and expiration date will be displayed on the hard copy questionnaire but not on the CATI instruments. Telephone interviewers will have the OMB authorization number in their study materials and can give this information to respondents upon request.

A.18. Exceptions to the Certification Statement

Not applicable. There are no exceptions to the certification statement.