Part C Pilot Test justification

NHES:2009 Pilot Test

Request for

IMT/OMB Review

Part C

February 20, 2009

TABLE OF CONTENTS

Section Page

PART C. JUSTIFICATION OF NHES:2009 QUESTIONNAIRES

C.1. Screeners 2

C.2. Early Childhood Program Participation Survey 4

C.3. Parent and Family Involvement in Education Survey 8

References 15

Appendix A: Survey Letters and Instruments A-1

PART C. JUSTIFICATION OF NHES:2009 QUESTIONNAIRES

NHES:2009 data will be collected using different test versions of a household Screener questionnaire and three topical questionnaires. The NHES:2009 instruments are as follows:

• The household Screener will be used to engage potential respondents and collect the information needed to identify eligible households and persons for topical surveys. Information that will be used to assess nonresponse will also be collected. The Pilot Test includes multiple versions of the Screener, which will permit the assessment of survey questions intended to engage the respondent and different ways of collecting the information needed to identify and sample eligible household members.

• The Early Childhood Program Participation Survey (ECPP), to be administered to a parent or guardian knowledgeable about the care and education of children from birth through age 6 and not yet in kindergarten (infants, toddlers, and preschoolers).

• The Parent and Family Involvement in Education Survey (PFI), to be administered to a parent or guardian knowledgeable about the education of children enrolled in school in kindergarten through grade 12 or homeschooled for those grades. For ease of respondent use, two questionnaires were developed for this survey: one for parents of children attending school and one for parents of homeschooled children.

As noted throughout this document, the NHES:2009 Pilot Test is being conducted to test methodological alternatives for the larger NHES:2011 Field Test. The testing of multiple versions of the household Screener reflects this goal.

The Pilot Test is not intended to result in published survey estimates for the topical surveys, ECPP and PFI. However, the surveys to be conducted in this Pilot Test address topics that have been addressed in past NHES collections and are planned for future collection. For this reason, we discuss their content below as well.

The Screeners, ECPP questionnaire, and two PFI questionnaires appear in Appendix A. It is important to note that question wording and content may be modified as a result of the cognitive interviewing. Topic areas are not expected to change. The questionnaires will be formatted in the TeleForm software and Voxco CATI program as described in section A.3.

C.1. Screeners

The household Screener is an essential part of the NHES survey instrumentation. It is used to confirm that the sampled address is a residential address, determine whether the household includes potentially eligible children or youth, and collect information required for conducting within-household sampling.

For NHES:2009 there are three versions of the Screener which will be randomly assigned to households. The three versions differ in their content. Content differences are the result of the inclusion of education policy and opinion questions and response propensity questions. Households will be randomly assigned to receive the version with additional questions on education policy and opinions (“engagement” version), the version without these questions (“core” version), or the screen out version. Within the three different content versions of the Screeners are different methods of enumerating children within the household. A description of the content of the Screeners follows.

Household Characteristics. The Core and Engagement versions of the Screener contain household characteristics questions in an effort to collect information about the composition of the household. A household respondent is asked about the number of adults (age 21 or older) by gender living at the address and number of children 20 or younger living within the household. The household respondent is asked about tenure (rent/own) and length of tenure at this address. Questions are also asked about highest level of education among household members and about linguistic isolation. These questions will be used to assess characteristics that are potentially associated with nonresponse to the topical questionnaire and future contact strategies. A telephone number for the Screener respondent is collected in case future mail contact attempts fail and telephone follow-up is necessary.

Opinions on Current Issues. This content applies only to the “survey” versions of the Screener, and the specific items are subject to change based on the results of cognitive testing. A household respondent is asked general questions on the quality and safety of schools in their community and about school quality nationally. Questions are asked about education or training courses taken by adults to improve skills, keep a job, or find a new job. Also, questions related to household literacy environment are asked, such as the availability of 10 or more books, possession of magazines or other reading materials, and the number of hours the respondent reads each week. These questions have been included on the advice of methodological experts in order to explore a method of increasing response by engaging the respondent with relevant and interesting questions that better reflect the topic and sponsoring agency. Further potential uses of these items are as measures related to nonresponse to the topical questionnaire.

Child Enumeration.

All three versions of the Screener have identical content for enumerating children. Respondents will be asked for the child’s first name, initials, or nickname so the selected child can be specified in the topical questionnaire. The respondent is also asked for the child’s age, gender, enrollment status and grade. These items are used to determine which topical questionnaire will be sent to the household for the sampled child and will also be used to specify the selected child in the topical interview.

Frequently Asked Questions (FAQs). The back of each version of the Screener contains FAQs, intended to address questions that may be common among respondents. The purpose of this material is to fully inform respondents and assure them of the sponsorship, legitimacy, and privacy protections of the study.

C.2. Early Childhood Program Participation Survey

The NHES:2009 ECPP interview draws questionnaire items from several previous NHES administrations, and is very similar to the NHES:2005 and NHES:2001 ECPP interviews. The following topics are included in the survey:

• Participation in relative care, nonrelative care, and center-based programs, and Early Head Start/Head Start;

• Care/program characteristics such as location, days and hours per week, and cost;

• Care provider characteristics such as language and race/ethnicity;

• Difficulty in finding care and child care selection criteria;

• Family educational activities such as visiting a library and reading to a child;

• Emerging literacy and numeracy;

• Children’s health and disability;

• Parent characteristics; and

• Household characteristics.

Below, the survey items are discussed in more detail. The descriptions are presented in the order the questions appear in the printed instrument.

Instructions

The first page of the questionnaire identifies the selected child using information from the Screener and provides the respondent with instructions for completing the questionnaire. Contact information is provided (a toll free number) in case there is confusion about the selected child or the household has no child fitting the description given.

Section 1. Early Childhood Care and Programs

A major focus of the ECPP Survey is addressed in the first three sections of the questionnaire, which focus on types of nonparental care and education: care by relatives, care by nonrelatives, and center-based programs. Three sets of items within Section 1 collect extensive information to describe the different types of nonparental care and education that children may receive. These items provide detailed information on the extent of current participation in the different types of care arrangements and programs, location, amount of time children spend in current arrangements or programs, and the financial cost of current care arrangements or programs to the child’s household. This information can in turn be related to children’s personal, family, and household characteristics. The following information is collected in each of these series of items:

• The number of current arrangements or programs of the respective type;

• The location of the arrangement or program (including whether programs are located at parents’ workplaces);

• The number of days and hours per week the child receives care or attends a program on a weekly basis, and the length of time in attendance of the arrangement or program;

• The language of the care provider (nonrelative and center-based care only);

• Whether the care provider will care for the child when the child is sick;

• Services provided by center-based programs; and

• The cost to the household of the arrangement or program, and assistance in paying for the arrangement.

In addition to these common items, the section on relative care contains an item concerning the relationship of the care provider to the child, and an item asking the age of the relative care provider. Age of the nonrelative care provider and if the parent already knew the care provider is also obtained in the nonrelative care section. The section on center-based care asks additional items on whether the center provides health services such as hearing, speech, or vision testing, and if the center is religiously affiliated.

Care Your Child Receives From Relatives (Questions 1-17). This section collects information regarding care provided on a regular basis by relatives other than the child’s parents. This includes grandparents, aunts or uncles, brothers or sisters, etc. Occasional babysitting is explicitly excluded. The care may be provided in the child’s home or in another home. For the child care arrangement in which the child spends the most time, key facets of care such as location, hours, and cost are collected. Limited information on other relative care arrangements is collected to provide a full accounting of time spent in care without undue burden.

Care Your Child Receives from Nonrelatives (Questions 18-35). This section collects information on care provided in a private home on a regular basis by nonrelatives. This includes home child care providers, regular sitters, or neighbors. Care may be provided in the child’s home or in another home. Occasional babysitting is explicitly excluded. Key indicators (e.g., location, hours, cost) are collected for the primary nonrelative care arrangement, and hours for other arrangements are collected to provide a full accounting of time spend in care.

Day Care Centers and Preschool Programs Your Child Attends (Questions 36-54). This series collects information on programs provided for groups of children in day care centers, preschools, and pre-kindergartens. This can include daycare centers, preschools, prekindergartens, or center-based Head Start or Early Head Start programs. Key indicators (e.g., location, hours, cost) are collected for the primary center-based program arrangement, and hours for other arrangements are collected to provide a full accounting of time spend in care.

Early Head Start/Head Start (Questions 11, 29, 47, 54). Head Start is a federally funded early childhood education program for disadvantaged children. Most participants are 3 to 5 years old. Participation in an Early Head Start program may begin at birth. In each section concerning nonparental care and education, respondents are asked whether this primary arrangement is Head Start. Following the collection of relative, nonrelative, and center-based care arrangements, respondents are asked if the child has ever attended Head Start or Early Head Start. These items help to characterize the arrangements in which a child participates.

Section 2: Finding and Choosing Care for Your Child (Questions 55-58)

Access to care and early education programs is a topic of concern to experts in child development and early education and to policymakers. This section asks parents if they feel they have good choices for child care where they live and the difficulty parents have had finding the type of child care they want. The last item addresses the importance of factors in choosing child care and early education arrangements.

Section 3: Family Activities (Questions 59-63)

Activities with potential educational benefits that families can do together are measured in this section of the questionnaire. Parents are asked about reading to their children and the amount of time spent reading every day. In addition, parents are asked about other activities such as telling a story, teaching letters or numbers, or doing arts and crafts. Also asked of parents is whether someone in the family has visited a library with the child in the past month. These items are indicators of the richness of the child’s educational environment at home and the parents’ involvement in promoting the child’s development.

Section 4: Things Your Child May Be Learning (Questions 64-71)

This series measures the child’s accomplishment of developmental tasks that indicate readiness for school. Analysts and policymakers are interested in tracking progress in emerging literacy and numeracy across time and examining the relationship of emerging literacy and numeracy with preschool program participation and family educational activities.

Section 5: This Child’s Health (Questions 72-80)

This section includes items about the child’s overall health, any disabilities, and disability services the child may be receiving. The presence of disabilities is an important risk factor and is related to children’s development and educational experiences in the preschool years as well as their later experiences in school. Disability status may be associated with access to nonparental care and education programs.

Section 6: Child’s Background (Questions 81-88)

Questions about the child’s month and year of birth, race/ethnicity, and country of origin, and language will be used to examine differences in children’s participation in care and education programs. For children whose primary language is not English, a question about participation in English language education is asked. English language education may help children to participate in other early education and prepare for school.

Section 7: Child’s Mother or Female Guardian (Questions 89-104) and

Section 8: Child’s Father or Male Guardian (105-119)

These sections collect information on the child’s parents or guardians who reside in the household: mother and father, or other guardians if parents are not present. The topics for mothers include age at first becoming a mother or guardian, native language, country of origin, educational attainment, and employment status. The same items are asked for fathers, except for the age at first becoming a parent. These items on parent characteristics measure factors that may be associated with participation in early care and education and may also indicate risk factors such as low parent education that could be associated with participation, child development, and preparation for school.

Household Characteristics (Questions 120-123)

This final series of items collects information on the relationship of people in the household to the child, receipt of public assistance, and household income. Household composition is an indicator of resources for care and opportunity for interaction within the household. Receipt of public assistance and household income are used to classify families according to the economic resources available to them, and to examine the relationship of the family’s economic status to early childhood program participation. These variables, taken along with parent characteristics, can be used to identify children who may be at risk in terms of development and educational preparation.

Section 10: Questions about You (Questions 124-135)

This section contains questions collecting paradata for methodological purposes. Information about the respondent, respondent telephone use, computer and Internet use, and questionnaire completion will be used to examine response propensity, mode propensity, and possible sources of response bias. In addition, these data can be used to develop future contact strategies.

C.3. Parent and Family Involvement in Education Survey

The PFI questionnaires draw upon the PFI Surveys of 1996, 2003 and 2007. Two questionnaires were developed for ease of respondent self-administration: a questionnaire for parents of children enrolled in school (PFI-Enrolled) and a questionnaire for parents of children who are homeschooled (PFI-Homeschooled). The following topics are included in the PFI Survey:

• School characteristics such as control (public/private) and school choice (PFI-Enrolled);

• Student experiences in school, academic grades and grade repetition (PFI-Enrolled);

• Parent and family involvement/participation in school including involvement in school decision-making and barriers to such participation (PFI-Enrolled);

• School practices to involve and support families (PFI-Enrolled);

• Family involvement in children’s schoolwork (PFI-Enrolled);

• Homeschooling (PFI-Homeschool);

• Family activities such as arts and crafts, visiting a library, or attending a play or concert (PFI-Both);

• Children’s backgrounds (PFI-Both);

• Children’s health and disability (PFI-Both);

• Parent characteristics (PFI-Both)

• Household characteristics (PFI-Both)

School enrollment or homeschooling status, and grade are collected for all children in the Screener and displayed at the beginning of the self-administered topical interview. Homeschooling parents are asked detailed questions on homeschooling, home activities with their children and their child’s health. Parents of children enrolled in kindergarten through high school, are administered questions on their children’s school and their experiences in school. Parents of children in elementary or secondary school are asked about their children’s experiences in school and feedback from teachers on school performance, their involvement in school, school practices to involve families, and their involvement in homework. All parents including parents of homeschoolers are asked questions on their home activities and family involvement outside of school. Finally, parents of all children are administered items on their children’s health and disability and demographic characteristics, the characteristics of parents/guardians residing in the household, and household characteristics.

Below, the survey items are discussed in more detail. The descriptions below begin with the questionnaire for parents of children enrolled in school. Items common to both questionnaires are noted, and a discussion of homeschool-only questions follows.

PFI-Enrolled Questionnaire

Instructions

The first page of the questionnaire identifies the selected child using information from the Screener and provides the respondent with instructions for completing the questionnaire. Contact information is provided (a toll free number) in case there is confusion about the selected child or the household has no child fitting the description given.

Section 1: Child’s Schooling (Questions 1 through 19)

This section covers questions that are asked of all parents with children in elementary, middle or high school who are not homeschoolers. The section provides detailed information on the characteristics of the school that children attend. This covers topics such as whether the school is a private or public school, whether it is a school that has been assigned or is a school that parents have chosen for their children, whether a charter school, and whether a Catholic or church-related school. The characteristics of schools provide important contextual variables that may be related to parent and family involvement and children’s educational experiences.

A series of questions (Questions 6 through 9) asks for information on whether parents chose the school that their children are enrolled in, including the information that parents obtained about the school before making a choice and whether the school was the parent’s first choice for their child.

For the Pilot Test, a list of schools in the family’s area will be provided on the last page of the questionnaire. Identifying the child’s school permits linking data about schools from existing data sources at NCES, notably the Common Core of Data and Private School Universe Survey. This permits the analysis of family experiences in the context of school characteristics including public/private control, religious affiliation, school size, urbanicity, standardized test performance, and school population measures. These measures can also be used to assess the accuracy of parent responses about school characteristics.

This section of the instrument also includes items that address specifics of children’s and families’ experiences with schools that may be associated with parent involvement:

• Student experiences in school including grades and academic performance (Questions 13 and 19);

• Teacher feedback on child’s school performance and behavior (Question 14);

• Grade repetition since starting kindergarten (Questions 15 and 16 );

• Suspension or expulsion from school (Question 17); and

• Parents’ plans for their children’s education after high school (Question 18).

Section 2: Families and Schools (Questions 20-25)

A core area of interest in the PFI Survey is parents’ involvement in their children’s schools. This section includes questions on the frequency and nature of parents’ involvement in school including attending meetings, volunteering, serving on school committees, and other types of contact with school staff and teachers. Another aspect of parent’s contact with the school involves school practices to communicate with parents regarding their child’s progress and placement and the parent’s expected role. Schools contact parents for various reasons including students’ performance, school volunteering opportunities, other methods and opportunities that can help children in their education, planning for attending college or work after completion of education. The nature of contact could include contact via letters or newsletters as well as email. Finally, parents are given the opportunity to express their level of satisfaction with the school.

Section 3: Homework (Questions 26-31)

Parent and family involvement in homework is another form of support for children’s education; Section 3 addresses this topic. This section includes questions about the amount of homework that children receive, the home environment that families create for children to do homework, and the involvement of parents and other family members in ensuring that children complete their homework.

Section 4: Family Activities (Questions 32-34)

Parent and family involvement in non-school activities with potential educational benefits represents another way that parents and families can support children’s education and development. Parents of children in elementary school are asked about other activities such as telling a story, doing arts and crafts, playing sports and games or getting involved in other projects. Parents are asked to report on the activities done with their child in the past week and in the past month with anyone in the family. Parents of older children who are in middle or high school are asked similar questions about activities that are appropriate to their age. Again, parents are asked to report on the activities done with their child in the past week and in the past month with any family member. All questions in this section are also addressed to parents of homeschoolers with the appropriate grade equivalent; PFI-Homeschooled Questionnaire, Section 2, Questions 18-20.

Section 5: Child’s Health (Questions 35-43)

This section includes items to identify disabled children and to determine whether children with disabilities are receiving services. The presence of disabilities is an important risk factor for all children and is related to children’s development and educational experiences in school. These items will also provide the data to analyze the accessibility of special education and other programs and plans for disabled children. These items also appear in the PFI-Homeschooled Questionnaire, Section 3, Questions 21-29.

Section 6: Child’s Background (PFI-Enrolled, Questions 44-51)

This section collects demographic information on the child’s race/ethnicity and country of origin. These characteristics can be correlated with the various levels of family involvement in children’s education. These questions also appear in the PFI-Homeschooled Questionnaire, Section 4, Questions 30-37).

Section 7: Child’s Mother or Female Guardian (Questions 52-70)

Section 8: Child’s Father or Male Guardian (Questions 71-88)

These sections collect information on the child’s mother or female guardian and on the father or male guardian who resides in the household. The topics for mothers include age at first becoming a mother or guardian, native language, country of origin, educational attainment, and employment status. The same items are asked for fathers, except for the age at first becoming a parent. These items on parental characteristics measure risk factors that could be associated with each of the children’s development and progress through school. Other questions within both parent characteristics sections can be used to relate family involvement in education with parents’ characteristics including their education, choice of employment as well as on the number of hours they work. These questions also appear in the PFI-Homeschooled Questionnaire, Section 5, Questions 38-53 (for mothers) and Section 6, Questions 54-68 (for fathers).

For parents of children who attend school, questions about school accommodations for mothers and fathers whose primary language is not English are also included in these sections. These are important indicators of possible barriers to parent involvement.

Section 9: Your Household (Questions 89-92)

This final series of items collects information about who lives in the household with this child, receipt of public assistance, and household income. Receipt of public assistance and household income can be used to classify families according to the economic resources available to them, identify child risk factors, and examine the relationship of the family’s economic status to school participation. These items also appear in the PFI-Homeschooled Questionnaire, Section 7, Questions 69-72.

School Identification (Questions 93-94)

Following the household section, parents will be asked to identify their child’s school. A list of schools in the household’s ZIP Code will be provided (Question 97) to allot the respondent to select a school without having to write in the name and address. If the school is not on that list, the respondent is asked to provide the information (Question 94).

Section 10: Questions about You (Questions 95-106)

This section contains questions collecting paradata for methodological purposes. Information about the respondent, respondent telephone use, computer and Internet use, and questionnaire completion will be used to examine response propensity, mode propensity, and possible sources of response bias. In addition, these data can be used to develop future contact strategies. These items also appear in the PFI-Homeschooled Questionnaire, Section 8, Questions 73 to 84.

PFI-Homeschooled Questionnaire

Section 1: Homeschooling (PFI-Homeschool, Questions 1 through 17)

NHES is one of the few sources of data on homeschooling in the United States and is the only nationally representative survey on this topic. The items in the PFI-Homeschooled questionnaire include the prevalence of home schooling among sample respondents, the location and time spent homeschooling every week, and whether homeschooling organizations play a role in such instruction either directly by providing curricula and other materials. The reasons for homeschooling by parents are also covered in this section.

Other sections of the PFI-Homeschooled Questionnaire are described above; they are sections that also appear in the PFI-Enrolled Questionnaire.

References

Blumberg, S.J. and Luke, J.V. (2008). Wireless substitution: Early release of estimates from the National Health Interview Survey, January-June 2009. National Center for Health Statistics. Available from: http://www.cdc.gov/nchs/nhis.htm. December 17, 2008.

Brick, J.M., Hagedorn, M.C., Montaquila, J., Brock Roth, S., and Chapman, C. 2006. Monetary Incentives and Mailing Procedures in a Federally Sponsored Telephone Survey: Methodology Report. U.S. Department of Education. Washington, DC: National Center for Education Statistics.

Dillman, D.A., Sinclair, M.D., and Clark, J.R. (1993). Effects of questionnaire length, respondent-friendly design, and difficult questions on response rates for occupant-addressed Census mail surveys. Public Opinion Quarterly, 57, 289-304.

Dillman, D.A., Smyth, J.D., and Christian, L.M. (2008). Internet, mail, and mixed mode surveys: The Tailored Design Method. New York: Wiley.

Hagedorn, M., Roth, S.B., Carver, P., Van de Kerckhove, W., Smith, S., and Mulligan, G. (forthcoming). National Household Education Surveys Program of 2007: Methodology Report. National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC.

U.S. Department of Labor. (2008). National Compensation Survey: Occupational Wages in the United States: July 2007. Washington, D.C.: Bureau of Labor Statistics. Available online at http://www.bls.gov/ncs/ocs/sp/nctb0298.pdf