ED Response to OMB Password

September 11, 2011

MEMORANDUM

To: Shelly Martinez, OMB

From: Sarah Grady, NCES

Through: Kashka Kubzdela, NCES

Re: NHES 2015full scale data collection – NCES response to OMB passback (OMB# 1850-0768 v.10)

The TWS topical module is now.  Other than that, can you confirm that there are no changes to the topical modules from their last use (2012)?  If there are changes, we’d like to see a summary of those please.

The CWS (Credentials for Work) module and TWS (Training for Work) module are both new to NHES:2015. The TWS is being fielded as a pilot. With regards to the Early Childhood Program Participation Survey (ECPP) and the Parent and Family Involvement in Education Survey (PFI) modules, minor changes were made. The following text was added to the NHES:2015 Surveys subsection of preface in Supporting Statement Part A:

“As a result of analysis of the NHES-FS results, some of the proposed content in the 2015 PFI, related to homeschooling, has been revised. NHES has traditionally experienced under-reporting of homeschoolers, and the switch to the mail mode likely exacerbated under-report of homeschoolers because of the use of self-administered screener, which was a change from prior NHES administrations when a trained interviewer helped respondents who homeschooled their children (full-time or part-time) navigate to the homeschool path rather than the enrolled path. However, at the second stage of the mail survey, we found evidence that the NHES is yielding over-reports of homeschooling. Specifically, results from the NHES-FS indicated that a smaller proportion of the respondents to the ASPA who indicated that their child was homeschooled would actually be considered a homeschooler under NCES reporting conventions. NCES considers any child who attends school for more than 25 hours per week to be enrolled rather than homeschooled. Therefore, although we still believe that the mail screener operation results in under-reporting of homeschooling, we know from the NHES-FS that when we ask respondents to a questionnaire designed for enrolled students whether or not the child is homeschooled, we get over-reporting.

For the reasons discussed above, we have revised the 2015 PFI-Enrolled questionnaire so that it:

1. Moves the homeschooling questions to the end of the first section (we believe that having the homeschooling question toward the beginning of the first section of the NHES-FS ASPA instrument led respondents to believe that we were trying to ask them something about their child’s enrollment rather than asking if the child may actually be homeschooled for some or all classes).

2. Adds an additional confirmation question which attempts to further filter out students who are enrolled rather than homeschooled from the homeschooling sequence (the NHES CATI instruments used up until 2007 utilized multiple confirmation questions to try to clarify homeschooling status; adding multiple confirmation questions to NHES 2015 will parallel the approach used in the CATI instruments).

3. Asks parents of students of homeschoolers who receive the PFI-Enrolled questionnaire their main reason for homeschooling (the item comes from the PFI-Homeschooling questionnaire) so that we can get more accurate estimates of this key data point.

Because the homeschooling questionnaire design poses these challenges and also attempts to measure constructs that are observed in a small proportion of the school-aged population, we plan to cognitively test these items in early October 2014 with 5-9 respondents. NCES staff will conduct the cognitive interviews, and will recruit participants through personal networks and contacts with homeschooling policy organizations. If the proposed content is found to introduce new sources of measurement error, we will change the content to address the error. Should the cognitive interviews result in any wording changes, we will provide the updated questionnaire through a change request.”

The details of the changes made are provided below, and reflected in updated NHES 2015 Appendix B & C Survey Instruments.pdf.

Changes to NHES screener instrument from 2012 to 2015

**Note: The 2012 screener asked for an enumeration of only children in the household because the 2012 NHES only included child topical surveys. However, since the 2015 NHES includes the ATES survey, the 2015 screener asks for an enumeration of all individuals in the household (adults and children).

1. CHANGE: The 2012 age item asked for the individual’s age in years, with a check box for babies less than one year old. The 2015 version (Item 3) asks for the person’s month and year of birth because the 2014 NHES:FS data indicated that using age or date of birth yielded no statistically significant differences in unit or item nonresponse.

2. CHANGE: The 2012 item listed “homeschool” as the second option, but the 2015 item has “homeschool” as the first option in the list of enrollment options. Due to a history of screener-level under-reporting (as discussed in part A), homeschool is now the first option in the list of enrollment types on the screener.

Changes to NHES ECPP topical instrument from 2012 to 2015

1. ADDITION: On page 10, item h about Quality Rating and Improvement Systems (QRIS) was added to 2015 instrument as a factor that parents may consider when choosing care (Section 2, Item 58.) The item was developed in collaboration with the Office of Early Learning within the Department of Education’s Office of Elementary and Secondary Education in order to measure the usage of QRIS among parents.

2. CHANGE: Item 84 used to include a box above the response options that said “Child no longer has condition.” We found in the 2012 data that respondents were often marking that box and marking the boxes below, too, so we opted to delete it.

3. CHANGE: Item 90 added a third response option in 2015 for children who live at two addresses for an equal amount of time to accurately capture all possibilities.

4. CHANGE: Item 91 brackets were enlarged around the first two response options for the 2015 instrument because it is more noticeable to respondents.

5. CHANGE: “Live in this household” was underlined in items 109 and 110 to call to the respondents’ attention to the fact that they should only be reporting on people who live with the child in the household.

6. CHANGE: Parent marital status became a series of three items (95-97) in 2015 instead of the single question used in 2012 (also for Parent 2 items 115-117) because these items reflect current Federal research on accurate measurement of same-sex marriage.

Changes to NHES PFI Enrolled topical instrument from 2012 to 2015

1. C HANGE: The item #2 below was deleted in favor of an expanded homeschool section, as described in Part A and reflected in items 26-30 below (Items 26-28 were adapted from the NHES 2007 instruments. Items 29 and 30 are directly equivalent to items on the NHES PFI-Homeschooling questionnaire):

2. CHANGE: Capitalized “Advanced Placement” and added (AP) in 2015 instrument for greater clarity.

3. ADDITION: An additional question about reason for internet instruction was added to the 2015 instrument in order to ascertain why the child was taking courses from the internet. It was developed and tested by AIR and NCES, and the question stem was adapted from the “main reason for homeschooling” question.

4. CHANGE: In item 51 removed the “child no longer has condition” box that was on the 2012 version. We found in 2012 that respondents were often marking that box and marking the boxes below, too, so we opted to delete it.

5. CHANGE: Added third option to 2015 instrument for children who spend equal time at two addresses (item 57).

6. CHANGE: Enlarged brackets around first two response options in 2015 instrument.

7. CHANGE: “Live in this household” was underlined in items 109-110 to call to the respondents’ attention the fact that they should only be reporting on people who live with the child in the household.

8. CHANGE: Parent marital status became a series of three items (62-65) in 2015 instead of the single question used in 2012 (also for Parent 2 items 85-87) because these items reflect current Federal research on accurate measurement of same-sex marriage.

Changes to NHES PFI Homeschool topical instrument from 2012 to 2015

1. CHANGE: Removed the “child no longer has condition” box that was on the 2012 version. We found in the 2012 data that respondents were often marking that box and marking the boxes below, too, so we opted to delete it.

2. CHANGE: Added third response option to item 42 on the 2015 version for children who spend equal time at two addresses to accurately capture all possibilities.

3. CHANGE: Brackets enlarged around first two response options of item 43 in 2015 version so that respondents would notice the navigation.

4. CHANGE: “Live in this household” was underlined in items 109-110 to call to the respondents’ attention the fact that they should only be reporting on people who live with the child in the household.

3. CHANGE: Parent marital status became a series of three items (47-49) in 2015 instead of the single question used in 2012 (also for Parent 2 items 67-69) because these items reflect current Federal research on accurate measurement of same-sex marriage.

4. DELETION: Across topical surveys the 2012 item 136 on ECPP, 108 on PFI Enrolled, and 88 on PFI Homeschool, was dropped for the 2015 version because it is provided on the frame by the address vendor.

3