MEMORANDUM
To: Shelly Martinez, OMB
From: Andy Zukerberg, NCES
Through: Kashka Kubzdela, NCES
Re: Response to questions on proposed NHES content (OMB# 1850-0768 v.7)
In developing our original list of items dropped, we used the most recent version of each topical survey that was fielded. For the Parent and Family Involvement in Education (PFI) survey, this may have been misleading as the 2007 PFI was a special administration of the form that included several items from the School Readiness (SR) survey. These items are not a regular part of the PFI interview and were dropped when we reverted back to the regular PFI survey. If the redesign is successful, we hope to field the School Readiness survey and other NHES modules in upcoming years.
Students with disabilities:
NCES indicated that questions were dropped because it appeared that researchers weren’t using the information collected or they could collect the data through a more targeted question (in some cases). Please describe how the students with disabilities data is/was being used by researchers.
We did not drop any items from the 2007 disability section. We have actually expanded the disability section to include functional disabilities and additional conditions. After the August 11 briefing, we consulted with the Office of Special Education Programs and others within the Department and revised our question stems and some response categories to better conform to recent legislative changes. The revised items are presented at the end of this memo. It is possible that the comment regarding deletion refers to the low birth weight items that were part of the School Readiness survey in 2007. These items were dropped because they were collected as part of the 2007 School Readiness survey to provide additional background context in which to evaluate children's development. The 2011 and 2012 NHES will not field a School Readiness survey and therefore the additional background context is not relevant to the 2011 and 2012 NHES.
School Choice:
What is the rational for dropping the two school choice questions, especially given there doesn’t seem to be a feasible alternative data source.
These were one-time items collected only in 2007 that were intended to enhance our traditional school choice items. There were some data quality concerns around the key school choice item we dropped:
Does your public school district let you choose which public school you want (CHILD) to attend, either in your own school district or another district?
Respondents that previously reported their child was in a school of choice or a charter school, indicated in this item that their district does not allow choice. While the item had these issues, there is some thought within NCES that it could still be beneficial to include. One option would be to talk with Westat about the implications of adding it to our experimental questionnaire (we are conducting a split panel test with alternate wording and items in 2011) and monitor the results from the field test. Given the proximity to the start of the field test, it may be difficult to include the item at this point. We recommend leaving it out in 2011 and spending some time over the next year refining the item for inclusion in the 2012 national data collection. We could conduct cognitive testing and/or debriefings with respondents from the 2011 field test to improve the quality of the item.
ECPP:
Given that the revised questionnaire drops questions on care arrangements other than the primary care, we suggest adding a question about the stability of that care as low income families in particular often cycle in and out of care. Such a question might be “How many different child care providers have you used in the past year?”
Because we agree that stability of arrangements is an interesting area and would be useful to include in NHES, we tested two different items that attempted to measure this issue in our recent cognitive lab work but were unable to identify a question that was reliable enough to include in the survey. The general issues around this item stem from respondent’s difficulty in defining an arrangement. For example, some participants wondered if babysitters counted. Respondents with children in center based care were unsure if they should count each teacher or just the overall school. Despite an instruction to exclude care from parents, some respondents continued to include themselves in the count. For these reasons, we decided not to include the question in 2011. We recognize the value of data on stability and are interested in continuing work on this item for future studies.
We’d like to explore adding a question on whether the care provided is licensed. This would apply to all three types of care (relative, non-relative, and center-based). We recognized that most families in all income brackets are not likely to know accurately whether their child care provider is licensed. However, we are placing increasing emphasis on Quality Rating and Information Systems and it would be helpful to know the extent to which parents use such information.
This issue has been raised in discussions around other early childhood education surveys. In addition to the concern you have raised about parent’s knowing if a provider is licensed, we are concerned about social desirability issues. Will parents be willing to report that their child is in an unlicensed care arrangement or will all report that it is licensed? There are complexities in asking across all types of care as states differ on licensing requirements, especially for relative care. This topic is well suited to the NSECE as it will have a provider component which can be used to validate responses and look at accuracy of parents reporting.
On the issue of time spent in child care, there are questions in each section on days each week and hours each week. What’s ED’s view on adding months per year? This is sometimes an issue in center-based care.
We collect information on when the child started the reference arrangement to examine longevity of arrangements. Months per year would only be able to be analyzed for children who have been in care for the entire year preceding the survey. It would be complicated for parents of infants and children that recently placed their child in an arrangement to provide this information. We recommend against adding an item about months to the ECPP.
PFI:
Could a question be added about family/child mobility and how this relates to schools, that is, get information on the extent to which the child does or does not stay in one school all year? It’s a significant issue in many urban schools and it might be helpful to get a better handle on this problem. Have mobility questions been included in the past in a household survey, and if so, what has been the experience with them?
Past PFI’s have measured household mobility through a question about how many times the household moved in the past 3 years. However, this does not reflect school mobility. The PFI collects the month when the child started in their current school. This allows researchers to identify children that have changed schools during the current school year. Measuring change of schools in more detail is complicated by the differences in grade ranges for elementary, middle, and high schools nationwide. If OMB thinks it is desirable to measure school based mobility, we recommend starting with an item that asks how many schools the child has attended since starting kindergarten.
Coordination with HHS surveys:
In response to questions about whether specific information could be gathered from other sources, the answer for ECPP-related questions was often “Possibly National Survey of Early Care and Education (NSECE).” What steps is ED taking to coordinate ED surveys and HHS surveys to ensure broad coverage while eliminating unnecessary duplication? Is there on-going coordination? If not, what might be a feasible approach to coordination?
Project Officers from both agencies recognize the importance of collaborating on these studies and have been proactive in ensuring coordination. We have met with Ivelisse Martinez-Beck, the NSECE project officer, to discuss progress on the NHES redesign and NSECE design. At these meetings we have discussed everything from questionnaire development, including sharing results from cognitive lab work, to operational issues. The NSECE is scheduled to be fielded at the same time as the NHES. We have discussed the possibility of de-duplicating sample frames as both will be using an address based sample. While the two studies overlap in some areas, we have different objectives and methodologies that must be balanced. The NSECE focuses on care of children up to age 13, while NHES focuses on children through age 6 and not in kindergarten. NHES provides a regular overview of early child care representative at the national level. The NSECE occurs less regularly and oversamples low income populations. As a result of this focus, the NSECE collects more detailed data on program participation than NHES.
We have talked with HHS (Office of Planning, Research and Evaluation, Administration for Children and Families) about participating in an optional survey of child care workers that they have proposed conducting in conjunction with the NSECE. This study would provide information on background, tenure, and other characteristics of care providers. We feel that this optional study would greatly enhance our knowledge about provider quality and would be an asset to users of both NHES and NSECE. We have indicated our willingness to participate in the design of this study and HHS has agreed to involve us in the design process.
With regards to the school choice piece, we feel it is of sufficient importance that we really do not want to see it dropped. While there is a preference to include it in the 2011 field test, we would be okay with the NCES proposal to work on a better question “on the side” to include in the 2010 collection.
As noted in the original crosswalk, a total of four school choice items were dropped. However, most of these items were used for the first time in 2007 and not considered a regular part of the NHES-PFI. The questions we received from OMB last week referred to two dropped items and we focused on PG4 (see below) in our response. We assume that this is the item referred to above, and we can add it back to the experimental form in the field test. Ideally, we would like to debrief some of the respondents on this item to help us improve it for full scale collection in 2012. We plan to submit a separate request under the 1850-0803 generic clearance to conduct respondent debriefings during the field test and will include this item in that request.
Upon a phone conversation with OMB on October 5th, 2010, we have added the following question to the Alternate Topical PFI questionnaire (NHES 2011 alternate PFI enrolled.docx; question number 7), and an accompanying justification to Part C of this package (Part C NHES 2011-2012 Field Test.docx):
Does your public school district let you choose which public school you want this child to attend?
This may include applying to a magnet program in a public school, transferring to another public school within the district, or transferring to a public school outside of the district.
No
Yes
Don’t know
With regards to the following question about a mobility item (pasted below for convenience):
“Could a question be added about family/child mobility and how this relates to schools, that is, get information on the extent to which the child does or does not stay in one school all year? It’s a significant issue in many urban schools and it might be helpful to get a better handle on this problem. Have mobility questions been included in the past in a household survey, and if so, what has been the experience with them?
Past PFI’s have measured household mobility through a question about how many times the household moved in the past 3 years. However, this does not reflect school mobility. The PFI collects the month when the child started in their current school. This allows researchers to identify children that have changed schools during the current school year. Measuring change of schools in more detail is complicated by the differences in grade ranges for elementary, middle, and high schools nationwide. If OMB thinks it is desirable to measure school based mobility, we recommend starting with an item that asks how many schools the child has attended since starting kindergarten.”
We also consider this item of sufficient importance that we would like to see it pursued. Can you explain further the reasoning behind why measuring change in schools in more detail is complicated? Has NCES ever field-tested a question such as, “How many different schools has the child attended in the current school-year?”
We understand the importance of curricula continuity/incontinuity which was the primary purpose of the question about when the child started to attend the current school. In the 2007 NHES-PFI, approximately 3% of the K-12 students changed school during the current school year. Given this low starting point, we would not anticipate capturing enough students moving schools more than once during a school year to derive stable estimates based on data about multiple moves in the current year. Instead, we would propose asking a question similar to the past ECLS item: “How many times has your tenth grader changed schools since he or she entered first grade? Do not count changes that occurred as a result of promotion to the next grade or level (for instance, a move from elementary school to a middle school or from a middle school to a high school in the same district).”
For the self administered NHES, we would propose wording it (if child is not in kindergarten):
”How many times has this child changed schools since he or she entered first grade?
Do not count changes that occurred as a result of moving to the next grade or level (for example, do not count moving from elementary school to a middle school or from a middle school to a high school).”
Per a phone conversation with OMB on October 5th, 2010, this question has not been included in the NHES 2012 questionnaire.
Disability Items:
PFI
In general, how would you describe this child's health?
Has a health or education professional told you that this child has any of the following conditions?
a. A specific learning disability
b. Intellectual disability
c. A speech or language impairment.
d. A serious emotional disturbance
e. Deafness or another hearing impairment
f. Blindness or another visual impairment not corrected with glasses
g. An orthopedic impairment
h. Autism
i. Pervasive Developmental Disorder (PDD)
j. Attention deficit disorder, ADD or ADHD
k. Developmental delay
l. Traumatic brain injury
m. Another health impairment lasting 6 months or more
Is this child receiving services for his/her condition?
Are these services provided by any of the following sources?
a. Your local school district
b. A state or local health or social service agency
c. A doctor, clinic or other health care provider
Are any of these services provided through an Individualized Educational Program (IEP)?
Did any adult in your household work with the service provider or school to develop or change this child's IEP?
During this school year, to what extent have you been satisfied or dissatisfied with the following aspects of this child's IEP?
a. The service provider's or school's communication with your family?
b. The child's special needs teacher or therapist?
c. The service provider's or school's ability to accommodate the child's special needs?
d. The service provider's or school's commitment to help your child learn?
Is this child currently enrolled in any special education classes or services?
Does this child's condition interfere with his/her ability to do any of the following things?
a. Learn?
b. Participate in sports, clubs, or other organized activities?
c. Attend school on a regular basis?
d. Make friends?
ECPP
In general, how would you describe this child's health?
Has a health, education, or early intervention professional told you that this child has any of the following conditions?
a. A specific learning disability
b. Intellectual disability
c. A speech or language impairment
d. A serious emotional disturbance
e. Deafness or another hearing
f. Blindness or another visual impairment not corrected with glasses
g. An orthopedic impairment
h. Autism
i. Pervasive Developmental Disorder (PDD)
j. Attention deficit disorder, ADD or ADHD
k. Developmental delay
l. Traumatic brain injury
m. Another health impairment lasting 6 months or more
(If child is under 3 years old) Has a health, education, or early intervention professional told you the child is "at-risk" for a substantial developmental delay?
Is this child receiving services for his/her condition?
Are these services provided by any of the following sources?
a. Your local school district
b. A state or local health or social service agency
c. A doctor, clinic, or other health care provider
Are any of these services provided through an Individualized Family Service Plan (IFSP), or an Individualized Educational Program (IEP)?
Did any adult in your household work with the service provider or school to develop or change this child's IFSP or IEP?
Since September, to what extent have you been satisfied or dissatisfied with the following aspects of this child's IFSP or IEP?
a. The service provider's or school's communication with your family?
b. The child's special needs teacher or therapist?
c. The service provider's or school's ability to accommodate the child's special needs?
d. The service provider's or school's commitment to help your child learn?
Is this child currently enrolled in any special education classes or services?
Does this child's condition interfere with his/her ability to do any of the following things?
a. Learn?
b. Participate in play with other children?
c. Go on outings?
d. Make friends?
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Authorised User |
| File Modified | 0000-00-00 |
| File Created | 2021-02-01 |