December 6, 2011
MEMORANDUM
To: Shelly Martinez, Kashka Kubzdela
From: Gail Mulligan, Jill Carlivati
Subject: Responses to Passback (11-17-2011, 11-30-2011, and 12-1-2011) for the ECLS-K:2011 Spring First and Fall Second Grade National Data Collections Clearance Package (OMB# 1850-0750 v.10)
Is ECLS-K:12 still using newsletters to parents as was used in ECLS-B and in the prior round of ECLS-K?
If so, please submit a copy.
We are considering using a newsletter in future rounds of the ECLS-K:2011, however we do not have one at this time. A newsletter could not be developed for the spring first-grade round because the data from the kindergarten rounds of the ECLS-K:2011 are not yet approved for release.
We would like to see some notification to parents that the study is annual and until fifth grade, as the current consent materials focus solely on the current round of collection. It doesn’t appear that parents have been told or reminded of that fact since initial recruitment, when a sentence was included in the study brochure. It is our understanding that the newsletter was a way in past studies to remind parents that the study was recurring over time. While we are flexible on the vehicle and exact wording, we do want to see reference to the future follow-ups included somewhere in this package.
We have added text to the parent letters for spring first grade and fall second grade reminding parents that the study goes until 5th grade (a revised Appendix A is attached). Specifically, we changed:
“We are looking forward to talking with you and your child during this academic year.”
to
“We are looking forward to talking with you and your child during this academic year and to your continued participation through fifth grade.”
Please clarify the references to a 30% and 50 (to 60)% subsample in the supporting statement. It may be useful to attach actual estimates to each of these as well as how they relate to each other.
Related to the subsampling over movers, we have updated the text in part B, pages B5, B22, and B23 (a revised Part B is attached). Based on final sample size information for the kindergarten waves, we believe that we can subsample movers at a rate of 50 percent and maintain the desired sample sizes for each round.
The 30 percent subsample mentioned here and the 50 percent subsample mentioned above are referring to different subsamples that were initially selected independently of one another. The 30 percent subsample refers to the subsample of all study children who are included in the fall first and second grade collections. The 50 percent subsample refers to the group of all study children who are flagged to be followed if they move from the school in which they were sampled. Some of the children in the 30 percent subsample for the fall collections are also part of the 50 percent of children flagged to be followed if they change schools. Thus, some children are part of both subsamples. We have included a footnote at the bottom of page B-5/6 to help clarify this.
Please update the subsampling plans, included for subgroups, given the passage of time since the supporting statement was written.
We have updated the text related to subsampling of movers in part B, pages B5, B22, and B23. Based on final sample size information for the kindergarten waves, we believe that we can subsample movers at a rate of 50 percent and maintain the desired sample sizes for each round.
Please explain the rationale for differential school incentives described in footnote 6 of SS A9. Please also explain how this will be handled operationally (e.g., the letter for clearance to transfer schools mentions the incentive, so should there be two separate transfer school letters?).
Our remuneration plan is intended to acknowledge the burden that schools experience from, among other things, the multiple days assessors are in the school and providing space for a team of assessors. The burden is much less when there are fewer children. This exception primarily applies to transfer schools, which typically have only one sampled child who has transferred from an original sample school. Since only one sampled child is in the school, the visit is less than ½ a day and the space required only needs to accommodate one testing area. Because there are fewer requirements for space and time, our staff have much more flexibility with the schedule than in schools that have more children.
Both the spring first grade and fall second grade letters mention remuneration for school staff. School staff in transfer schools will receive individual incentives, so that text is accurate. Transfer schools will not receive the $200 incentive given to the school as an organization, and we appreciate your noting the problem with the fall second grade transfer school principal letter. We have edited the fall second grade letter to remove reference to this incentive. The sentence related to staff incentives that appears in the fall second-grade letter is now identical to the sentence related to staff incentives that appears in the spring first-grade letter.
In SS A 10, please either remove the affidavit of nondisclosure and/or remove references to CIPSEA, as recently discussed with NCES.
We have removed the affidavit of non disclosure from p A10, as well as the reference to it as an exhibit in both the table of contents and the text.
Please provide the “Best Practices” guide referenced in footnote 6 to SS B.
We referenced this guide in error in the footnote. Such a guide was used by assessors for the ECLS-K and the intention had been to use it for the ECLS-K:2011. However, it was not used. The contractor decided that recruitment would be better done by school recruiters who are experienced with getting schools on board and focused their resources on school recruiters and their interaction with schools. Also, we lacked adequate time to update the guide to make it applicable to the ECLS-K:2011. Instead of the guide, team leaders had access to the respondent materials on their laptops, which they could reference if necessary, when speaking to school staff. They could also request additional copies from the contractor’s home office if school staff requested them. We have corrected the footnote.
Since the brochure and a timeline are going to continue to be used, they should be included in this package. Please provide.
The brochure is being used and has been included at the end of the revised Appendix A. The timeline is no longer being used as it presents information only for the kindergarten year. We apologize for our oversight in not removing this reference before submitting the package. We have deleted all references to the timeline in the text of part B.
Please discuss in more detail estimated base year response rates and work done to date and planned for looking at nonresponse bias of base year nonrespondents. Specifically, what analysis of substantive variables (rather than standard demographics) is possible and planned?
We have updated the text in the attached part B, pages B22-B23, with this information.
Has NCES discussed the disclosure risk to teachers or others associated with providing certificates to teachers with the study name on it?
Due to the issue of disclosure risk, we have eliminated the certificate of appreciation from the respondent materials appendix (A) and will not use it for these national data collections. We also have eliminated the reference to it on page A-25 of Part A.
In a memorandum dated March 13, 2009, in response to questions from OMB, NCES indicated the following:
Questionnaire B
Q1. Please include students receiving services through IFSPs in this question.
Response: The special ed questionnaires submitted in the revised package will incorporate this recommended change.
After Q5, please add a question about whether the child had an IFSP during the year prior to kindergarten.
Response: The special ed questionnaires submitted in the revised package will incorporate this recommended change.
Subsequently, it does not appear that there changes were made. Please confirm:
Whether these changes were made to the base year special education teacher questionnaires. [Please provide a copy of the questionnaire actually administered in that year.]
Whether they were meant to be included in the questionnaires currently pending at OMB.
These specific changes were not incorporated into the special education teacher questionnaire. Though this exclusion was intentional, it is not clear whether we communicated the reason for not making this change to OMB and we apologize for the oversight. To explain:
During the March 2009 OMB review of the package we submitted for clearance for the first ECLS-K:2011 field test and preliminary clearance for the kindergarten national data collections, OMB had asked that we ascertain certain information about children having an IFSP. We proposed including a question in the parent interview asking about children having an IFSP and indicated that we would also make these changes to the special education teacher questionnaire. We did add the question to the fall kindergarten parent interview that was submitted in the request for approval for national data collection in February 2010 and cleared in May 2010. The special education teacher questionnaire cleared in May 2010 did not have the noted changes.
The original questions that were referenced were the following:
1. Is this student currently receiving special education services or gifted/talented services through an IEP? MARK ONE RESPONSE ONLY.
Special education services due to a disability (GO T QUESTION 2)
Gifted/talented services (SKIP TO END. YOU DO NOT NEED TO COMPLETE SPECIAL EDUCATION TEACHER
QUESTIONNAIRE A.)
5. Did this child have an IEP during the year prior to kindergarten? MARK ONE RESPONSE ONLY.
Yes
No
Don’t know
In subsequent research about IFSPs, we determined that services are provided through an IFSP for children from birth to age 3. After some internal discussion, we decided that it didn’t seem appropriate to ask about IFSPs in these locations, which are asking whether a child in kindergarten currently has an IFSP or whether the child had a IFSP the year before kindergarten. We felt that the parent would be the best reporter of such information, especially if the child had an IFSP when he/she was very young. Before we submitted the kindergarten national data collection clearance package in February 2010, we met with OMB (on 2-18-2010) to discuss our field test experiences and to brief OMB on larger changes they could expect in the forthcoming clearance package. Our notes indicate that we discussed the additions we had made to our questionnaires as a result of OMB requests during review of the field test package, and specifically that we had noted the addition of the question to the parent interview. However, our notes do not mention discussion of not making changes to the special education teacher questionnaire. Given that changes based on prior OMB requests were a topic of discussion at the briefing, it seems possible that this issue was discussed at that meeting. However, since we do not have written documentation to that effect, we cannot say for certain that that was the case. Again, we are very sorry for the oversight. Given the age limit for receipt of services through an IFSP, we do not believe we’d want to make these changes for the spring first-grade questionnaire, but please let us know if you disagree.
We would like to request permission from OMB to make a small change to the respondent materials. The main respondent materials for spring first grade and fall second grade (i.e., not the hearing screening materials), include the following sentence: “The hearing screenings are being conducted in collaboration with the National Institute on Deafness and Other Communication Disorders (NIDCD), which is part of the U.S. Department of Health and Human Services.”
Between the time we submitted these materials in July and the time we submitted revised hearing screening materials in September, NIDCD indicated that they preferred to be identifies as part of the National Institutes of Health (NIH). Thus, in the hearing screening materials, they are referred to as follows: “National Institute on Deafness and Other Communication Disorders (NIDCD), which is part of the National Institutes of Health (NIH).”
For consistency across the sets of materials and in accordance with NIDCD’s preference, we would like to change the reference from HHS to NIH in all letters. If this is agreeable to OMB, we will make the change in relevant materials before OMB’s approval.
We are okay with everything except are still confused about the sampling plan: From what I can tell, the base year study respondents are the new “universe” for the first grade follow ups. It’s not clear where the 30 percent comes into play. You indicate that 30 PSUs will be selected from the national sample to construct the fall first grade and fall second grade subsamples. Do you mean 30% of the PSUs or n=30 (B1.4, p B-5)? By contrast, you indicate in B1.3 (p B-4-5) that the Spring first grade follow up sample will consist of 1) all base year respondents who stayed in their original school, and 2) a 50% subsample of those students who move schools. From the way this reads, there are two different sampling plans for the spring and fall assessments, with the spring assessment being given to a larger sample than the fall ones (from only 30 PSUs). Please clarify.
ECLS-K responded that you are correct that the “universe” for both the fall and spring samples is base-year (i.e., kindergarten) respondents. “For the spring samples, we subsample most movers to contain data collection costs (the exceptions are noted below), but there is no other sampling. For budgetary reasons, the fall collections are conducted with a subsample that is approximately 30 percent of the entire sample. Thus, it is correct that the spring collection includes a larger sample. However, when appropriately weighted the data always generalize to the cohort of children in kindergarten in the 2010-11 school year, regardless of the data collection round.
To explain a bit further, after kindergarten, the eligible sample for each spring collection is all children who are considered respondents in the base year, who have not moved outside the US, and for whom we have not received a firm refusal to participate at some point after kindergarten. Those children who still attend their base-year school are followed/included with certainty. Subsampling is used for most children who have moved from their base year school because it is more expensive (in terms of cost-per-completed case) to follow movers. Children with IEPs/IFSPs or who are language minorities (LM) or who are part of the fall subsample are followed with certainty. The remaining movers (i.e., 50 percent of the movers who are not IEP/LM kids or part of the fall subsample), are subsampled for following; the subsampling rate was set so that in total 50 percent of non-IEP/LM/fall subsample movers are followed in spring.
Regarding the sampling for the fall collections, there are 90 PSUs that were originally sampled for the study and in which the study schools and children were sampled. We sampled 33 percent of those original PSUS for the fall subsample, so n= 30. (Thus, 30 percent and n=30 are both correct.) This PSU sample is expected to result in a student sample that is about 30 percent of the total sample. Children included in the fall subsample are those who attended a school in kindergarten in the sampled PSUs and who are base-year respondents.
We are still struggling a bit. Please answer these questions.
Are movers in the 30 subsampled PSUs for the Fall of first grade collection sampled at a rate of 0, 50 or 100%?
100 percent (once they are followed into the fall of first grade). For clarification, for children in the fall subsample, subsampling of movers did take place between the spring kindergarten and fall first grade data collections as was described for the spring subsample. That is, any LM/IEP/IFSP children who changed schools between spring kindergarten and fall first grade were followed with certainty and 50 percent of the remaining movers were subsampled to be followed for the fall first grade collection However, there will be no more subsampling of movers beyond fall of first grade for children in the fall subsample; once a child is included in the fall subsample, that child is followed with certainty, regardless of whether that child changes schools.
Are all of those movers included in the Fall also included with certainty in the Spring (as we understand the Fall non-movers to be)?
Yes.
Is the sampling plan for the Fall and Spring of 2nd grade the exact same children as for the first grade? If not, how do they differ?
Currently, yes. We may need to reevaluate the subsampling rate for movers for the spring second-grade collection once we get farther out in the study and see our response rates/sample sizes. However, if we believe we would need to increase the rate, we would make sure to include that information in the package for approval of the spring second-grade collection.
Page
| File Type | application/msword |
| File Title | 2001 |
| Author | Daniel Princiotta |
| Last Modified By | kashka.kubzdela |
| File Modified | 2011-12-06 |
| File Created | 2011-12-06 |