Part B_ECLS-K_2011_OMB
Part B_ECLS-K_2011_OMB.doc
Early Childhood Longitudinal Study Kindergarten Class of 2010-2011
OMB: 1850-0750
Collection of Information Employing
|
B |
B.1 Universe, Sample Design, and Estimation
This current submission requests an update of the most-recent previously obtained clearance for the ECLS-K:2011 field test and full scale data collections (OMB No. 1850-0750). This submission describes the final procedures and instruments planned for the full-scale kindergarten data collections of the ECLS-K:2011, which have been informed by the experiences and results of the field test. Section B.1 includes information on the universe of interest and the sampling plan for the national study, which describes the plans for selecting primary sampling units (PSUs), schools, and kindergarten children within schools. Also discussed in Section B.1 are the precision requirements and target sample sizes.
B.1.1 Universe and Sample Design
The universe for the ECLS-K:2011 includes all children attending kindergarten in the 2010-11 school year in the 50 States and the District of Columbia. The sample design for ECLS-K:2011, which will be similar to that for the ECLS‑K, will produce a sample that is nationally representative of this population of children in the United States. In the base year (i.e., kindergarten year), children will be selected using a multistage probability design. In the first stage, 90 primary sampling units (PSUs) that are counties or groups of counties were selected with probability proportional to size (PPS). In the second stage, public and private schools offering kindergarten programs will be selected, also with PPS, within the sampled PSUs. This stage included oversampling of private schools to ensure that the sample includes enough students attending private schools to generate reliable estimates about them. The third-stage sampling units will be children in kindergarten programs and five-year-old children (i.e., children of kindergarten age) in ungraded schools and classrooms. Children will be selected within each sampled school using equal probability systematic sampling. Asians, Native Hawaiians, and Other Pacific Islanders will be sampled at a higher rate so as to achieve a minimum required sample size in order to generate reliable estimates for them. Although they will be oversampled as one group, the number of completed interviews for children in these groups is expected to be large enough in the kindergarten year to produce estimates for Asians separately from Native Hawaiians and Other Pacific Islanders. Only base year respondents will be included in the sample in subsequent years of the study. Due to the high cost of following children who change schools (i.e., “movers”), children who move from the school they attended in kindergarten will be subsampled for follow-up and inclusion in later rounds of collection. The subsampling rate will be around 50 percent but will vary between grade 1 and grade 5 by children's characteristics in order to preserve large enough groups of sampled children that are of particular analytical interest (e.g., language minority children (children from a home in which the primary language is not English)).
B.1.2 Precision Requirements and Sample Sizes
An objective of ECLS-K:2011 is to obtain a minimum level of reliability for estimates pertaining to analytical subgroups, such as Asians, Native Hawaiians and other Pacific Islanders, Blacks, Hispanics, private school kindergartners, and language minority children. Four precision requirements for the survey are identified and form the basis for the base year sample design and plans for the subsequent rounds. These requirements are the ability to do the following:
Measure a relative change of 20 percent in proportions across waves;
Measure a relative change of 5 percent in a mean assessment score across waves;
Estimate a proportion for each wave with a coefficient of variation (CV) of 10 percent or less; and
Estimate a mean assessment score for each wave with a CV of 2.5 percent or less.
The precision requirements that drive the sample design, which are the same as those used in the ECLS-K, are related to the ability to estimate changes over time and the precision of estimates in the grade 5 data collection for the sample as a whole, as well as for subgroups of analytic interest. The ECLS-K:2011 sample design began with the assumption, based on the ECLS-K experience, that at least 10,300 completed cases would be needed by the end of 5th grade to satisfy the study's precision requirements.
For the ECLS-K:2011, the minimum subgroup sample size is determined by first solving for the sample size needed to achieve the precision requirements under simple random sampling with 100 percent overlapping samples between waves using the formula:
where n is
the sample size per wave, α is the
significance level, β is the power term, z has the
standard normal distribution,
is the correlation between two waves, P1
and P2
are the two proportions being compared, Q1=1-
P1,
Q2=1-P2,,
,
and
.
When α=0.05,
β=0.80,
=0.75,
P1=0.30
and P2=0.36,
the sample size needed per wave is 241.1
Assuming a design effect of 4 (based on the ECLS-K), this subgroup
sample size would need to be further increased by a factor of 4 to
964, since the effective sample size is equal to the sample size
actually obtained divided by the design effect.
The assumptions used to arrive at the sample size by the end of the longitudinal study include the rates at which children move from the base year sampled school to other schools, the rates at which the movers will be subsampled after the base year (children who move between fall and spring kindergarten will not be subsampled), the rates at which the subsampled movers will be located, and the child completion rates. A complete case, also referred to as a respondent, is a child who has a completed assessment or a completed parent interview. We now know the movements of ECLS-K children after each data collection year and how successful we were at locating them for follow-up, and we have modeled the assumed rates for the ECLS-K:2011 on this experience. In ECLS-K, children who moved to another school (but not necessarily residence) were followed at a rate of 50 percent in grade 1, slightly higher in grade 3 so that all language minority children were retained, and slightly lower in grade 5 to accommodate a reduction in the overall sample size. The grade 5 subsampling rates varied according to child characteristics with the highest rate applied to language minority children. For the ECLS-K:2011, the overall subsampling rate will be 50 percent, with differential rates for subgroups of interest (e.g., a higher rate for Asians, Native Hawaiians, and Other Pacific Islanders, and language minority children).2
For the ECLS-K:2011, a sample of 900 responding schools (720 public and 180 private) with an average sample size of 23 children in each school would yield approximately 20,700 sampled children in the base year. Assuming a sample of approximately 20,700 kindergarten children in the base year, we expect that the sample size at the end of the grade 5 followup should be approximately 11,226 completes, which is higher than the minimum sample size of 10,300 needed to meet the study precision requirements. With the sampling rates for subgroups of interest described in the next section, the fifth grade sample size should be large enough to generate estimates that satisfy the precision requirements for each of the subgroups as well.
The four precision requirements are of equal importance for Hispanics, Blacks, and children of other races who are not part of the Asian or Native Hawaiian and Other Pacific Islander groups. However, these subgroups do not have an impact on the determination of the oversampling rates for special groups because their expected sample sizes exceed the required sample size for meeting the precision requirements. At the end of the grade 5 data collection for the ECLS-K, the distribution of completed cases for children who were a race/ethnicity other than Asian or Native Hawaiian/Other Pacific Islander was 11 percent Black, 19.1 percent Hispanic, and 61.4 percent all other races. For the ECLS-K:2011, by the end of the fifth grade collection, we expect to have approximately 1,399 Blacks, 2,150 Hispanics, and 6,900 children of other races who are not part of the Asian or Native Hawaiian and Other Pacific Islander groups. These sample sizes are large enough to satisfy all four precision requirements.3
We noted earlier that language minority children are another subgroup of analytical importance. We expect that 25 percent of the base year sample and 26 percent of the fifth grade sample will be language minority children, based on the data from the ECLS-K. The higher percentage of language minority children in the fifth grade year was due to language minority movers having been retained at a higher rate than other groups of children. For the ECLS-K:2011, with a base year sample of approximately 18,630 expected completes and a fifth grade sample of approximately 11,226 expected completes, we expect to have approximately 4,650 language minority completes in the base year and at least 3,000 language minority completes in the fifth grade year, given our plan to use higher mover retention rates for language minority children than for other groups. Language minority children will be identified by questions in the parent and teacher survey instruments asking about language spoken in the child’s home.
B.1.3 Sample of Primary Sampling Units
The first sampling stage in the ECLS-K:2011 was the selection of geographic areas or PSUs. Clustering the sample into relatively compact geographic units is necessary to control the cost of data collection. PSUs are counties or groups or counties, instead of states or school districts. In most cases, a state is too large of a unit to reduce data collection costs, while school districts do not administratively include private schools and there is no clear mapping of which private schools fall within the geographic boundaries of public school districts. In addition, information on district-level enrollment would be needed to reflect the enrollment of the corresponding private schools under such a design. Counties, on the other hand, have well-defined boundaries and the use of combined counties has the additional benefit of providing a more heterogeneous area, which may reduce the variance of estimates due to clustering.
The PSU frame for the ECLS-K:2011 was created using the 2007 population estimates from the Census Bureau. This PSU frame covers the 50 states and the District of Columbia. The PSUs respect state boundaries and are such that each one has a minimum population size of 380 5-year-old children. 4
A stratified sample of 90 PSUs was selected with probabilities proportional to size. The PSU measure of size is a function of the 5-year-old population in the PSU. Members of the Asian, Native Hawaiian, and Other Pacific Islander racial groups for which oversampling is required contribute more to the measure of size, so that the probability of sampling PSUs with a large proportion of children in these groups was increased. Thus, the measure of size for a PSU is the number of 5-year-olds who are not Asian, Native Hawaiian, or Other Pacific Islander plus 2.5 times the number of 5-year-olds who do belong to one of these racial groups.
PSUs with large measures of size were included with certainty. The remaining noncertainty PSUs were grouped into strata and two PSUs were sampled from each stratum. Census region, level of urbanization, minority status (e.g., percent of the population in the PSU who are Black, Hispanic, Asian/Native Hawaiian/Other Pacific Islander,5 or American Indian), and income level were used as stratification variables.
B.1.4 Sample of Schools with Kindergarten Programs
The second stage of sampling involves selecting samples of public and private schools that have kindergarten programs from within the sampled PSUs. The targeted sample size (i.e., the number of schools from which participation is needed) is 180 private and 720 public schools, for a total of 900 schools. We sampled larger numbers of schools to account for a certain level of expected school nonresponse. In the ECLS-K, private schools participated at a rate of 65 percent and public schools at a rate of 70 percent. Even though efforts will be made to raise the school response rate, we used the ECLS-K rates to compute the number of schools to be sampled initially. We sampled 180/0.65 or about 280 private schools, and 720/0.70 or about 1,030 public schools.
School Frames
Within each sampled PSU, the sampling frame is the list of all public and private schools offering kindergarten in the PSU. For the ECLS-K:2011, we used the sampling frame being developed for the 2010 National Assessment of Educational Progress (NAEP 2010). The primary sources for the NAEP school frame are the most recent Common Core of Data (CCD) and Private School Survey (PSS), which are NCES universe surveys of all public and private schools in the United States. This frame was supplemented with schools that educate children in the grades in which NAEP is administered that are not found in the CCD and PSS for some reason (e.g., the school is a new school and was not reported in the most recent CCD or PSS available). This frame includes all grades from pre-kindergarten to grade 12 (even though NAEP only selects children in grades 4, 8, and 12). School enrollment by grade and race/ethnicity as reported in the CCD and PSS are included in the NAEP school sampling frame. Charter schools are included in the public portion of the school frame, and we sampled them identically to all other schools in the frame.
Within each PSU, public schools with fewer than 23 kindergarten children were clustered together, and private schools with fewer than 12 kindergarten children were clustered together. Clustering was done before sampling to ensure that the target sample size of about 20,700 kindergarten children is met. Schools (or combined schools) were selected with probability proportional to size. As with the PSU sample, a weighted measure of size was computed taking into account the oversampling of Asians, Native Hawaiians, and Other Pacific Islanders as follows:
where 2.5 is the
oversampling rate for Asians, Native Hawaiians, and Other Pacific
Islanders, 6
is the estimated count of Asian/Native Hawaiian/Other Pacific
Islander kindergarten children in the school, and
is the estimated count of all other kindergarten children in school j
in PSU i.
The target number of sampled schools per PSU was calculated separately for public and for private schools and adjusted upward to offset anticipated school nonresponse and ineligibility rates. The number of schools allocated to each PSU was set proportional to the weighted measure of size of the PSU.
An issue in the sampling of schools is the potential for overlap between ECLS-K:2011 and the 2010 NAEP school samples. The largest PSUs and a few of the smaller PSUs in both surveys do overlap, and the largest schools within these PSUs may be selected for both ECLS-K:2011 and NAEP because they are selected with probability proportional to enrollment. This could result in a high response burden for the schools selected for both surveys, resulting in lower school cooperation rates. We considered minimizing the overlap between school samples in PSUs selected for both ECLS-K:2011 and NAEP but did not have to do this because the number of overlapping schools is small (only 28 sampled schools overlap with NAEP). It should be noted, however, that no child within a school sampled for both the ECLS-K:2011 and 2010 NAEP would be sampled for both studies, because the targeted grade levels are different (ECLS:K2011 will sample kindergartners while NAEP will sample fourth graders).
B.1.5 Sampling Kindergarten Children, Parents, and Teachers
The goal of the ECLS-K:2011 sample design is to obtain an approximately self-weighting sample of children, with the exception of Asians, Native Hawaiians, and Other Pacific Islanders who need to be oversampled to meet the sample size goals. Within each sampled school, field staff will obtain a complete list of kindergartners enrolled. Special care will be taken to avoid excluding children from the list because of disability or language barriers by having the field staff ask the school representative questions from a list designed to prompt for any exclusion of children with a disability or children whose native language is not English.
Two independent sampling strata will be formed within each school, one containing Asian/Native Hawaiian/Other Pacific Islander children and the second containing all other children. Asian/Native Hawaiian/Other Pacific Islander children will be sampled from the Asian/Native Hawaiian/Other Pacific Islander stratum with a sampling rate 2.5 times the rate of sampling for children who are not Asian, Native Hawaiian, or Other Pacific Islander. Within each stratum, children will be selected using equal probability systematic sampling. In general, the target number of children sampled in any one school will be 23. If the sampling unit is a cluster of small schools, the school rosters will be kept separate by appending them one after the other in the sampling frame. Sampling will be done systematically and with equal probability from the list so that if a school is small, fewer children will be sampled from the school. If a school has 23 children or more according to the school frame but turns out to have fewer than 23 kindergartners enrolled at the time of child sampling, all children in the school will be sampled. Twins will not be identified prior to sampling but they may enter the sample through this method of probability sampling.
Once the rostered children are sampled, parent contact information for each child will be obtained from the school. The information will be used to locate a parent or guardian and obtain parental consent for the child assessment and for the parent interview.
Teachers who teach the sampled children also will be included in the study and will be asked to fill out a series of teacher questionnaires. All teacher data will be linked to the sampled ECLS-K:2011 children they teach. This procedure for identifying teachers for the ECLS-K:2011 is different from the procedure used in the ECLS-K where, during the fall-kindergarten data collection, a census of kindergarten teachers taken at each school.7
B.2 Procedures for the Collection of Information
The national kindergarten data collection will include direct child assessments, height and weight measurement, parent interviews, school administrator and teacher (both regular classroom and special education) questionnaires, and child care provider questionnaires. Data will be collected twice, once in the fall and once in the spring. Computer assisted interviewing (CAI) will be the mode of data collection for the child assessment and the parent interview. School administrator, teacher, and child care provider data will be collected via self-administered questionnaires.
Pre-Assessment Activities. Data collection staff assigned to recruit schools for the study will work with the school coordinators8 to establish details of the schools’ participation. They will determine:
On-Site Schedule. The recruiter will discuss the schedule for data collection with the school coordinator. The dates for the assessment schedule and for obtaining the sampling lists will be set, making sure to avoid conflicts with any special events in the school’s calendar.
Assessment Locations. The locations within the school where the assessments will take place will also be determined. The goal will be to identify assessment locations that provide as little distraction as possible, that protect the privacy of the children, and that are as non-disruptive of school routine as possible.
Team leaders will make an advance visit to each school. Based on our experience with the ECLS-K and other NCES studies such as NAEP, it should be possible to complete all the pre-assessment activities with one in-person visit and telephone follow-up. Throughout these pre-assessment activities, we will establish a positive and cooperative working relationship with school personnel and the school community.9
During the fall pre-assessment visit, team leaders first will address any questions that the school coordinator or principal may have, then they will review the school coordinator’s role with him/her. A primary task to be conducted during the fall school visit is confirming the logistical arrangements for conducting data collection within the school that were determined during the recruitment process. A checklist of the arrangements that need to be agreed upon and the tasks to be completed will guide the pre-assessment visit. At the time of pre-assessment visit, the team leader will also determine whether the school coordinator is willing and available to conduct certain data collection activities, such as following up on consent forms. Other activities for the pre-assessment visit include: sampling kindergartners; identifying parents and teachers of sampled kindergartners; collecting contact information for parents; meeting with teachers; and distributing the teacher questionnaires.
The logistics of the spring assessment visit will be arranged during the fall assessment visit. Team leaders will call the school coordinator prior to the spring assessment visit to confirm the logistical arrangements for the spring data collection within the school. The team leader will mail the spring hard copy questionnaires to the school coordinator for distribution.
Child Assessment
Typically, the assessment visit will take about 3 days per school. The number of days for the site visit will depend on several factors, such as the number of participating children at the school, any restrictions on the assessment schedule (e.g., assessments only in the morning), and amount of space available for simultaneous assessments. The length of the site visit will be worked out with the site coordinator during the recruitment call and confirmed during the pre-assessment visit. The assessment team will arrive at the school on the appointed first day of assessments; they will immediately contact the school coordinator. The team leader will introduce the assessors to the school coordinator. The procedures to be used during the on-site data collection period will be discussed to ensure there is a common understanding of those procedures. The team leader also will confirm that all sampled children are still enrolled in the school and determine which children are at school that day. New contact information will be obtained for any children who may have left the school after sampling or between the fall and spring collections.
Assessors will be taken by school personnel to their assessment areas. The assessors will arrange their areas to remove potential distractions as much as possible and establish a comfortable environment for conducting the assessment. They will set up the assessment materials and log into the child assessment program on laptops that they will carry with them. All field staff will be provided with backup batteries, cords, etc., to ensure that data collection activities are not disrupted by equipment problems.
Once the assessment areas have been set up and assessors are ready to begin work, the school coordinator will introduce the ECLS-K:2011 team members to the teacher(s) whose children will be assessed. The teacher, in turn, will introduce the assessors to the class. Assessors will then escort the sampled children to the assessment areas, one-by-one, and conduct each 60-minute assessment. As discussed in section A, the assessments will consist of the following: a direct cognitive assessment of reading, mathematics, executive functioning, and measurement of children's weight and height, which will be obtained using instruments brought by the assessors. In the spring, a brief measure of science skills and knowledge will be added to the direct cognitive assessment.
All children, regardless of home language, will first be administered two subscales of the preLAS2000 to assess their level of basic English proficiency. All children will then be administered an assessment of English basic reading skills (EBRS). Children who achieve a minimum score on the preLAS2000 will continue with the remaining reading items and the math assessment in English. Spanish- speaking children who do not achieve a minimum score on the preLAS2000 will be administered a short test of their basic reading skills in Spanish (SBRS) and a math assessment in Spanish, and they will have their height and weight measured. These children will not be administered science or executive function assessments. Non-Spanish-speaking language minority children who do not achieve a minimum score on the preLAS2000 will receive no other direct assessments other than height and weight. Each child will then be returned to the classroom and the next sampled child will be assessed. At the end of each day, the data for completed assessments will be transmitted electronically to a central database.
It is expected that some children will be absent from school when the assessments are scheduled. Certain days throughout the field period will be designated as days on which some field staff will have no assessments scheduled, so that make-up assessments can be conducted on those particular dates. During the first wave of data collection, attempts will be made to conduct a make-up assessment for all children absent on their school’s assessment day who can be assessed at some point during the field period. Missing assessment data for sampled children during the first wave of data collection has implications for the ECLS-K:2011 throughout subsequent rounds, because it prevents the ability to look at growth in achievement across time, which is a key purpose of the study. Therefore, it is important to conduct kindergarten assessments with as many children as possible.
Teacher and School Administrator Questionnaires
The team leader will identify the classroom teachers of the sampled children who will receive the self-administered teacher questionnaire and enter the teachers’ names into the field management system (FMS), creating a link between each sampled child and his or her teacher. This linking system was first developed and used successfully for the ECLS-K.
The team leader will prepare the teacher materials at the time of the pre-assessment visit and will distribute the materials when he/she is introduced by the school coordinator to the teachers who will be asked to participate in the ECLS-K:2011. These materials will consist of a letter describing the ECLS-K:2011, instructions enumerating the teacher activities involved in the study, and a copy of the ECLS-K:2011 brochure (see Appendix H for the school coordinator letter and teacher letter). The introductory meeting will afford an opportunity to explain in person the purpose and importance of the study and to establish a positive relationship with the teacher.
Distributing the Teacher and School Administrator Questionnaires. In the fall and spring kindergarten collections, teachers will be asked to complete self-administered questionnaires about their background, curriculum, and instructional practices. Teachers of the sampled kindergartners also will complete child-level questionnaires about the ECLS-K:2011 children, which indirectly assess the children’s socioemotional and cognitive skills. The teacher evaluations will provide data from a source that has first-hand knowledge of the child and his/her abilities. In the fall, the team leader will identify teachers and distribute the teacher questionnaires and child-level questionnaires. In the spring, teacher questionnaires will be sent to school coordinators who will be asked to distribute the teacher questionnaire and child-level questionnaire. In both rounds, teachers will be asked to complete the teacher questionnaires before the school assessment visit. The average number of children per teacher is expected to be about 6. We plan to offer each teacher an incentive of $7 per child-level questionnaire. The incentives will be included in the package of instruments the teachers receive in the fall and in the spring. Team leaders will collect completed teacher questionnaires, with assistance from the school coordinator, during the assessment visits. Once all questionnaires have been collected, the team leader will mail the completed questionnaires to the home office via FedEx when all have been completed. If there are any questionnaires that are not completed and require follow-up collection, the team leader will mail the completed questionnaires at one time, and mail the remaining questionnaires once they are collected.
In the spring, the teachers or service providers of sampled children who are receiving special education services, i.e., special education teachers, will be asked to complete questionnaires about their background and qualifications. They also will be asked to answer questions about the types of services the ECLS-K:2011 child receives in a separate child-level questionnaire. The special education questionnaires will be distributed and collected in the same manner as the regular classroom teacher questionnaires described above. We plan to offer each special education teacher an incentive of $7 per child-level questionnaire. The incentives will be included in the package of instruments the special education teachers receive in the spring.
Also in the spring, school administrators will be asked to complete a self-administered questionnaire. Information about the school administration, the staff, and the building will be collected through this questionnaire. The school administrator questionnaire will be mailed to the principal or school administrator in advance of the spring assessment visit; on the first day of assessments at the school, the team leader will remind the school coordinator of the need for the school administrator to complete this instrument. The team leader for each school will collect the school questionnaire during the on-site assessment visit. School administrators will receive a $25 incentive for completing the questionnaire, which will be attached to the school administrator questionnaire during the spring data collection. If the school questionnaire has not been completed by the beginning of the last day on-site for assessments, the team leader will remind the school coordinator about it again. If the school questionnaire still is not completed by the time the team has finished its assessment work at the school, the team leader will ask for a specific date from the school coordinator and/or school administrator by which the school will send a completed questionnaire. Follow-up will continue until the questionnaire has been received.
Parent Interview
ECLS-K:2011 field staff who conduct the child assessments also will conduct telephone interviews with parents using a computer assisted telephone interviewing (CATI) instrument. Having the same staff members conduct both components of the study better links the activities that take place in the school with the parent interview, which may in turn promote greater parent participation. The list of parent interviews assigned to the field staff will be transmitted to the field staff via computer, with new cases being transmitted as they become available.
Flexibility in Scheduling Interviews. Procedures for conducting telephone interviews at times that are most convenient for parents and that allow sufficient flexibility will be used regarding the timing of the interviews.
To establish initial contact with a parent of a sampled child, field staff will be trained to place two weekday, three evening, and two weekend calls over a 2-week period. If, after these seven call attempts, no contact has been made with the parent by telephone, the field staff will visit the child’s home to explain the study and attempt to complete an in-person interview. If telephone contact is established, up to seven additional calls will be made to complete the parent interview. If the interview is still not completed after seven calls and the respondent has not actively refused to participate, the field staff will attempt an in-person interview. During the last few weeks of data collection, cases that have not yet been contacted or completed, and that have not been finalized as a refusal after attempts at refusal conversion, will be attempted as in-person interviews to improve response rates.
Non-English Interviewing. The ECLS-K:2011 sample design is expected to include a large proportion of children from linguistically isolated households. (A household is considered linguistically isolated if no one older than 14 speaks English very well.) In order to include these families in the ECLS-K:2011, special measures are required. It is expected that Spanish will be spoken in the majority of these households. Therefore, the parent interview will be fully translated into Spanish, and we will recruit field staff who are bilingual in Spanish and English to conduct these parent interviews. A number of Asian and other languages also will be spoken by parents of sampled children, but in much smaller numbers. The cost of having bilingual staff available who represent all these languages spoken in a large number of PSUs would be prohibitive, as is the cost of sending such staff out for extensive traveling across PSUs. We will recruit and train some bilingual staff in non-Spanish languages for our field effort; however, our primary approach for conducting parent interviews in non-English languages will be to provide for bilingual interviewing by having staff working from their homes through a Telephone Research Center. This is a highly efficient approach to addressing the need for bilingual interviewing in a national sample during a short field period. However, it is cost-prohibitive to develop a full translation of the questionnaire for less common languages. For this reason, we plan on using professional interpreters, as available, for interviews in non-English, non-Spanish languages. This approach was used for telephone interviewing in another NCES study (the Early Childhood Longitudinal Study‑Birth Cohort (ECLS-B)). An interpreting service was used to obtain interpreters for about 20 languages who were connected in a three-way conference call with an English-speaking interviewer and the respondent. Evaluations of the quality and cost of data obtained in this way have established that it can be an efficient way to collect data from respondents who speak less common languages. Translators will be required to sign the NCES Affidavit of Nondisclosure and Confidentiality Pledge described in section A.10.
Wrap-Around Care and Education Provider Questionnaires (WECEP)
The primary care and education providers of children who spend at least 5 hours a week in before- and/or after-school care arrangements will be asked to complete the self-administered questionnaires in spring 2011. If a child has more than one weekly care arrangement, the primary provider will be identified as the one who provides the most care to the child on a weekly basis. Child care providers will be identified in the fall 2010 Parent Interview. The WECEP will only be conducted with the parent’s permission, which will be requested during the parent interview. If permission to contact the provider is granted, parents will be asked to provide contact information for the provider. For center-based care arrangements, administrators of center-based programs will be asked to complete a short questionnaire related to administrative aspects of the center or school-based setting. Many of the same questions are asked of home-based care providers. Caregivers (both center-based and home-based) also will be asked to complete a questionnaire. Administration time is designed to be 18 minutes for the center-based center director questionnaires and 8 minutes for both the home-based and center-based caregiver questionnaires, and 12 minutes for the questionnaire on the ECLS-K:2011 child.
Mailing the WECEP Questionnaires. In the fall, the field staff will identify child care providers during the parent interview. Child care provider contact data will be processed by project staff. In early spring 2011, project staff will mail questionnaire packets for each child care center, which will include information about the study, a consent form signed by the parent giving ECLS-K:2011 staff permission to contact the care provider, center-level and child-level questionnaires for the administrator and the caregiver, a cover letter with directions on how to distribute the questionnaires, and incentive checks. WECEP providers will be offered an honorarium depending on the number of sampled children in their care. The honorarium will be included with the packet of questionnaires that are mailed to the provider. Center-based providers with one sampled child will be offered a $15 honorarium; center-based providers with two to five children will be offered a $20 honorarium; center-based providers with six to ten children will be offered a $30 honorarium and center-based providers with more than ten children students will be offered a $35 honorarium. The honorarium structure is slightly different for home-based providers as they have fewer questionnaires to complete and are expected to have fewer sampled children than center-based providers. Home-based providers with one sampled child will be offered a $10 honorarium; home-based providers with more than one child will be offered a $15 honorarium. An incentive structure such as this is consistent with the incentive structure used for teachers in the ECLS-K:2011, which also provides honoraria based on the number of study children we ask teachers to provide information for.
The incentives will be included in the package of instruments the child care providers (center-based and home-based) receive in the spring. The questionnaire packets will also include a prepaid FedEx mailer for the child care providers to return the completed questionnaires.
B.3 Methods to Secure Cooperation, Maximize Response Rates, and Deal with Nonresponse
This section describes methods for securing cooperation and gaining consent for the ECLS-K:2011 and the methods that will be used to maximize completion rates for child assessments, parent interviews, and teacher, school administrator, and wrap-around care provider questionnaires.
A major challenge in any survey today is obtaining high response rates, and this is even more important in longitudinal surveys where nonresponse can occur at multiple time points and nonresponse at the base year affects all subsequent waves of data collection. As in most longitudinal surveys, attrition is closely associated with those persons who move between waves; however, moving is defined somewhat differently in ECLS-K:2011 than in other surveys because it is triggered by a change in the school the sampled child attends, whether or not the child’s residence changes. In ECLS-K, 25 percent of children changed schools between kindergarten and first grade, and by the fifth grade 56 percent of children were in different schools than they were in for kindergarten. To the extent that parents take advantage of the opportunity to transfer their children from schools in “need of improvement” under ESEA, school mobility may be greater in ECLS-K:2011 than it was more than a decade earlier for the ECLS-K.
The main problem associated with nonresponse is the potential for nonresponse bias in the estimates produced using data collected from those people who do respond. Bias can occur when the people who do respond are systematically different from the people who do not. Two approaches that will be used to reduce the potential for bias are designing the data collection procedures and methods wisely to reduce nonresponse and using statistical methods of sampling and weighting to reduce the effect of nonresponse on the estimates. While the statistical approaches are important in controlling biases and costs, the data collection procedures are at the heart of a successful longitudinal survey.
B.3.1 Gaining Cooperation from a Variety of Sources
Cooperation issues loom large in any major school-based survey today. The demands of required testing, which have increased since the enactment of ESEA, may reduce time for and willingness to participate in voluntary studies like the ECLS-K:2011, so districts and schools may be increasingly less likely to cooperate. Parents are increasingly skeptical about the value of surveys and non-required tests for their children. Teachers are heavily burdened and often reluctant to spend time on non-teaching activities. The additional burden of a longitudinal survey (and the need to communicate clearly to parents and schools the expected burden of participation in a longitudinal survey) makes securing cooperation in the base year even more challenging. The base year must pave the way for concerted follow-up efforts in later rounds by collecting high quality data to help maintain cooperation and track movers.
The data collection plan approaches the school as a community. We aim to establish rapport with the whole community—principals/administrators, teachers, parents, and children. The school community must be approached with respect and sensitivity to achieve high initial response rates and maintain cooperation for future rounds of data collection. The ECLS-K:2011 field staff will be trained that all tasks—securing school and teacher cooperation, completing child assessments and parent interviews, and obtaining parent consent for the wrap-around care and education provider questionnaire—are but different aspects of a single case in their assignment, which is their responsibility to complete. Therefore, field staff will be responsible for conducting the direct assessments as well as the parent interviews and any required followup on the teacher questionnaires and ratings or the school administrator questionnaire. In securing the cooperation of teachers, we will meet with each school's kindergarten teachers during the pre-assessment on-site visit. This will provide the opportunity to explain the purpose and importance of the study in a way that printed information cannot. It will enable the distribution of the teacher questionnaires directly by the field staff, and it also will provide a forum for responding to teachers' questions and concerns and assuring them of the confidentiality of the study. Incentives have proven to be effective tools in achieving high response rates, and we plan to offer monetary incentives to schools, school administrators, teachers, and wrap-around care and education providers.
Secure State, District, and School Cooperation
A proactive and intensive approach will be used to secure the cooperation of the states, districts (dioceses for sampled Catholic schools), and schools. The process by which cooperation is sought is customized based on conditions in the local school systems. For example, for many states and districts an informational package may be sufficient as the first state-level step, but in some states (e.g., Florida) and districts (e.g., Detroit Public Schools, which requires a research proposal submitted a year ahead of the data collection), explicit consent to participate is required from the state by most districts and schools. A national database of district cooperation requirements is being used to develop a highly efficient, tailored approach to gaining cooperation from states and districts. The ultimate goal of the recruiting activity across all levels (state, district, school) is to make sure that the ECLS-K:2011 is on each school’s calendar before the beginning of the 2010‑11 school year. All state test directors and some early childhood coordinators received a courtesy call from NCES staff to inform them of the study, answer any questions, and discuss the importance of their participation. Therefore, the state and district enrollment process began in fall 2009 and school recruitment will begin in January 2010, 9 months before the kindergarten fall data collection.
The study’s procedures will be flexible enough to address the concerns and needs of states, districts/dioceses, as well as schools, to the greatest extent possible without compromising the systematic procedures that are essential to high-quality data collection. Securing the cooperation of states and districts/dioceses in the ECLS-K:2011 is being handled by a small group of contractor senior team leaders and in-house staff with extensive experience in this capacity, with team leaders having responsibility for school enrollment.
Senior field staff with district contact experience is conducting the district enrollment for districts requiring more than notification, as follows.
District and diocesan packets (described further below) including a personalized letter have been shipped via FedEx (with a bright label stating “Important: ECLS-K:2011 Material Inside”) to the senior district/diocesan education official.
A telephone contact was made to confirm receipt of the packet and identify the contact person. If the packet had not been received at the time of the first call, a callback appointment will be made. If a re-mail of the packet was necessary, it also was sent by an express service.
The field staff member completed a checklist concerning the district/diocesan approval procedure and time frame to have approval in the district.
A district coordinator was identified and received all the pertinent material about the ECLS-K:2011.
The district-wide requirements for obtaining parental consent for children’s participation in the study is being ascertained, or confirmed if already known. For the ECLS-K, some schools requested explicit consent from parents even though the district policy was one of implied consent.10 Thus, ascertaining the policy at the district level and having district representatives assure schools that the ECLS-K:2011 will be complying with district policy may help to limit the instances in which a more stringent and costly approach must be used for some schools.
Any additional information that is requested (extra packets, study instruments, or additional study details) is being sent promptly. Any requests for material not already approved for distribution are referred to the NCES study director immediately.
Any refusals are referred to the data collection task leader, along with the case record and reasons for the refusal. In some cases, the contractor project director discusses appropriate strategies to try and convert the refusal to a completed case with the NCES study director and makes recommendations for actions to be taken.
For some districts, the data collection task leader may recommend an on-site visit to prevent or convert a refusal. Such cases will be brought to the attention of the NCES study director and no on-site visits for this purpose will be made without the approval of NCES.
Contact with the districts will continue until participation is secured or a final refusal is received. The timing of contacts are customized to the specific case based on its approval procedures and information obtained during the contacts.
Information Packets For States, Districts, Dioceses, and Schools. An information packet has been developed for the ECLS-K:2011 to convey the study’s legitimacy, importance, and support to various school entities. The information in the packet is presented in a way that is clear and specific, yet concise, and emphasizes that the study team will work closely with schools to accomplish the study with the least burden and disruption possible. The cover letter transmitting the packet is concise, engaging, and convincing. It comes from the Commissioner of NCES and lists the study endorsers on the letter head, so that the recipient will immediately understand the importance and the widespread support of the study.
Information packets have been customized for each intended audience; for example, materials for state agency personnel address the procedures that will be used for contacting the school districts and schools. Materials for the districts address the selection of and procedures for contacting individual schools within their districts. The information packet for all levels of the school system also will include letters of endorsement from appropriate organizations; these letters will be customized for public and private schools (for example, lists of endorsers will be customized by those most pertinent to a public or private school). Endorsements have been obtained from the following organizations (Note: * indicates that organization endorsed the ECLS-K):
|
|
Secure Participation of Schools. Field staff with experience enrolling schools in the ECLS‑K and other NCES studies will be assigned to this activity and trained in early February 2010. Each school recruiter will be assigned a specific set of schools and will receive a laptop containing field management system files for the schools.
We will obtain district/diocesan approval to contact the schools before doing so. Some school districts or diocesan contact persons will wish to contact the schools before we do so. We will adhere to any district requirements that district personnel contact the schools first, that they disseminate the school information packets, or that they receive copies of all communications with schools. We will document and adhere to the procedures to be used within each school district.
For the ECLS-K:2011, it may be necessary for school recruiters to schedule meetings with the district contact person and school representatives to discuss the study and its procedures. Such meetings will be scheduled after the information packets have been received by each principal/school administrator, in order to maximize the usefulness of the meeting and limit the amount of time spent presenting material that is available in the information packets. We do not plan to proactively offer such meetings to all districts, as that could result in a substantial cost burden to the project. However, we will be prepared to hold such meetings when they are necessary to gain the cooperation of schools and districts.
The school information packets will be sent to the principal or headmaster of each sampled school by 2-day FedEx. Several business days after the mailing, school contacts will begin. The school contact procedure will be similar to the district contact procedure described earlier. The first step will be confirming the receipt of the packet or mailing a new package if the first one was not received. The ECLS-K:2011 study staff member who contacts the school will determine the school cooperation decision. If the school agrees to participate, that staff member will then complete a checklist concerning additional information regarding the school needs, the school coordinator for data collection, the parent notification/consent procedures that must be followed (emphasizing district policy), and the dates of visits for the pre-assessment visit and child assessments. The data collection task leader will review cases that initially refuse to participate and decide on appropriate strategies to try and convert the refusal, consulting the project director as needed, and possibly enlisting the assistance of district or diocesan staff to encourage school participation.
Secure Cooperation of the Children’s Parents
Reaching parents and obtaining their consent must be accomplished quickly to support the data collection goals within the time frame of the study. After the sampling of children within schools, the team leader will prepare an information/consent packet for the parent/guardian of each child. The packet will contain a letter to the parents, a parent permission form, and study brochure. The packet will introduce the ECLS-K:2011, explain its purpose and importance, tell parents what is involved in participation, and specify the consent procedure that is being used by their school.
Before compiling the parent packets, we will ascertain the policy of the district regarding parental notification and consent, assuring schools that we are following district procedure. Parent consent procedures will be discussed in the district and school recruitment process, as noted above. We know from experience on the ECLS-K that schools generally require one of two types of consent: implicit or explicit. Both types of consent require that parents be notified that their children have been selected for the study. With implicit consent, the school does not require verbal or written consent for the child to participate. Parents are asked only to notify the appropriate person if they do not want their child participating in the study. If there is no effort to refuse participation, consent is assumed. With explicit consent, children may participate only if their parents provide written or oral consent for their children to do so.
For the ECLS-K:2011, all parents will be sent a consent form in the advance mailing with information about the study. Depending on school requirements, one of two consent forms will accompany the parent notification letter and advance materials. Explicit consent forms will be sent if the school requires that parents sign a consent form and return it to the school before their child can participate in ECLS-K:2011. Parents will be asked to return the form promptly with their agreement to participate. An implicit consent form will be sent if the school does not require a signed consent form for participation; parents will be asked to return the form only if they object to their child participating in the study. Compared to implicit consent, the use of the explicit consent procedure requires considerably more follow-up, which is costly and can be problematic when the data collection period is brief. For this reason, we will encourage the use of implicit consent whenever possible.
Obtaining a high response rate in the ECLS-K:2011 requires that active parent follow-up be conducted in schools using an explicit consent procedure. The team leader will ascertain during the pre-assessment visit whether the school coordinator is available and willing to undertake this follow-up. However, the team leader may need to conduct this activity in order to obtain responses from parents before the first day of assessments. The general study approach assumes that responsibility for parent follow-up will be placed with data collection staff, who are trained for this procedure, rather than with the school coordinator, who may have limited time and may be reluctant to prompt parents to reply or to attempt to persuade a reluctant or initially refusing parent to take part in the ECLS-K:2011. Data collection staff will call each parent to confirm receipt of the study materials, confirm consent for the child to participate, and conduct the parent interview. Data collection staff will be trained to discuss the study with the parent to ensure parents understand that their child is participating in ECLS-K:2011 and that any questions or concerns they have about the study can be addressed so parents can give informed consent. For parents with telephones, follow-up will be conducted initially by telephone; bilingual interviewers will make calls to language minority parents. In-person visits will be made for those parents without telephones.
Parent Contact Information. When possible, the team leader will obtain contact information from the schools for the parents of sampled kindergartners at sampling time, enter the information into the FMS, and transmit it for inclusion in the study respondent database. We understand that some schools may be reluctant to provide this information before parents have agreed to participate in the study. We will work with schools to identify any steps field staff can take to help the school obtain consent from parents and will ensure that study procedures accommodate schools who want to provide contact information only after explicit consent is received from parents.
In cases in which explicit consent is required, we will include a space on the consent form for the parent to enter the name of the parent who is most knowledgeable about the child's education, a current address, and home and work telephone numbers (if any). Collecting this information will be beneficial for future rounds of data collection because it will help in locating those families that move without providing updated information to the school. For example, this additional information may lead us to people who may have new contact information on the parent (e.g., coworkers, neighbors) which can help with locating efforts.
Monitoring Receipt of Parent Consent Forms. Regardless of whether the school chooses to handle the receipt of consent forms or gives study staff the responsibility for this task, the team leader will maintain close contact with the school regarding receipt of parent consent forms. If the school coordinator has chosen to have the ECLS-K:2011 data collection staff conduct follow-up on the parent consent forms, the team leader and assessors will check frequently with the school to ascertain whether forms have been returned to the school. The school coordinator will receive a form listing the children and parents and can use the form to check off receipt of the consent forms from parents. A folder will also be provided to store the forms.
Audio-Documentation of Parent Consent. As an additional step in obtaining parent consent, we will seek explicit verbal consent from all parents for their children to participate in the ECLS-K:2011 in an audiofile at the beginning of the fall kindergarten parent interview (and in the spring for parents who did not participate in the fall). This will serve as additional confirmation of consent for those parents who already provided consent, either implicit or explicit, before they were called. It also provides an additional method by which study staff can obtain explicit consent from those parents for whom it is required but who have not yet returned their consent form. Because we must obtain parent permission, whether explicit or implicit, before the child assessment takes place, this strategy should help to have the necessary consent in place at the time field staff visit schools to conduct assessments, since the fall parent interview occurs close in time to the fall child assessments. Also, having recorded parent consent makes it easier to encourage school and teacher participation in instances where students transfer from one school to the next; this issue is discussed further below.
B.3.2 Methods to Maximize Response Rates
Parent Interviews
There are four main areas that can be focused on in order maximize completion rates for the parent interviews: (1) flexibility in scheduling interviews, (2) non-English interviewing, (3) locating parents of children who transfer schools, and (4) avoiding refusals and converting initial refusals to completed interviews.
Flexibility in Scheduling Interviews. Effective calling patterns are essential for achieving high response rates on all telephone surveys. Previous experience shows that individual respondent schedules (work, classes, recreational activities, vacations, etc.) have a more negative effect on response when call attempts are limited to a short time span. A larger percentage of the numbers that are noncontacts after the first call attempt will be converted to a positive outcome if the call attempts are distributed across a longer time span. Completion rates improve when interviewers call on different days of the week and at varying times of the day and evening.
To establish initial contact with a parent of a sampled child, field staff will be trained to place two weekday, three evening, and two weekend calls over a 2-week period. These calls will be made in a nonsequential set of targeted time periods called “time slices.” The time slices and required number of calls are as follows:
Required Number of Calls
Weekday 10 a.m. to 3 p.m. 1
Weekday 3 p.m. to 6 p.m. 1
Weekday 6 p.m. to 9 p.m. 1
Weekday 6 p.m. to 7:30 p.m. 1
Weekday 7:30 p.m. to 9 p.m. 1
Saturday or Sunday, 10 a.m. to 8 p.m. on separate weekends 2
If, after seven call attempts, no contact has been made with the parent, the field staff will be instructed to review the case with the team leader for additional instruction on how to proceed. The supervisor may instruct the field staff to do one or more of the following: (1) send a letter to the parent; (2) contact the school coordinator to see if the school can help or offer any insight into contacting the parent; (3) contact one of the other contacts listed for the parent, if any; (4) contact the nonresident parent, if applicable; (5) assign the case to another field staff for a fresh approach and a new voice; or (6) conduct an in-person visit.
Once contact is established, up to seven additional calls will be made to complete the parent interview. If the interview is not completed after these seven additional calls and the respondent has not explicitly refused, the field staff may be instructed to attempt an in-person interview. During the last few weeks of data collection, noncontact and uncompleted cases will be visited in-person as appropriate to improve response rates.
Non-English Interviewing. In the base year of the ECLS-K, approximately 7 percent of the parent interviews were conducted in a language other than English. With the growth of immigration of non-English speakers, the percentage of parents who prefer to participate while speaking a language other than English is expected to be larger in the ECLS-K:2011. To achieve high response rates, it is important that study procedures work to include these parents to the greatest extent possible. As described in the data collection procedures section, we will hire and train field staff who are bilingual in Spanish and English to conduct fully translated parent interviews in Spanish and use professional interpreters, as available, for interviews in non-English, non-Spanish languages.
Locating Parents of Transfer Children. Locating parents of transfer children is critical for maintaining high completion rates for parent interviews overall. It is expected that some children will transfer schools between the fall and spring data collections. A tracking system database with household contact and school information will be developed at the beginning of the study and sample tracking activities such as the following will be conducted to locate children who transfer schools: (1) the entire household (parent) address database will be submitted to postmasters for address corrections (ACR); (2) a respondent mailing will be sent to parents asking them to report any changes in sampled children’s schools and/or home addresses; and (3) schools will be asked to provide updated contact information, if they have it. Household and school updates resulting from these activities will be recorded in the ECLS-K:2011 tracking system database. While this OMB package requests approval for the fall and spring kindergarten collections, long-term study plans are to follow the sample children through fifth grade. Development of this tracking database is an important activity for the maintenance of the sample over time; it will be updated with new information over the lifetime of the study.
Refusal Avoidance and Conversion Procedures. Achieving an acceptable parent response rate will require active and effective refusal conversion efforts. This activity must begin as soon as the parent consent forms start being returned to the school. A key factor in converting refusals is the ability of the team leaders and assessors to clearly and confidently convey the purpose and importance of the study and the benefits that will be derived from it. This will be a focus of the field staff training. The training materials for averting refusals include information about becoming thoroughly familiar with the study, including answers to frequently asked questions (FAQs) and responses to respondent objections, drafting responses in the interviewer's own words to FAQs, practicing saying these responses, and diagnosing respondent objections and quickly responding with a response tailored to the objection. The training includes self-analysis by recording responses and listening to them, preparing answers for different situations, using the voice effectively, and role-plays between trainer and interviewer and between interviewers. Averting refusal training will focus specifically on addressing reasons for refusals on the parent interview component of the ECLS-K:2011 study.
During the parent interview data collection period, supervisors and field managers will review initial refusals with the field staff, putting a particular emphasis on reviewing the interviewer record of calls, which will be available to supervisory staff on a weekly basis. If a refusal occurs, the interviewer will be instructed to record key demographic information about the refusing respondent (e.g., sex, approximate age) and the respondent’s reason(s) (if given) for refusing to participate. This information will be evaluated by the team leader to determine the best strategy for converting refusals. Cases identified for refusal conversion will be assigned to a select group of field staff identified as possessing the necessary skills to act as refusal converters. Field managers will hold telephone conferences with the identified field staff to review the refusal conversion procedures and discuss strategies for converting refusals.
Child Assessments
There are two main areas that can be focused on in order to maximize completion rates for the child assessments: (1) conducting assessments with children who are absent on scheduled assessment days and (2) locating transfer children.
Absent Children. It is expected that some children will be absent from school during the time that assessments are scheduled at their school. Days will be set aside throughout the field period in which some field staff have no assessments scheduled, so that make-up assessments can be conducted. A make-up assessment will be conducted for any child who can be assessed during the field period. If an in-school assessment cannot be scheduled, team leaders will contact parents to make arrangements for in-home assessments for absent children, if possible.
Locating Transfer Children. As was the case with the parent interview, locating transfer children, and particularly the new school in which they are enrolled, is critical for maintaining high completion rates for child assessments overall. The activities described above for the parent interview with respect to tracking the sample serve to help maintain high response rates for the child assessments as well, since parents must be located to gain consent for the child assessments.
There is an additional consideration to locating children who transfer schools, which is the need to contact their new schools and teachers and encourage them to participate, thereby allowing the children to be assessed in the school. This issue is discussed further in the next section.
School and Teacher Instruments
There are three main areas that can be focused on in order to maximize completion rates for the school and teacher hard-copy instruments: (1) early distribution of instruments to schools and teachers, (2) effectively communicating the importance of school administrator and teacher participation to school personnel in schools to which ECLS-K:2011 children have transferred between the fall and spring of the kindergarten data collection, and (3) the efforts made by supervisory staff to avoid refusals and to convert initial refusals to cooperating respondents.
Early Distribution of Instruments. Feedback from school administrators and teachers in the ECLS-K indicated that there would be increased participation if they had had more time to complete the hard-copy instruments. During the preassessment visit in the fall of the kindergarten school year, most of the sampled children’s regular classroom and special education teachers, as well as the school administrators, will be identified. Also during the preassessment visit, team leaders will distribute regular classroom teacher questionnaires, along with teacher incentive checks, and ask teachers to complete them before the assessment visit. For the spring data collection, the plan is to send the school and teacher questionnaires, along with their incentive checks, in February of the school year. This schedule will allow 2 months of additional time for these respondents to complete and return the instruments to the school coordinator for field staff to collect on assessment day.
Effective Communication with New Schools to Which Sample Children Transfer. The participation of newly identified school administrators and teachers can be increased by effectively communicating information about the ECLS-K:2011, including the goals of the study, what the study measures, the various study components, why it is important that schools and teachers participate, the study activities to date, plans for the future, and perhaps some results from the ECLS-K. Effective respondent materials, as well as telephone contact by supervisors who are trained to convey this information efficiently and completely, will help maximize the participation of schools to which sample children transfer. In addition, parental consent will be recorded for all children (as mentioned earlier), so that a record of consent will be available for new schools. This should make it easier to get new schools and teachers to participate, because they will have written documentation of the parent’s consent for the student to participate in the study.
Maximizing Response Rates for Hard Copy Instruments. Team leaders will be trained to maximize the response rates for the hard-copy instruments, which will include being flexible in collecting the questionnaires from respondents, following up with the respondents to prompt the completion of the questionnaires, and returning after the assessment visit to pick up late questionnaires. Team leaders will be trained to apply the general refusal aversion techniques to the collection of hard copy questionnaires. These techniques will include analyzing the reasons for refusal, responding appropriately, and using the voice effectively.
Special Considerations in Obtaining Cooperation. District and school personnel have stated that they face increasing demands upon their schools for a variety of non-instructional activities, including requirements for state and district assessment. Sensitivity to these concerns is essential to gaining cooperation for the ECLS-K:2011, and it must be made clear to school system personnel at all levels that the ECLS-K:2011 staff is more than willing to work with them to facilitate their participation with the least burden and disruption possible.
We plan to provide incentives to schools in the form of donations to unrestricted school funds. Most schools have some type of fund that can be used for school needs or activities. Sometimes such funds are managed by the school itself or by the PTA/PTO. Making a donation to such a fund provides the school with the means to use the funds in a way that best meets its needs. An honorarium of $200 per school is recommended.
Wrap-around Early Care and Education Provider Questionnaires
As discussed previously, the wrap-around care and education provider instrument (WECEP) was first fielded as part of the ECLS-B. Experiences on that study show that it is important that care providers be aware that they have parents' permission to provide information about the study children in their care. As part of the WECEP questionnaire packet, a consent form signed by the parent giving ECLS-K:2011 staff permission to contact the care provider will be included. In order to build response rates, child care providers will be sent an honorarium whose amount will depend on the number of sampled children in the center as burden increases with the number of children in the center. Center-based providers with one sampled child will be offered a $15 honorarium; center-based providers with two to five children will be offered a $20 honorarium; center-based providers with six to ten children will be offered a $30 honorarium and center-based providers with more than ten children students will be offered a $35 honorarium. The honorarium structure is slightly different for home-based providers as they have fewer questionnaires to complete and are expected to have fewer sampled children than center-based providers. Home-based providers with one sampled child will be offered a $10 honorarium; home-based providers with more than one child will be offered a $15 honorarium. An incentive structure such as this is consistent with the incentive structure used for teachers in the ECLS-K:2011, which also provides honoraria based on the number of study children we ask teachers to provide information for. Providing child care providers with an honorarium to participate will reduce the potential for needing to extend the field period in order to convert refusals and, thereby, help avoid delays in data delivery. The honorarium will be included in the questionnaire packet.
We intend to send out the questionnaires early in the spring, following the approach of early distribution of the hard-copy instruments (see above). Project staff will begin creating the questionnaire packets in the winter of 2010-11 for mailing of the questionnaire packets in February 2011. This schedule will allow additional time for these respondents to complete and return the instruments via prepaid FedEx mailers, as well as allow for additional time for follow-up within the field period.
Statistical Approaches to Nonresponse
High response rates in the initial kindergarten data collection are very important because any nonresponse biases incurred at that stage are likely to persist in the longitudinal estimates even if attrition is minimized in the subsequent waves. We will analyze nonresponse in the initial school cooperation rates and focus efforts on obtaining rates that are as equal as possible across the major subgroups. For example, non-Catholic private schools have the lowest response rates in surveys of this type and thus additional efforts are required to improve their response rates. This approach contrasts with blindly trying to increase overall response rates by spending resources on those groups that already have the highest response rates, which might raise the overall response rate more but does not overcome bias that would be present in the estimates due to lower response rates for certain subgroups.
We also will subsample movers using a scheme that is similar to that used in ECLS-K to reduce nonresponse bias. The subsampling itself does not reduce nonresponse bias, but the subsampling does enable the same fixed resources to be allocated to a smaller number of children so that higher response rates for subgroups can be achieved. The most expensive children to survey are the movers because collecting data on movers requires additional efforts to get permission from the all the entities from which consent is required (e.g., new districts and school administrators). Also, cost per completed case is increased when there are fewer children per school, and it is often the case that when children change schools, they are the only study child in the school to which they moved.
The form of subsampling that we propose is to include all movers in the rare groups of policy interest in the sample to protect the power for analyzing these children and to subsample other movers. For example, for the ECLS-K Asians, Native Hawaiians, and other Pacific Islanders, as well as language minority children, were retained at much higher rates. Other children who moved were subsampled at a rate of about 50 percent. We will use the same subsampling plan for the ECLS-K:2011. This subsampling rate reduces data collection costs for the movers and ensures that the variances of the estimates are not greatly inflated by using much lower subsampling rates.
At the end of data collection, several methods will be used to study response and the potential for nonresponse bias to exist in the data. First, response rates at both the unit and item levels will be examined as indicators of potential nonresponse bias. Response rates will be calculated separately for each component (e.g., response rates will be calculated for the child assessment separately from response rates for the parent interview) and for both the overall sample and for subgroups of analytic interest. More in-depth bias analyses will be conducted for any component for which at least an 85 percent response rate was not achieved. This bias analysis will include a comparison of sample estimates from the respondents to population values computed from the sampling frame and a comparison of estimates from the ECLS-K:2011 to estimates from other sources (CCD, PSS, NHES). Weights also will be developed for use with the survey data that, when applied, will adjust for nonresponse and reduce the potential for any nonresponse bias to be present in the final weighted data. Comparisons will be made between ECLS-K:2011 estimates of test scores and distributions of children computed using the weights that include adjustments for nonresponse and estimates produced using weights without nonresponse adjustments (i.e., base weights).
B.4 Individuals Responsible for Study Design and Performance
The following individuals are responsible for the study design and the collection and analysis of the data on ECLS-K:2011.
Gail Mulligan, NCES (202) 502-7491
Chris Chapman, NCES (202) 502-7414
Karen Tourangeau, Westat (301) 251-8265
Christine Nord, Westat (301) 294-4463
Thanh Lê, Westat (301) 610-5105
1 The assumptions underlying the calculation of sample size noted here are: a two-tailed test of differences with significance level alpha of 0.05 and power beta of at least 80 percent; estimating proportions of 30 and 36 percent (i.e., a 20 percent relative change); and a correlation between assessment scores from different waves of 0.75. This assumed correlation of assessments comes from experiences in the ECLS-K. Specifically, looking at difference estimates computed between grade 1 and grade 3, and between grade 3 and grade 5 of the ECLS-K, the estimated correlations in assessments between consecutive waves were found to be very high (between 0.72 and 0.98), for an average of 0.75.
2 The study also plans to subsample movers who have an IEP or had an IFSP at higher rates. However, this specific sample protection is part of an option to the contract that has not yet been awarded, so specifications for following such children have not been worked out yet. Since all children who change schools between fall and spring of kindergarten will be followed, this sample protection becomes necessary only between kindergarten and first grade. It will be further explained in a future clearance package for the first grade collection.
3 The race/ethnicity categories are mutually exclusive. Therefore, children who are identified as Asian or Native Hawaiian or from another Pacific Islander group do not appear in any other race/ethnicity group.
4 Since the target population for the study is kindergartners, and age 5 is the modal age for kindergartners, number of 5-year-olds is used as the measure of size for each PSU.
5 For the purpose of PSU selection, these groups were included together in the stratification scheme.
6 Children in these racial groups will be oversampled together as one group.
7The ECLS-K study included the collection of data from a nationally representative sample of kindergarten teachers, allowing for the generation of estimates at the teacher level. The ECLS-K:2011 will include only a nationally representative sample of children.
8 The school coordinator will be identified by the school administrator and act as a liaison to the study.
9During the time we are trying to secure parent cooperation, with the school's permission we will look for ways to inform the school community about the study through the school’s communication systems and tools, for example by posting news about the survey on the school’s website; submitting articles about the survey to the school newsletter; and discussing the survey at a PTA meeting or on Back-to-School Night. To make it easier for field supervisors to initiate this communication, sample letters to parents, articles for newsletters, and PowerPoint presentations that can be customized will be included in an ECLS-K:2011 Best Practices Guide that will be given to field supervisors.
10 These two types of consent are explained in the section about securing cooperation with children’s parents.
|
|
|
Early
Childhood Longitudinal Study |
B- |
|
| File Type | application/msword |
| File Title | Collection of Information Employing |
| File Modified | 2010-05-03 |
| File Created | 2010-05-03 |
© 2024 OMB.report | Privacy Policy