OMB 83c

PAPERWORK REDUCTION ACT

CHANGE WORKSHEET

Agency/Subagency U.S. Department of Education/ Institute of Education Sciences, National Center for Education Evaluation		OMB Control Number 1850-0882 v.2
Enter only items that change Current Record New Record
Agency form number(s)
Annual reporting and record keeping hour burden
Number of respondents	30,500	52,100
Total annual responses	30,500	52,100
Percent of these responses collected electronically	0%	41%
Total annual hours	20,000	30,800
Difference		10,800
Explanation of difference Program Change		The burden has increased to include the Math/ELA and Special Education teacher surveys. 10,800 increase
Adjustment		0     
Annual reporting and record keeping cost burden (in thousands of dollars)
Total annualized capital/startup costs	0	0
Total annual costs (O&M)	0	0
Total annualized cost requested	0	0
Difference		0
Explanation of difference Program Change		0
Adjustment		0
Other change** The incentive structure which was approved in the recruitment package (NOA #1850-0882, 5/25/11) has been described and further discussed. Changes to questions in the teacher surveys have been included. Please see attachment for further explanation.
Signature of Senior Officer or designee:      	Date:      	For OIRA Use            

**This form cannot be used to extend an expiration date

OMB 83-C

Changes to NLTS 2012 Data Collection Package

This package has been changed to reflect an increase in burden from the inclusion of the Math/ELA and Special Education teacher surveys, changes to questions in the teacher surveys, and the inclusion of incentives which were approved in the original recruitment package submitted for this study.

1. Burden Estimates

The burden estimates for this package now reflect the inclusion of the Math/ELA teacher survey and the Special Education teacher survey. These surveys were not included in the package approval we received on 1-31-2012 because OMB requested more information about teacher incentives.

2. Changes to Question in the Teacher Surveys

Special Education teacher survey:

The School Program Questionnaire (special education teacher) now includes the following questions which were added based on feedback from OMB regarding the receipt of Vocational Rehabilitation services while in school.

• B5a. Did/does this student receive any services funded by Vocational Rehabilitation Services this school year (2011-2012)?

  • Answer categories are: Yes, No, or Don’t Know.

• B5b. Which of the following services did/does this student receive from Vocational Rehabilitation Services?

  • Answer categories are Yes, No, or Don’t Know for each.

{Career counseling; Goal setting and career planning; Job assessment and appraisal; Health advice and promotion that supports working; Case management, referral, and service co-ordination; Interventions to remove environmental, employment and attitudinal barriers; and Job development, and placement services, including assistance with employment and job accommodations.}

Math/ELA teacher survey:

The Math/ELA Teacher Survey has been modified with regard to question B2. The use of the term ‘multimedia’ was confusing and we have reordered the list of instructional materials to place “multimedia” closer to the first four types of materials (none of which refers to the computer), and we have combined prior items i. and h. We believe that by moving the computer questions down the list and combining two of the response categories, we will reduce any confusion. Given that we already have parenthetical definitions next to these items, it was not clear how else we would elaborate.

3. Teacher Incentives

As requested, we have carefully considered the approach to setting teacher incentives that has been used in the current ECLS-K study. We believe, however, that the approach to incentives must fit the study design and data collection strategy, and be sufficient to promote participation given the saliency, amount of effort required, and other factors likely to affect survey completion. In our view the ECLS-K incentive structure is not appropriate for NLTS 2012 given the large differences in the basic study design. We also feel our proposed amount falls within established precedent for similar efforts. But most importantly, changing the incentives – previously approved in our recruitment package and communicated to districts-- would be damaging to this and other studies. Below, we outline our reasoning.

Possible Negative Consequences of Changing Approved Incentive Structure

After our back and forth with OMB on January 31, 2012, we confirmed with our contractor that that they have already described the specific incentives for study participants in discussions with LEA officials and incorporated these into Memoranda of Understanding with participating districts. This action was based on OMB’s approval of our sampling and recruitment package (NOA #1850-0882, 5/25/11) and the extra clarification of our incentive plan for youth, parents, teachers, and principals contained in our response to OMB questions.¹ Approval of incentives at that time was important, because they are a key component of the information we convey to districts during recruitment efforts.

As a result, we believe that, whatever the merits of the ECLS-K incentive structure would be for NLTS 2012 (see below for our assessment), changing course at this point would likely have adverse effects on participation in NLTS 2012 and possibly on other studies conducted on behalf of IES. Both we and the contractor are quite concerned about having to tell districts about a change, particularly because at face value it seems like a large change ($25 vs. $7). While we understand that OMB would prefer to reduce the teacher incentives, we ask for understanding that the prior approval makes it extremely difficult to do so at this time.

Difference between ECLS-K and NLTS 2012 Study Designs

Before we remembered the prior approval of the incentives and checked with the contractor, we did examine whether the ECLS-K incentives could apply to NLTS 2012. We agree that the teacher incentive plan for ECLS-K is well designed to support data collection in that study, which examines the experiences of a nationally representative sample of kindergarten students. And, on the surface, the design of the study is quite similar to NLTS 2012. However, the ECLS-K relied on school level sampling and recruitment, with student sampling resulting in an average of 30 per selected school. Under this design, study activities have high visibility and salience for school staff as well as students. Because members of the ECLS-K study team visit the school at each round of data collection, study resources are spent on developing a longer-term relationship with school leaders and staff. In this context of high saliency, participation by teachers is expected. Moreover, because a single grade is targeted, it is very likely that individual teachers in a school will have multiple study participants in their classrooms and therefore receive a significant amount under the study incentive plan. For example, a school with 3 teachers in the relevant grade for data collection would receive approximately $70 ($7 for each of about 10 students).

The NLTS 2012 sampling strategy is quite different, out of necessity. In order to obtain a nationally representative group of students across the 13 IDEA disability categories, some of which are quite rare, we sampled districts (not schools) and are focusing our resources on obtaining buy-in at that level. We then sample students with disabilities from district lists. Given the wide distribution of students with disabilities, we estimate approximately 25 students per DISTRICT, with the sample spread across schools in some districts and across grade levels. As a result, there are likely to be very few teachers or students participating in each school, with little peer pressure to complete NLTS 2012 data collection. We estimate that the vast majority of teachers participating in each survey will be providing information for only one student. In this context, we believe it is very important to provide an adequate base amount as an incentive.

Consistency of NLTS 2-12 Incentives with Amounts Approved by OMB for Other Studies

Finally, the planned $25 incentive per student for the teacher surveys falls within the Guidelines for Incentives for NCEE Impact Evaluations (memo dated March 22, 2005) and is consistent with teacher incentives that OMB has approved for other recent IES studies. The Guidelines suggest $20 for medium burden and $30 for high burden in an impact evaluation. An incentive of $20 for the Teacher Incentive Fund (TIF) Impact Evaluation was approved in fall 2011. The Supporting Statement Part A for that study notes that $20 is important despite the fact that TIF is salient to the teachers in districts receiving TIF funding. In light of 1) the low salience of NLTS 2012 for teachers who are asked to report on individual students -- due to the very dispersed nature of the NLTS 2012 student sample and the lack of direct contact between members of the contractor study team and staff at individual schools; 2) the small number of responses expected per teacher; and 3) the need to request that teachers complete the questionnaire during a relatively short period of time near the end of the school year, we believe $25 per student is a reasonable amount and consistent with the Guidelines.

1 OMB reviewers requested a full description of all incentives (in questions received on 4/22/2011), which we provided in response to Question 12.6 of our submission dated 5/9/2011.