Proposed Incentive Plan

PISA 2009 -- Field Test 2008 Sampling and Proposed Incentives –

This note is to request approval of a nonsubstantive change to 1850-0755 in order to conduct sampling and recruiting of schools for the field test in 2008 of the 2009 full scale survey and assessment.

The field trial requires a minimum student sample of 1,200 students, which means we should select a sample of between 40-41 schools with 2 replacement schools per original to provide for an adequate participating student sample. This allows for school and student non-response and also for school level and within-school level exclusions.

A formal probability sample of schools is not required for the PISA field trial. It is sufficient that the samples of schools be representative of a broad range of schools from across the United States. To achieve this end we propose a stratified convenience sample of schools having grades 9 and 10 (where most 15-year-olds are found) and having an estimated enrollment of at least 35 students per grade selected in the most populous states: California, Texas, Florida, Michigan, and New Jersey. Selecting schools in these states will meet the criterion of national representation. Within these states, schools will be stratified by grade span, public/private status, NAEP region, location (urban/suburban/town/rural, etc), and minority enrollment (above or below 15 percent). These variables will replicate the design of the main sample. The stratification will be implicit, where the frame is sorted by the school characteristics. The sample will be a stratified systematic sample, with sampling probabilities proportional to measures of size, where the measure of size is the estimated number of 15-year-olds in ninth or tenth grade. We propose in addition the identification within this framework of two similar replacement schools for each sampled school to ensure an adequate student yield.

The KeyQuest sampling software provided by ACER will be used to the extent possible in selecting student samples. Student lists will be formed according to international specifications and eligible students will be sampled with equal probability.

Payments or Gifts to Respondents

Currently, the minimum response rate targets required by OECD are 85 percent of original schools and 80 percent of students, while NCES’ minimum response rate targets are 85 percent at the student level. These high response rates (85 percent is the target for the school level and 85 percent is the target for the student level of PISA) are becoming increasingly difficult to achieve in school-based studies.  Neither PISA 2000, 2003 nor PISA 2006 reached the school response rates targeted for the study.  Thus, the contractor for PISA 2009 proposed to test the following incentive system during the field trial to determine if it would be effective in generating the required response rates.  This system is based on but not the same as that used in PISA 2006.

 

Incentives for school coordinators.  The role of the school coordinator is critical for the success of the study.  The coordinator is expected to:  coordinate logistics with the data collection contractor, supply a list of eligible students for sampling to the data collection contractor, communicate with teachers, students, and parents about the study to encourage participation, assist the test administrator in ensuring the sampled students attend the testing session, and assist the test administrator in arranging for make-up sessions as needed.  The study will provide $150 to school coordinators for those schools that participate and meet acceptable rates of student response in the field trial.  The rate for the main study will depend on whether the ERA is part of the data collection plan.

 

Incentives for students.  The study will provide a $20 incentive for students participating in the assessment during regular school hours.  This is higher than the rate used in PISA 2006 due to inflation and increases in the Federal minimum wage.  Students participating in the ERA during regular school hours will receive an additional $10 incentive.  Students participating in the assessment during non-school hours will be offered a $50 incentive (to compensate for missed activities, work, etc.) with an additional $15 incentive for those participating in the ERA.  Incentives for students will only be provided with the explicit permission of the school principal.

 

Incentives for schools. In order to meet the minimum school response rates mandated by the international governing board, we believe it is necessary to offer school incentives.  We plan to offer schools a school incentive of $300 if the school participates in the field trial assessment.  The rate for the main study will depend on whether the ERA is part of the data collection plan.