Pilot test results

APPENDIX

Some Formal Pilot Pretest Results
Monetary incentives and response rates
Under OMB Control No.: 648-0511, two pilot pretests were conducted to test survey protocols
with particular emphasis on determining the effect on response rates of three monetary incentive
amounts, $2, $5, and $10. The first pilot survey employed the $2 and $5 treatments, while the
second pilot survey (conducted under a worksheet change) used a $10 incentive for all
respondents.
Total response rates (calculated as the number of completes over the total eligible respondents)
for each treatment are listed in Table A-1.
Table A-1. Response Rates by Incentive Amount
Incentive Amount
Response Rate
(Total eligible sample size)
$2
34.9%
(192)
$5

49.0%
(200)

$10

57.0%
(142)

Statistical tests of differences between the response rates of the three treatments suggest that the
$5 treatment and $10 response rates are significantly larger than the $2 treatment, with
corresponding p-values of 0.00235 and 0.000281, respectively (for a one-sided statistical test
with a null hypothesis of equal response rates). In addition, the $10 treatment response rate is
statistically different from the $5 treatment response rate at the 10% level (p-value of 0.0711).
Correlation between general attitudinal questions and stated preferences
Results from the pretest implementation also provide additional evidence of the correlation of
both questions Q1 and Q3 with the choice question responses (Q11, Q13, and Q14). As shown
in Table A-2, the more positive a person’s expressed reaction to the Endangered Species Act
(Q1), the more likely they are to say the status quo (SQ) alternative is the worst choice in the
choice questions, which is equivalent to saying they have a positive willingness to pay for
protecting Steller sea lions. Specifically, very few individuals with a negative reaction to the
Endangered Species Act (ESA), and thus perhaps a negative view of protection threatened and
endangered species, were willing to pay a positive amount of money for alternatives that involve
doing more and paying more to protect Steller sea lions. Conversely, there was a large number
of individuals with positive reactions to the ESA that said they prefer alternatives that do more
and cost more to doing nothing more.
1

Table A-2. Evidence of Correlation Between Responses to Q1 and Choosing the Status Quo
Alternative in Stated Preference Choice Questions from Pretest Implementation
Q1 - When you think of the Endangered
Species Act, how positive or negative is
your general reaction?

Q11 - SQ is worst
choice

Q13 - SQ is worst
choice

Q14 - SQ is worst
choice

Mostly positive

56

58

56

Somewhat positive

25

29

26

Neutral

10

11

10

Somewhat negative

4

3

3

Mostly negative

1

0

0

Another general attitudinal question that appears correlated to responses to the choice questions
is Q3. As Table A-3 shows, individuals who disagreed with the statement that “Protecting
threatened and endangered species is important to me” generally did not choose the status quo
alternative as the worst choice, which is consistent with the idea that these individuals would
generally prefer to not spend money on alternatives that do more to protect Steller sea lions.
Also, individuals who agreed with the statement tended to dislike the status quo alternative.
Table A-3. Evidence of Correlation Between Responses to Q3 (part 1) and Choosing the Status
Quo Alternative in Stated Preference Choice Questions from Pretest Implementation
Q3 - Protecting threatened and
endangered species is important to me

Q11 - SQ is worst
choice

Q13 - SQ is worst
choice

Q14 - SQ is worst
choice

Strongly disagree

1

1

0

Somewhat disagree

3

1

1

Neither agree or disagree

6

5

5

Somewhat agree

38

42

42

Strongly agree

50

55

50

Responses by Survey Stage
In each survey treatment, the majority of completed surveys were returned before the telephone
interviews were conducted. Specifically, 78% of all completes were received for the $10
treatment, 83% for the $5 treatment, and 82% for the $2 treatment. Following the telephone
contact, another 14%, 6%, and 18% of completes were received. The remainder of completes in
each treatment was received after the second full mailing was sent out.

2