1018-0023 supporting statement B

SUPPORTING STATEMENT B FOR
PAPERWORK REDUCTION ACT SUBMISSION
OMB CONTROL NUMBER 1018-0023
MIGRATORY BIRD SURVEYS
3-165, 3-165A-E, AND 3-2056J-N
1. Describe (including a numerical estimate) the potential respondent universe
and any sampling or other respondent selection method to be used. Data on
the number of entities (e.g., establishments, State and local government units,
households, or persons) in the universe covered by the collection and in the
corresponding sample are to be provided in tabular form for the universe as a
whole and for each of the strata in the proposed sample. Indicate expected
response rates for the collection as a whole. If the collection had been
conducted previously, include the actual response rate achieved.
Migratory Bird Hunter Survey: The potential respondent universe is all licensed
migratory bird hunters in the 49 States that have migratory bird hunting seasons, about
3,800,000 individuals. The universe is stratified by: (1) State, and (2) hunters' hunting
experience and success the previous season. A systematic sample is selected within
each stratum from the names and addresses in the order in which they are received.
Stratum-specific universe and sampling data for forms 3-2056J, 3-2056K, 3-2056L, and
3-2056M, are given in Tables 1-4. Response rates for all four form types are about 60%
nationally.
Parts Collection Survey: About 94,000 duck wings and 23,000 goose tails are
collected and examined by biologists out of a universe of 13,500,000 ducks and
3,800,000 geese harvested. These parts are obtained from about 6,500 successful
waterfowl hunters who return form 3-165 out of a universe of 1,135,000 active waterfowl
hunters. State figures are given in Table 5. The sample of hunters who are sent form
3-165B consists of about 2,000 successful hunters from a sample universe of about
220,000 active woodcock, snipe, rail, gallinule, and band-tailed pigeon hunters. About
11,000 wings are collected and examined out of a universe of about 500,000 birds
harvested.
Experimental Mourning Dove Wing Survey: The sample of hunters who will be sent
from 3-165E consists of about 1,800 successful mourning dove hunters from a sample
universe of about 1,145,000 active dove hunters. We will be soliciting wings from the
first week of the hunting season only. We estimate that we will collect and examine
about 25,000 wings out of a universe of about 8,860,000 birds that will be harvest
during the first week of the mourning dove hunting season.

Appendix B. Effects of certified mail on response rates and survey results
BACKGROUND
This study was conducted as part of our annual surveys of migratory bird hunters during
the 2001-02 and 2002-03 hunting seasons. We receive our survey sample frame
(complete list of names and addresses of all migratory bird hunters in the United States)
each year from the state wildlife agencies, who obtain that information when they issue
hunters their annual hunting licenses. The states also ask migratory bird hunters a series
of screening questions about the species they hunted and their hunting success the
previous year, and provide us with that information as well. We use this prior-year
information as a predictor of current-year hunting activity and success. We assign each
hunter to success/activity strata for ducks, geese, doves, and woodcock based on his/her
responses to the screening questions. We assign hunters to one “duck” stratum and one
“goose” stratum, each consisting of three levels: “None” - did not hunt or bagged 0 ducks
(geese) last year; “Bagged 1-10” ducks (geese) last year; and “Bagged >10” ducks
(geese) last year. Dove survey stratification also is comprised of three levels (“None” did not hunt or bagged 0 doves last year; “Bagged 1-30” doves last year; and “Bagged
>30” doves last year), whereas stratification for woodcock consists of two levels: “Yes” hunted woodcock last year; and “No” - did not hunt woodcock last year.
All surveys are conducted using Dillman’s Total Design Method for mail surveys
(Dillman 1978, Dillman 1991). Our survey packet consists of a diary-format survey
form; a personalized letter that explains the purpose of the survey, instructions for
completing the survey, and why participation is vital to the survey’s success; and a
postage-paid envelope for returning the survey to the Service at the end of the hunting
season. Survey participants are asked to report how many days they hunt for the birds
specified on the survey form, and how many of those birds they shoot. The first request
is sent to sampled hunters either before or during the hunting season, depending on when
we receive the sample frame data from the states. We send reminder postcards at the
close of the season asking sampled hunters to return their completed survey forms. Two
to three weeks after the reminder postcard, we send a follow-up packet via regular mail to
all hunters who have not yet responded. Finally, three to four weeks later, we send an
additional follow-up packet to the remaining non-respondents.
During the developmental stages of these surveys (1995-1997), we sent all final mailings
by certified mail. However, this was expensive, and it soon became apparent that
although certified mail elicits a high response rate, many people have a negative
perception of certified letters and they resent getting surveys in that manner. Therefore,
when the surveys were expanded to include all states in 1998, we only sent the last
mailing (still by certified mail) to a 20% sample of the remaining non-respondents. But
this resulted in low overall response rates. So, in 2001 and 2002 we conducted an
experiment to determine whether the cost of certified mailings, both financial and in
terms of ill will, was offset by higher response rates that would increase the reliability of
survey results. We did this by sending our final mailing by certified mail to one group of
hunters, and by regular mail to another group.

METHODS
We conducted the comparison on three surveys, using pairs of similar states. The state
pairs were selected based on geographic proximity and similarity in hunter numbers and
response rate histories. For the 2001-02 surveys, hunters in the first state of each pair
listed below were sent the final request by certified mail, and hunters in the other state
received the final request by regular mail. We reversed the order for the 2002-03
surveys.
Waterfowl (ducks and geese) survey
West: Oregon and California
Southwest: New Mexico and Oklahoma
Midwest: South Dakota and North Dakota
Midwest: Iowa and Indiana
East: Pennsylvania and New York
Southeast: North Carolina and Georgia
Dove survey
West: Utah and Colorado
Southwest: Oklahoma and New Mexico
Midwest: Kansas and Illinois
Southeast: South Carolina and Alabama
Woodcock survey
Great Lakes: Minnesota and Wisconsin
Mid-Atlantic: Maryland and Virginia
Northeast: Connecticut and New Hampshire
Southeast: Mississippi and Tennessee
Rather than using an analysis where we treated states as paired, we used a crossover
design to compare stratum-specific response rates for the final response wave, certified
mail versus regular mail. The experimental design effectively allowed each state to act as
its own control, and the crossover analysis provided for more statistical power to detect a
difference between treatments than using a paired type of analysis. We also obtained
stratum-specific estimates of hunter participation (proportion of people who hunted) and
hunter success (number of birds they harvested) from the final response wave. This
enabled us to compare the effects of certified mail versus regular mail on the estimates
that these surveys provide, again using the crossover analysis.
RESULTS AND DISCUSSION
Final requests sent by certified mail consistently elicited significantly greater response
rates than requests sent by regular mail. Certified mail response rates for every stratum in
every survey were about double the response rates we obtained using regular mail
(Figures 1-4). However, when we compared estimates of the proportion of active hunters

(Figure 5-8) and the number of birds harvested (Figures 9-12) generated from the last
response wave of certified versus regular mailings, we found no significant differences in
any stratum of any survey. Thus, although certified mail increased response rates
significantly, the increased response rates did not change the surveys’ results. We
believe that this suggests there is little or no non-response bias in our estimates, and that
this is due to our stratification methods.

Ducks - Response Wave 3
0.60

0.44
0.50

0.43
0.39

Response Rate

0.40

Certified
Regular

0.30
0.23

0.21

0.21

0.20

0.10

0.00
1

2

3

Success Strata

Figure 1. Response rates of duck hunters by treatment (certified vs. regular mail) and
success stratum. Error bars denote 95% confidence intervals.
Geese - Response Wave 3
0.50
0.42

0.43

0.40

Response Rate

0.40

0.30
0.23

0.22

0.18

Certified
Regular

0.20

0.10

0.00
1

2

3

Success Strata

Figure 2. Response rates of goose hunters by treatment (certified vs. regular mail) and
success stratum. Error bars denote 95% confidence intervals.

Doves - Response Wave 3
0.60

0.42

0.44

0.42
0.50

Response Rate

0.40

0.30

0.22

Certified
Regular

0.22
0.17

0.20

0.10

0.00
1

2

3

Success Strata

Figure 3. Response rates of dove hunters by treatment (certified vs. regular mail) and
success stratum. Error bars denote 95% confidence intervals.

Woodcock - Response Wave 3
0.60
0.46

0.46

0.50

Response Rate

0.40
0.27
0.25
Certified
Regular

0.30

0.20

0.10

0.00
1

2
Success Strata

Figure 4. Response rates of woodcock hunters by treatment (certified vs. regular mail)
and success stratum. Error bars denote 95% confidence intervals.

Ducks - Response Wave 3
1.00
0.78

0.90

0.75

Proportion of Respondants that Hunted

0.80
0.61

0.57

0.70

0.60
Certified
Regular

0.50
0.29

0.28

0.40

0.30

0.20

0.10

0.00
1

2

3

Success Strata

Figure 5. Proportion of respondents that hunted ducks, by treatment (certified vs. regular
mail) and success stratum. Error bars denote 95% confidence intervals.
Geese - Response Wave 3
0.90
0.75
0.67

0.72

0.66

Proportion of Respondants that Hunted

0.80

0.70
0.48

0.46

0.60

0.50
Certified
Regular
0.40

0.30

0.20

0.10

0.00
1

2

3

Success Strata

Figure 6. Proportion of respondents that hunted geese, by treatment (certified vs. regular
mail) and success stratum. Error bars denote 95% confidence intervals.

Doves - Response Wave 3
0.90
0.76
0.80

Proportion of Respondants that Hunted

0.67
0.70

0.60

0.61

0.60

0.50
0.38

Certified
Regular

0.40

0.30

0.30

0.20

0.10

0.00
1

2

3

Success Strata

Figure 7. Proportion of respondents that hunted dove, by treatment (certified vs. regular
mail) and success stratum. Error bars denote 95% confidence intervals.

Woodcock - Response Wave 3

0.60

0.32

0.32

Proportion of Respondants that Hunted

0.50

0.40

Certified
Regular

0.30

0.07
0.20
0.03

0.10

0.00
1

2
Success Strata

Figure 8. Proportion of respondents that hunted woodcock, by treatment (certified vs.
regular mail) and success stratum. Error bars denote 95% confidence intervals.

Ducks - Response Wave 3
20
15.42

15.83

Mean Seasonal Bag

15

Certified
Regular

10

4.72

4.12

5
1.48

1.70

0
1

2

3

Success Strata

Figure 9. Mean seasonal duck harvest, by treatment (certified vs. regular mail) and
success stratum. Error bars denote 95% confidence intervals.
Geese - Response Wave 3
15

9.83

10
Mean Seasonal Bag

7.05

Certified
Regular

5
2.59

2.04

1.34

1.30

0
1

2

3

Success Strata

Figure 10. Mean seasonal goose harvest, by treatment (certified vs. regular mail) and
success stratum. Error bars denote 95% confidence intervals.

Doves - Response Wave 3
35.00
28.50

25.36

30.00

Mean Seasonal Bag

25.00

20.00
Certified
Regular
15.00
9.95

10.47

10.00
3.53

4.70

5.00

0.00
1

2

3

Success Strata

Figure 11. Mean seasonal dove harvest, by treatment (certified vs. regular mail) and
success stratum. Error bars denote 95% confidence intervals.

Woodcock - Response Wave 3
3

1.52
1.40

Mean Seasonal Bag

2

0.49

Certified
Regular
0.16

1

0
1

2
Success Strata

Figure 12. Mean seasonal woodcock harvest, by treatment (certified vs. regular mail)
and success stratum. Error bars denote 95% confidence intervals.