Document

Testing a Web-push Design for Estimating Recreational Fishing Effort

ICR 202607-0648-005 · OMB 0648-0652 · Object 170834000.

Document Viewer [pdf]

Status: Original and derived artifacts are available for this document.

Download: pdf

Primary: pdfSource: application/pdf

Loading document viewer…

Document Metadata

Record metadata
File Type	application/pdf
File Title	Testing a Web-push Design for Estimating Recreational Fishing Effort
File Modified	2026-07-07
File Created	2026-07-07
Conversion State	complete

Extracted Text

Testing a web-push design for Estimating
Recreational Fishing Effort
Marine Recreational Information Program Report

FY-2018

Project: Testing a Web-push Design for Estimating Recreational Fishing Effort
Rob Andrews - Author, NOAA Fisheries, Office of Science and Technology

Testing a Web-push Design for Estimating Recreational Fishing Effort

Table of Contents
1. Is it Influential Scientific Information?
2. Has it had sufficient Peer Review?
3. Report Title
4. Background
5. Executive Summary
6. Methods
7. Results
8. Discussion/Conclusions/Recommendations
9. References
10. Appendix

Page 2

Testing a Web-push Design for Estimating Recreational Fishing Effort

Testing a Web-push Design for Estimating
Recreational Fishing Effort
1. Is it Influential Scientific Information?
N

2. Has it had sufficient Peer Review?
N

3. Report Title
Testing a web-push design for Estimating Recreational Fishing Effort

4. Background
NOAA Fisheries implemented the Marine Recreational Information Program (MRIP) Fishing
Effort Survey (FES) in 2015 to estimate recreational shore and private boat fishing effort for
residents of coastal states in the Atlantic and Gulf of Mexico regions. The FES is a crosssectional, self-administered mail survey that asks residents of sampled households to report the
number of recreational, saltwater fishing trips taken by each household member during a twomonth reference wave. The sample frame is derived from the USPS Computerized Delivery
Sequence File (CDS) and includes all residential addresses within each coastal state. Each
year, the survey is administered for six independent waves beginning with wave 1
(January/February) and ending with wave 6 (November/December).

In a 2016 review of MRIP, the National Academies of Sciences Engineering and Medicine (NAS)
recommended that the program evaluate electronic data collection options for the FES (NAS
2017). As a result, MRIP is considering web-based data collection designs. There are several
potential benefits to web-based survey designs, including reduced data collection costs,
improved data accuracy, reduced response burden and/or tailored questions resulting from
automated skip patterns, and more timely access to survey data and estimates. However, there
are also notable challenges. For example, web surveys provide poor household coverage and
generally achieve lower response rates than mail surveys. In addition, switching data collection
modes can have unanticipated impacts on survey measures (Couper, 2011).

Much of the recent research on web-based surveys has focused on mixed-mode designs that
combine web reporting with another data collection mode, usually paper. Web and paper
reporting are uniquely compatible because both are self-administered, and survey requests,
usually consisting of a postcard or invitation letter, can be delivered to sample members through

Page 3

Testing a Web-push Design for Estimating Recreational Fishing Effort

the mail. Mixed-mode designs can either be concurrent, where respondents are offered a
choice of reporting modes (e.g. paper or web), or sequential, where respondents are first
encouraged to respond via one mode before being provided the option of a second mode (De
Leeuw, 2018). A principal goal of both concurrent and sequential mixed-mode designs is to
maximize the number of web responders and subsequently reduce mailing costs.

Data collection protocols for concurrent designs also known as choice designs - usually include
a paper questionnaire, but also provide an invitation for sample members to complete an online
questionnaire. Choice designs may offer promised incentives to encourage web response. The
Residential Energy Consumption Survey (RECS), administered by the U.S. Energy Information
Administration, tested a variety of choice designs during the 2015 survey administration (Biemer
et al. 2018).

In contrast, sequential designs, such as web-push designs, encourage sample members to
complete an online questionnaire before providing a paper version. The National Household
Education Survey (NHES), which is administered by the U.S. Census Bureau on behalf of the
National Center for Education Statistics, recently transitioned from a random-digit-dial telephone
survey design (1991-2011), to a self-administered, mail-based design (2012-2016) and finally to
a web-push design. The NHES web-push design was tested in 2016 and 2017 and
implemented for the full-scale data collection in 2019.

Research has demonstrated that both concurrent and sequential mixed-mode designs generally
achieve lower response rates than mail-only designs (Smyth et al. 2010, Messer and Dillman
2011, Lesser et al. 2016). A notable exception is research on the American Community Survey
(ACS) in which some mixed mode treatments attained higher response rates than mail-only
controls (Matthews et al. 2012). Results from studies comparing response rates between
concurrent and sequential designs are mixed. Most studies observed higher response rates for
concurrent designs (Smyth et al. 2010, Lesser et al. 2016). However, others measured higher
response rates for sequential designs (Matthews et al. 2012), while still others measured similar
response rates for concurrent and sequential designs (Biemer 2016, Bucks et al. 2019). One
consistent result in comparisons between sequential and concurrent designs is that sequential
designs with web as the initial reporting mode achieve significantly more web responses than
concurrent designs (De Leeuw, 2018). Consequently, web-push designs provide the greatest
opportunity for cost savings and improving data quality.

The objective of this study was to evaluate the feasibility of a web-push design for the MRIP
Fishing Effort Survey. Study results were compared to those from the standard, mail-based
FES. Specifically, we evaluated the following items:

Page 4

Testing a Web-push Design for Estimating Recreational Fishing Effort

Overall response rates,

The proportion of sample reporting via the web instrument (web-push design only),

Timeliness of data collection,

Demographic composition overall, as well as for web and paper respondents,

Data quality, including editing and imputation rates,

Key survey measures, including estimates of shore and private boat fishing activity.

5. Executive Summary
NA

6. Methods
Fishing Effort Survey Design

The MRIP Fishing Effort Survey (FES) is a bi-monthly (wave), cross-sectional mail survey
designed to estimate the total number of private boat and shore-based recreational, saltwater
fishing trips taken by residents of coastal states during two-month reference waves. Each year,
the FES is administered for 6 waves in Hawaii, North Carolina and the states along the Gulf of
Mexico and for 5 waves (wave 2 wave 6) in the states along the Atlantic coast. For each wave,
the FES utilizes address-based samples (ABS) covering Hawaii and 16 coastal states along the
Atlantic coast and Gulf of Mexico (Maine through Alabama). The sample frame is derived from
the USPS Computerized Delivery Sequence File (CDS) and includes all full-time (nonseasonal), residential addresses, with the exception of PO boxes that are not flagged as the
only way to get mail. Sampling is stratified both geographically and by angler license status.
Within each state, sampling is stratified into coastal and non-coastal regions defined by

Page 5

Testing a Web-push Design for Estimating Recreational Fishing Effort

geographic proximity to the coast. Generally, counties with borders that are within 25 miles of
the coast are in the coastal stratum and all other counties are in the non-coastal stratum. Rhode
Island, Connecticut, Delaware and Florida are not geographically stratified due to relatively
consistent rates of fishing among counties.

Within geographic strata, addresses are matched to the National Saltwater Angler Registry
(NSAR), which consists of state lists of licensed saltwater anglers. This creates two additional
strata; license matched (households with one or more licensed anglers) and license unmatched
(households that cannot be matched to NSAR). Within each stratum, addresses are selected in
a single stage using simple random sampling.

The questionnaire (Appendix A) asks residents of sampled households to report the total
number of shore and private boat recreational fishing trips taken by each household member (up
to 5) during the reference wave. The data collection period for each wave begins one week
prior to the end of the wave with an initial survey mailing. The timing of the initial mailing is such
that materials are received prior to the end of the reference wave. The initial mailing is delivered
by regular first class mail and includes a cover letter stating the purpose of the survey, a survey
questionnaire, a post-paid return envelope and a $2 prepaid cash incentive. One week following
the initial mailing, a thank you/reminder postcard is sent via regular fist class mail to all sample
units. Three weeks after the initial survey mailing, a follow-up mailing is delivered to all sample
units that have not responded to the survey. The follow-up mailing is delivered via first class
mail and includes a nonresponse conversion letter, a second questionnaire and a post-paid
return envelope. Data are collected for approximately 13 weeks following the initial survey
mailing for each reference wave. However, preliminary estimates are generated from surveys
returned within four weeks of the initial survey mailing.

FES Web-Push Design

The FES web-push design was tested in Massachusetts, New York, North Carolina and Florida
during wave 5 (September/October), 2018 through wave 1 (January/February), 2019. The
sampling design for the web-push treatment was identical to the FES. Independent samples
were selected for each state and reference wave. Table 1 provides the initial sample sizes for
the base FES and the web-push treatment for each state and reference wave.

Table 1. Sample size allocations to FES and web-push treatment.

Page 6

Testing a Web-push Design for Estimating Recreational Fishing Effort

Page 7

Testing a Web-push Design for Estimating Recreational Fishing Effort

State/Wave

FES

Web-Push
Page 8

Testing a Web-push Design for Estimating Recreational Fishing Effort

Massachusetts Total 8,854
Wave 5, 2018
2,658
Wave 6, 2018
6,196
Wave 1, 2019
NA
New York Total
8,483
Wave 5, 2018
2,841
Wave 6, 2018
5,642
Wave 1, 2019
NA
North Carolina Total 11,924
Wave 5, 2018
2,292
Wave 6, 2018
3,331
Wave 1, 2019
6,301
Florida Total
4,160
Wave 5, 2018
1,585
Wave 6, 2018
1,423
Wave 1, 2019
1,152
Overall Total
33,421

8,854
2,658
6,196
NA
8,483
2,841
5,642
NA
11,924
2,292
3,331
6,301
4,160
1,585
1,423
1,152
33,421

We used multiple logistic regression to predict characteristics of households that responded via
the web instrument (versus the paper instrument), as well as households that responded to
initial (preliminary) survey request. We evaluated differences between FES and web-push
samples for response rates, demographic characteristics and survey measures using paired ttests, where FES and web-push estimates for each state and wave formed a pair. Analysis
The web questionnaire was designed to be as consistent as possible to the FES paper
questionnaire. The instrument was programmed in Voxco and optimized for mobile devices.
Screen shots of the instrument are included in Appendix B. Data collection for each wave
began 5-6 days prior to the end of the wave with an initial mailing that included a $2.00 prepaid
cash incentive and cover letter. The cover letter described the survey and included the survey
URL and a unique access code. A bi-fold reminder postcard that included the URL and access
code was sent to all nonrespondents approximately 10 days after the initial mailing. A third
mailing was sent to all nonrespondents approximately two weeks after the initial mailing. This
mailing included a refusal conversion letter with instructions for completing the survey online, as
well as the FES paper instrument and a BRE. A final bi-fold postcard urging nonrespondents to
complete the mail questionnaire but also providing the URL and access code was mailed 10
days after the paper questionnaire
.

7. Results
Response Rates

Page 9

Testing a Web-push Design for Estimating Recreational Fishing Effort

Figure 1 shows weighted response rates (AAPOR RR2) for the FES and web-push design by
state and reference wave. Overall, FES response rates (31.9) were 8.75 points higher than
web-push response rates (23.3), which is a significant difference (p<0.0001). Among states and
waves, differences between FES and web-push response rates ranged from 7.2 points to 11
points.

Figure 1. Weighted response rates (AAPOR2) by state and reference wave. Addresses
returned by the post office as undeliverable have been removed from the denominator of the
response rate calculation.

Figure 2 shows cumulative response rates for the FES and web-push. As expected, response
for the web-push outpaced the FES through the first several days of data collection. However,
FES response increased rapidly beginning 7-10 days after the initial survey mailing, and
response rates quickly outpaced those for the web-push design. Overall, the median response
time for the FES (14 days) was actually a day less than the web-push (15 days).

Figure 2. Cumulate response rate curves for FES and web-push samples.

Page 10

Testing a Web-push Design for Estimating Recreational Fishing Effort

Figure 3 shows the final distribution of web-push responses by reporting mode. Overall, more
than two thirds of web-push respondents used the web-based questionnaire. The percentage of
respondents reporting online was fairly consistent among states, ranging from 66% (NC, NY and
FL) to 70% (MA).

Figure 3. Distribution of web-push responses by reporting mode by state and reference wave.

Page 11

Testing a Web-push Design for Estimating Recreational Fishing Effort

Data Quality

In terms of data quality, we compared the frequency of data edits resulting from illogical values
as well as item nonresponse for key survey measures. Table 2 describes illogical scenarios
requiring an edit, and Table 3 provides editing rates for the FES and web-push designs, as well
as for each of the data collection modes within the web-push design. Editing rates reflect the
percentage of responding households that required a specific type of edit[1]. Overall editing
rates for the FES and web-push designs were 4.95 and 3.10 percent, respectively. Within the
web-push design, editing rates for the paper and pencil survey and web survey were 6.41
percent and 1.48 percent, respectively.

Table 2. Types of illogical values requiring a data edit.

Page 12

Testing a Web-push Design for Estimating Recreational Fishing Effort

Table 3. Data editing rates. Edits are not mutually exclusive, so the sum of records across edit
types will not match the total number of records that required an edit.

Table 4 provides item nonresponse rates for key survey items. As with editing rates, item
nonresponse rates reflect the percentage of responding households that are missing at least
one key data element. Key data elements include the reported number of household members
and the number of shore and private boat fishing days during the two-month reference wave for
each household member. In addition, a returned survey may be missing all data elements,
including demographic and fishing information, for an individual household member (i.e. an
Entire Person is missing). In these cases, the count of complete person sections is less than
the reported number of household members.

Table 4. Item nonresponse rates. Survey items are not mutually exclusive, so the sum of
records across items will not match the total number of records that were missing an item.

Page 13

Testing a Web-push Design for Estimating Recreational Fishing Effort

Overall item nonresponse rates are similar between treatments; item nonresponse rates for the
FES and web-push were 12.48 percent and 13.63 percent, respectively. However, the
treatments differed with respect to the specific missing items. In the FES, item nonresponse
was highest for the number of fishing days (shore and/or boat fishing). In the web-push,
respondents were most likely to exclude all information for one or more household members.
Differences between FES and web-push treatments are the result of differences in item
nonresponse between the paper and online questionnaires.

Survey Measures

Within the web-push sample, we used multiple logistic regression to explore characteristics of
households that responded to the survey via the web instrument (Table 5). Households with
seniors and black-alone households had significantly lower odds of responding to the web
survey than the paper survey. No other demographic characteristics were significant predictors
of web response.

Table 5. Multiple logistic regression predicting web response (1) versus paper response (0)
among all responders.

Page 14

Testing a Web-push Design for Estimating Recreational Fishing Effort

Table 6 compares weighted demographic distributions between the full FES and web-push
samples. Distributions for the samples were similar for all demographic characteristics with the
exception of the presence of seniors (p=0.0144), which was higher for the FES, and white alone
(p=0.0054), which was higher for the web-push sample. Table 6 also compares estimated
demographic characteristics to control values published by the U.S. Census Bureau. The
direction and magnitude of differences between estimated and control values are similar for both
survey designs. Both designs overestimated home ownership and the percent of households
with seniors and underestimated households with children, mean household size and the
percent of the population that identifies as black alone and Hispanic.

Table 6. Comparison of demographic composition between FES and web-push samples.
Estimates are the average of weighted estimates, across states and waves. The weights used
to compare demographic characteristics were adjusted for nonresponse, but do not include
calibration adjustments to population control totals. Difference and significance values are for
comparisons between survey designs.

Page 15

Testing a Web-push Design for Estimating Recreational Fishing Effort

In terms of key survey measures, we examined household fishing prevalence and mean fishing
days per household (Table 7). We evaluated shore and private boat fishing separately, and
unless otherwise noted, all estimates are for 2-month reference waves. FES estimates were
larger than web-push estimates for all measures. However, differences between estimates were
significant only for mean boat fishing days per household (difference of 0.83 boat fishing days
per household) and shore fishing prevalence (difference of 1.34 percent). Comparisons
between FES and web-push designs for key survey measures by state and reference wave are
provided in Appendix 3.

Table 7. Comparisons between FES and web-push designs for key survey measures.
Estimates are the average of weighted estimates, across states and waves*.

As noted previously, FES and web-push samples were not significantly different with respect to
most demographic characteristics (Table 6), and while samples were significantly different for
the percentage of households with seniors and white alone, the differences were relatively small
and not likely to explain large differences in survey measures. While household composition
was similar, we did observe differences between treatments in reporting fishing participation by
children among households that included at least one child household member and reported at
least one fishing day, a significantly higher proportion of households in the FES treatment
reported that a child participated in fishing (Table 8). While significant, differences between FES
and web-push treatments for child participation can explain only a small portion of the overall
difference between treatments as child anglers accounted for only 5-15% of total fishing
activity.*Comparisons between FES and web-push designs for key survey measures by state
and reference wave are provided in Appendix 3.

Table 8. Comparison between FES and web-push designs for the percentage of fishing
households with children that reported participation during the reference wave by at least one
child household member.

Page 16

Testing a Web-push Design for Estimating Recreational Fishing Effort

[1] Values for Wireless Only, Own Home, 3+ Household Members, Child in Household and
Senior in Household are the percent of households with the attribute. White Alone, Black Alone
and Hispanic are the percent of the population with the attribute.

[2] Control value estimated from the National Health Interview Survey, administered by the
National Center for Health Statistics

[1] A single household may require multiple types of edits as well as multiple instances of a
single type of edit.

8. Discussion/Conclusions/Recommendations
The mail-only FES design achieved significantly higher response rates than the web-push
design. This result is not surprising and is consistent with results from previous studies
comparing mixed-mode and mail survey designs (Smyth et al. 2010, Messer and Dillman 2011,
Lesser et al. 2016). In a meta-analysis of research studies, Groves (2006) demonstrated that
nonresponse rate alone is not a strong predictor of the magnitude of nonresponse bias. In the
present study, differences in response rates between treatments are quite large (7.2-11
percentage points), suggesting that the risk for nonresponse bias is substantially higher in
the web-push design than the mail-only design.

Page 17

Testing a Web-push Design for Estimating Recreational Fishing Effort

While response rates were lower, the web-push design effectively pushed respondents to the
online questionnaire; approximately 70% of respondents utilized the online instrument. This
finding demonstrates the potential cost savings of the web-push design - at a production scale,
we estimate that non-labor costs (e.g. printing, postage, materials) of the web-push design
would be approximately 15% lower than the FES. However, the cost benefit of the web-push
design is eliminated when we consider survey participation the estimated cost of the web-push
design would be approximately 15% higher than the FES on a per-complete basis. Patrick et al.
(2018) and Messer and Dillman (2011) reported similar results when comparing costs for webpush and mail-push designs. At present, transitioning to a web-push design would result
either in a higher cost to achieve a desired effective sample size (i.e. a fixed level of
precision) or, for a fixed cost, a reduced number of completed surveys.

A second potential benefit of online surveys is improved data quality. Online instruments
provide the capacity to include real-time editing, including logic checks, range checks, etc.
Additionally, web instruments allow for complex skip patterns tailored to each respondent. In the
present study, our goal was to maintain as much consistency as possible between paper and
web instruments, so we did not include complex data editing functions into the web instrument.
Despite efforts to maintain consistency, we did observe differences in data quality between web
and paper instruments. Across data collection modes, editing rates for the web-push
sample were modestly lower than FES editing rates (4.95% vs. 3.10%). However, editing
rates for web responses were only 1.48%, even without built-in data editing processes.

Item nonresponse rates were also similar for FES and web-push samples (12.48% vs. 13.35%).
However, the sources of missing items were different between the two treatments. For the FES,
item nonresponse was highest for the number of fishing days during the wave (8.5%). In these
cases, respondents provided demographic information, but failed to answer the fishing
questions for one or more household members. In this scenario, we assume that the fishing
questions are not applicable to the household member (i.e. they did not fish), and we impute
zeros. For the web-push treatment, the nonresponse rate for fishing days was considerably
lower at 4.6%. However, web-push respondents were more likely to exclude an entire
household member than FES respondents (10.85% vs. 5.86%). It is not clear if household
members are excluded because the substantive fishing questions are inapplicable or if
respondents are purposefully terminating surveys due to fatigue or even forgetting household
members. In this case, we can assume that fishing questions are inapplicable, but we are
unable to make judgements about household member demographic characteristics, which may
affect weighting adjustments. The purpose of weighting adjustments is to reduce bias resulting
from differential response among households with different characteristics i.e. the goal is for the
responding sample to accurately represent the population. Weighting adjustments in the
web-push design may be less effective as a result of higher item nonresponse for
weighting variables.

Page 18

Testing a Web-push Design for Estimating Recreational Fishing Effort

A third perceived benefit of online surveys and web-push designs is the accelerated timeframe
in which data are available for analysis and estimation. As expected, we began receiving a
substantial number of web responses within 3 days of the initial survey mailing. In contrast, we
did not receive the first FES response until day 7. Consequently, web-push response outpaced
the FES through the first 10 days of data collection. However, FES response rates eclipsed
web-push response rates by day 12, and the median response time for the FES was a day
shorter than that of the web-push design (14 days vs. 15 days). Currently, we produce
preliminary effort estimates from surveys returned within approximately four weeks of the initial
mailing[1]. The FES design actually results in a larger number of responses than the webpush design within the preliminary estimation schedule.

We observed significant differences between web-push and FES samples for key survey
measures. FES estimates were larger than web-push estimates for all key survey measures,
and differences between FES and web-push estimates were significant for mean boat fishing
days per household and shore fishing prevalence. Differences between designs in reporting
fishing activity for children may contribute to the observed differences in survey measures, but
the magnitude of this effect is likely to be small. As noted above, differential measurement
errors related to reporting mode may also contribute to the differences. De Leeuw (2018)
suggests that the risk for differential measurement errors across self-administered modes is low
if questions are similarly designed and administered. Similarly, Dillman et al. (2014) identify
three factors that can result in differential measurement effects across survey modes, 1)
presence/absence of an interviewer, 2) aural versus visual communication, and 3) differences in
survey questions. The present study utilized self-administered questionnaires, so differences
in measurement between web and paper instruments could only have resulted from
differences in question construction and/or administration. We were careful to utilize
similar instructions and question wording for the two instruments. However, the instruments
deviated in how questions were presented to respondents. Specifically, the web instrument
was optimized for mobile devices, so respondents could only see one question at a time,
while the paper questionnaire presents all of the questions as soon as the respondent
opens the survey package.

We also speculate that web and paper surveys differ with respect to the environments in which
they are completed, and that these differences may contribute to differential measurement
errors. For example, a paper questionnaire is a physical thing that may sit on a table or
countertop for several days, attracting the attention of multiple household members, before it is
completed and returned. In contrast, a web questionnaire may be a more personal experience,
completed by a single individual, with little notice or input from other household members. In this
respect, web surveys may be more similar to a telephone interview than a mail survey. In a
study comparing telephone and mail survey estimates of recreational fishing activity, Andrews et
al. (2014) propose a type of screening error, which they refer to as a gatekeeper effect.
According to the gatekeeper hypothesis, the individual who answers the telephone for an
interview may not be the most knowledgeable about the survey topic and may subsequently
screen the household out of the eligible sample, resulting in under-reports of household fishing

Page 19

Testing a Web-push Design for Estimating Recreational Fishing Effort

activity. We suggest that web surveys, which are likely to be completed by the individual
who opens the mail, may be susceptible to similar errors, resulting in under-reporting of
fishing activity. Future research should focus on differential measurement errors between web
and paper instruments.

The web-push design proved to be a reasonably effective method for collecting recreational
fishing data. However, unless response rates improve, the design is not a cost-effective
alternative to the FES mail survey. In addition, differences between the web-push and FES
designs for estimates of key survey measures would disrupt the recently calibrated time series
of recreational fishing catch and effort estimates. We recommend additional testing of the
web-push design, focusing on improving response rates and resolving differences
between FES and web-push estimates, prior to consideration of the methodology as a
valid alternative to the existing FES design.

[1] The FES estimation schedule is determined by the availability of data from the
complementary Access Point Angler Intercept Survey (APAIS). Estimates from the APAIS,
which are combined with FES estimates to estimate total catch, are generally available one
month after the conclusion of each survey wave.

9. References
Andrews, R., J. M. Brick, and N. A. Mathiowetz. 2014. Development and Testing of Recreational
Fishing Effort Surveys: Resting a Mail Survey Design. National Oceanic and Atmospheric
Administration Fisheries, Office of Science and Technology, Silver Spring, Maryland.

Biemer, P.P., J. Murphy, S. Zimmer, C. Berry, G. Deng, and K. Lewis. 2018. Using Bonus
Montetaryy Incentives to Encourage Web Response in Mixed-Mode Household Surveys.
Journal of Survey Statistics and Methodology 6: 240-261.

Bucks, B., M.P. Couper, and S.L. Fulford. 2019. A Mixed-Mode and Incentive Experiment using
Administrative Data. Journal of Statistics and Survey Methodology 0: 1-18.

Page 20

Testing a Web-push Design for Estimating Recreational Fishing Effort

Couper, M.P. 2011. The Future of Modes of Data Collection. Public Opinion Quarterly
75(5):889-908.

De Leeuw, E.D. 2018. Mixed-Mode: Past, Present, and Future. Survey Research Methods
12(2): 75-89.

Dillman, D., J. Smyth, and L. M. Christian. 2014. Internet, Phone, Mail, and Mixed-Mode
Surveys: The Tailored Design Method (4th ed.), Hoboken, NJ: John Wiley & Sons.

Groves, R. M. Nonresponse Rates and Nonresponse Bias in Household Surveys. Public
Opinion Quarterly 70: 646-675.

Lesser, V.M., L.D. Newton, D.K. Yang, and J.C. Sifneos. 2016. Mixed-mode Surveys
Compared with Single Mod Surveys: Trends in Responses and Methods to Improve
Completion. Journal of Rural Social Sciences 31(3): 7-34.

Matthews, B., M. Davis, J. Tancreto, M. F. Zelenak, and M. Ruiter. 2012. 2011 American
Community Survey Internet Tests: Results from Second Test in November 2011. American
Community Survey Research and Evaluation Program, #ACS12-RER-21, May 14, 2012.

Messer, B. & Dillman, D. (2011). Surveying the General Public Over the Internet Using
Addressed Based Sampling and Mail Contact Procedures. Public Opinion Quarterly 75(3),
429457.

National Academies of Sciences, Engineering, and Medicine. 2017. Review of the Marine
Recreational Information Program. National Academies Press, Washington, D.C.

Patrick, M.E., M.P. Couper, V.B. Laetz, J.E. Schulenberg, P.M. OMalley, L.D. Johnston, and
R.A. Miech. 2018. A Sequential Mixed-Mode Experiment in the U.S. National Monitoring the
Future Study. Journal of Statistics and Survey Methodology 6:72-97.

Page 21

Testing a Web-push Design for Estimating Recreational Fishing Effort

Smyth, J.D., D.A. Dillman, L.M. Christian, and A.C. ONeill. 2010. Using the Internet to Survey
Small Towns and Communities: Limitations and Possibilities in the Early 21st Century. American
Behavioral Scientist 53(9):142348.

Page 22

Testing a Web-push Design for Estimating Recreational Fishing Effort

10. Appendix
"Appendix C FES Web Push Comparisons by State and Wave", page 1

Figure C1. Comparisons between the FES (blue) and web-push (green) designs for private boat fishing
prevalence.
0.12

Boat Fishing Prevalence

0.1
0.08
0.06
0.04
0.02
0
1

FES

Web Push

Figure C2. . Comparisons between the FES (blue) and web-push (green) designs for mean number of
boat fishing days per household. Estimates are for those households that reported at least one day of
boat fishing.
7

Mean Boat Fishing Days

6
5
4
3
2
1
0
1

5
FL

MA
FES

6
NY

Web Push

Page 23

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix C FES Web Push Comparisons by State and Wave", page 2

Figure C3. Comparisons between the FES (blue) and web-push (green) designs for shore fishing
prevalence.
0.12

Shore Fshing Prevalence

0.1
0.08
0.06
0.04
0.02
0
1

FES

Web Push

Figure C4. . Comparisons between the FES (blue) and web-push (green) designs for mean number of
shore fishing days per household. Estimates are for those households that reported at least one day of
shore fishing.
8

Mean Shore Fishing Days

7
6
5
4
3
2
1
0
1

5
FL

MA
FES

6
NY

Web Push

Page 24

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix A FES Mail Survey Questionnaire", page 1
0835186348

14
14. What is this person's race? Mark one or more boxes.
White
Black, African-American
Asian
American Indian or Alaska Native
Native Hawaiian or other Pacific Islander

Please think only about recreational saltwater
fishing in North Carolina.

15 How many days did this person go
recreational saltwater fishing from the SHORE
in North Carolina?

The shore includes docks, bridges, causeways,
beaches, banks, or any other shore-based place
or area. Do not include freshwater fishing.

Did not recreational saltwater fish from shore
in last 12 months Go to question 16

Number of days saltwater shore
fishing in January and February of
2018

Number of days saltwater shore
fishing in last 12 months, including
January and February

16 How many days did this person go recreational
saltwater fishing from a private or rental BOAT
that returned to shore in North Carolina?

Do not include freshwater trips or trips where a
paid captain or crew helped locate and catch fish.

Did not recreational saltwater fish from
private boat in last 12 months

Number of days saltwater boat
fishing in January and February of
2018

Number of days saltwater boat fishing
in last 12 months, including
January and February

Number of days saltwater boat fishing
in last 12 months, including
January and February
Please return your survey in the enclosed
postage-paid envelope.

If you have more people in your household,
continue to Household Member 5. If you have
answered for all people in your household,
please return your survey.

Weather and Outdoor
Activity Survey

14
14. What is this person's race? Mark one or more boxes.
White
Black, African-American
Asian
American Indian or Alaska Native
Native Hawaiian or other Pacific Islander

13
13. Is this person of Hispanic, Latino, or Spanish origin?
Yes, of Hispanic origin
No, not of Hispanic origin

TME

NT OF C

Public reporting burden for this collection of information is estimated to average 10 minutes per response, including the
time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and
completing and reviewing the collection of information. Send comments regarding this burden estimate or any other
suggestions for reducing this burden to Rob Andrews, NOAA Fisheries Service, 1315 East-West Hwy., Silver Spring, MD
20910.
No personally identifiable information will be collected through this survey. Responses will only be associated with a
unique, randomly assigned identification code. Any public release of survey data will be without identification as to its
source or in aggregate statistical form. All survey data will be stored on secured, password protected servers, and all
transfer of survey data will utilize secure file transfer protocols.

RTI International
5265 Capital Boulevard, Raleigh NC 27690-1652
4

D ATMOSPHER
AN
IC

13
13. Is this person of Hispanic, Latino, or Spanish origin?
Yes, of Hispanic origin
No, not of Hispanic origin

C
NI

North Carolina

TRATION
NIS
MI
AD

Age in years

12. How old is this person?
12
If less than 1 year, mark 0 years

99999923

12. How old is this person?
12
If less than 1 year, mark 0 years
Age in years

301

OMB#: 0648-0652
Exp. Date: 1/31/2020

HOUSEHOLD MEMBER 5
11
11. What is this person's gender?
Male
Female

NATIONAL OC
EA

HOUSEHOLD MEMBER 4
11
11. What is this person's gender?
Male
Female

ALGPBLDMDKGLBNFOHLGK
AOHMPCMPKODEFNJLKFGK
ALOOLNGOEPICLBBBALOK
AIFBIPEPPMLJEODKBOIK
ACOGECGGGKGCMOOEMCEK

0000014

Page 25

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix A FES Mail Survey Questionnaire", page 2
6859186340
This survey should be filled out by an adult member of the household. Complete and
return this form even if no one in your household participates in any of these activities.
START HERE
Please carefully follow the steps below when completing this survey.
• Use only a blue or black ink pen that does not blot the paper
• Make solid marks inside the response boxes

Example
RIGHT
WAY

• Do not make other marks on the survey
1
1. How do members of this household obtain
information about the weather, including
current weather conditions, forecasts, and
warnings? Mark all that apply.
Television
Radio
Newspaper
Internet
Other

2 During the past 12 months, has anyone in
this household had to evacuate or seek
shelter due to a severe weather event, such
as a tornado, hurricane, or thunderstorm?
Yes
No

3 In your area, how often do the advanced
warnings you get for severe weather events
allow you enough time to prepare properly?
All the Time
Some of the time
Rarely
Never

WRONG
WAY

7
7. Which of the following best describes how
your household receives telephone calls?
All are received on cell phones
Most are received on cell phones
Some are received on cell phones and
some on landline phones
Most are received on landline phones
All are received on landline phones
No calls are received on cell phones or
landline phones
8 Which of the following best describes this
house, apartment, or mobile home?
Owned with a mortgage or loan
Owned (without a mortgage)
Rented
Occupied without payment or rent
9 How long have you lived at this address?
1 year or less
Less than 5 years, more than 1 year
5 years or more
10 How many people, including all adults and
children, live in this household?
Number of people

4 During the past 12 months, has anyone in
this household visited a public beach,
national seashore, coastal state park, or
other coastal nature reserve or protected
area?
Yes
No

Please answer the next section for each
member of your household, starting with
yourself. Please answer for all people in
your home, including people who fish
and people who do not fish.
If you have more than 5 people living at
this address, answer for the oldest
members of the household.

5 During the past 12 months, has anyone in
this household been freshwater fishing in
North Carolina?

Please use the calendars to help answer
questions 15 and 16.

Yes
No

6 During the past 12 months, has anyone in
this household been saltwater fishing in
North Carolina?

HOUSEHOLD MEMBER 1 (YOU)

HOUSEHOLD MEMBER 3

11
11. What is this person's gender?
Male
Female

12
12. How old are you?
If less than 1 year, mark 0 years

12
12. How old is this person?
If less than 1 year, mark 0 years

12. How old is this person?
12
If less than 1 year, mark 0 years

Age in years

13
13. Are you of Hispanic, Latino, or Spanish origin?
Yes, of Hispanic origin
No, not of Hispanic origin

13
13. Is this person of Hispanic, Latino, or Spanish origin?
Yes, of Hispanic origin
No, not of Hispanic origin

14
14. What is your race? Mark one or more boxes.
White
Black, African-American
Asian
American Indian or Alaska Native
Native Hawaiian or other Pacific Islander

14
14. What is this person's race? Mark one or more boxes.
White
Black, African-American
Asian
American Indian or Alaska Native
Native Hawaiian or other Pacific Islander

Please think only about recreational saltwater
fishing in North Carolina.

15 How many days did you go recreational
saltwater fishing from the SHORE in
North Carolina?

15 How many days did this person go
recreational saltwater fishing from the SHORE
in North Carolina?

The shore includes docks, bridges, causeways,
beaches, banks, or any other shore-based place
or area. Do not include freshwater fishing.

Did not recreational saltwater fish from shore
in last 12 months Go to question 16
Number of days saltwater shore
fishing in January and February of
2018

Did not recreational saltwater fish from shore
in last 12 months Go to question 16

Number of days saltwater shore
fishing in January and February of
2018

Number of days saltwater shore
fishing in last 12 months, including
January and February

16 How many days did you go recreational
saltwater fishing from a private or rental
BOAT that returned to shore in North Carolina?

16 How many days did this person go recreational
saltwater fishing from a private or rental BOAT
that returned to shore in North Carolina?

Do not include freshwater trips or trips where a
paid captain or crew helped locate and catch fish.

Did not recreational saltwater fish from
private boat in last 12 months
Number of days saltwater boat
fishing in January and February of
2018
Number of days saltwater boat fishing
in last 12 months, including
January and February
If you have more people in your household,
continue to Household Member 2. If you have
answered for all people in your household,
please return your survey.

Yes
No

HOUSEHOLD MEMBER 2

11
11. What is your gender?
Male
Female

Did not recreational saltwater fish from
private boat in last 12 months

Number of days saltwater boat
fishing in January and February of
2018
Number of days saltwater boat fishing
in last 12 months, including
January and February
If you have more people in your household,
continue to Household Member 3. If you have
answered for all people in your household,
please return your survey.

Number of days saltwater boat
fishing in January and February of
2018
Number of days saltwater boat fishing
in last 12 months, including
January and February
If you have more people in your household,
continue to Household Member 4. If you have
answered for all people in your household,
please return your survey.

Page 26

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 1

RTI International
NOAA FES Push to Web Survey Screenshots
Please note: Questions for household members 2-10 are identical. The order of shore fishing and boat
fishing questions is randomized, however, the randomized order will be the same for all household
members. There are two household members in this example. If information is entered for 10
household members, go to end.

INTRO

Page 27

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 2

WEATHER

Page 28

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 3

EVAC

Page 29

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 4

WARNING

Page 30

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 5

BEACH_FLAG

Page 31

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 6

FRESH_FISH

Page 32

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 7

SALT_FISH

Page 33

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 8

HH_PHN

Page 34

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 9

HH_DESC

Page 35

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 10

HH_YEARS

Page 36

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 11

HH_MEMBERS

Allows zero but then goes to next question assuming respondent should ask HH Member 1 items.

Page 37

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 12

ROSTERINTRO

THIS BEGINS HOUSEHOLD ROSTER SECTION. AFTER LOOP ONE AND FOR ALL LOOPS 2-10, THE
WORDING IS “this person” instead of “your”. The second loop will include intro statement after
checking to ensure there is an additional household member: “Now we’d like to ask about the next
member of your household.” GENDER_P(X).
HEADER IN QUESTION BOX FOR HOUSEHOLD LOOPS READS: “HOUSEHOLD MEMBER: PERSON 1
(YOU)” FOR 1ST LOOP AND “HOUSEHOLD MEMBER: “HOUSEHOLD MEMBER: PERSON X” FOR
SUBSEQUENT LOOPS. This text is included in a separate box.

Page 38

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 13

GENDER_P(1)

Page 39

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 14

AGE_P(1)

LOGIC: 2 DIGITS, 0-99 LOOP 1, AND 0-99 FOR LOOPS 2-10
Note mobile response is validated, not restricted, so the respondent will see "Please enter a numeric
value. Reponses can't be greater than 2 digits.”

No out of range flag.

Page 40

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 15

ORIGIN_P(1)

Page 41

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 16

RACE_P(1)

Page 42

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 17

STATEFISH_P(1)

IF RESPONSE IS NO, SKIP OUT OF RANDOMIZED SETS SECTIONS TO CHECK TO SEE IF THERE IS
ANOTHER HOUSEHOLD MEMBER (HH_MORE_P1)

RANDOMIZATION OF SHORE AND BOAT QUESTION SECTIONS BEGINS HERE. RANDOMIZATION OF
SETS IS THE SAME ORDER FOR ALL MEMBERS IN A HOUSEHOLD.

Page 43

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 18

SH_FLAG12PX

IF RESPONSE IS NO, SKIP TO BT_FLAG12PX. IF RESPONDENT SKIPS ITEM, “YES” BRANCHING WILL BE
FOLLOWED. NUMBER OF DAYS ITEMS ASKED.

Page 44

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 19

SH_TRIP_12PX

Page 45

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 20

BT_FLAG12PX

IF RESPONSE IS NO, CHECK TO SEE IF ADDITIONAL HOUSEHOLD MEMBERS HH_MORE_P1

Page 46

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 21

BT_TRIP12PX

Page 47

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix B Web Instrument Screen Shots", page 22

HH_MORE_P1

IF RESPONSE IS NO, GO TO END 2

Page 48

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix C Comparisons by State and Wave", page 1

Figure C1. Comparisons between the FES (blue) and web-push (green) designs for private boat fishing
prevalence.
0.12

Boat Fishing Prevalence

0.1
0.08
0.06
0.04
0.02
0
1

FES

Web Push

Mean Boat Fishing Days

6
5
4
3
2
1
0
1

5
FL

MA
FES

6
NY

Web Push

Page 49

Testing a Web-push Design for Estimating Recreational Fishing Effort

"Appendix C Comparisons by State and Wave", page 2

Figure C3. Comparisons between the FES (blue) and web-push (green) designs for shore fishing
prevalence.
0.12

Shore Fshing Prevalence

0.1
0.08
0.06
0.04
0.02
0
1

FES

Web Push

Mean Shore Fishing Days

7
6
5
4
3
2
1
0
1

5
FL

MA
FES

6
NY

Web Push

Page 50