0648-0783 Supporting Statement B

SUPPORTING STATEMENT

U.S. Department of Commerce

National Oceanic & Atmospheric Administration

Survey To Collect Economic Data From Recreational Anglers Along the Atlantic Coast

OMB Control No. 0648-0783

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The target population for the NARFS III is recreational anglers who have fished for North Atlantic cod or haddock in federal waters off the coast of Maine, New Hampshire, or Massachusetts in the past five years. Our sample frame will be drawn from 2022 recreational fishing license/registry databases maintained by the states of Maine, New Hampshire, and Massachusetts. Table 1 displays information about these databases in 2021, as the 2022 registry data is not yet complete. Note that the composition of the 2021 database is unlikely to differ much from that of the 2022 database.

Given their lower per-unit cost relative to mail surveys and feedback from NARFS I and II focus group participants, web surveys will account for about three quarters of the total number of surveys distributed.

Table 1. 2021 saltwater recreational fishing licensees by state and contact information availability

State	Total # of licensees	Mailing address	Email address	Mailing and email address
Maine	62,434	62,109	37,456	37,182
New Hampshire	31,269	30,863	4,924	4,835
Massachusetts	186,467	186,422	139,403	139,385
Total	280,170	279,394	181,783	181,402

As discussed in our response to question A1, the NARFS III is directly comparable to the NARFS I survey that was conducted in 2019. The NARFS III will follow the same email/mail sampling methodology used for the NARFS I. The response and eligibility rate from the NARFS I survey provides the basis for those expected of the NARFS III.

The NARFS III sampling methodology involves sending mail invitations containing a web link to participate to a list of potential respondents drawn using stratified random sampling from the state license/registration databases. We expect more than three quarters of the completed responses will come from the web version of the survey, with about a quarter of the completed responses derived from the mail version of the survey. This expectation was realized in the NARFS I as 81% of the respondents that completed the survey used the web version; the remaining 19% that completed the survey did so by filling out the mail version.

The expected response rate from the NARFS I provides the basis for that expected in NARFS III. Table 2 shows the NARFS I response rates by state. Survey invites are the total number of license-holders that the NARFS I attempted to contact. Eligible completes are sampled records from anglers that fished for cod or haddock during the previous five years in Maine, New Hampshire, or Massachusetts. NARFS III will use the same five-year eligibility definition. Finally, ineligible completes shown in Table 2 refer to anglers that completed the NARFS I, but did not fish for cod or haddock during the previous five years. These anglers were only asked to answer a subset of the total questions in the survey since they did not fish for the target species during the previous five years. NARFS III will follow this same approach. Anglers that indicate they did not fish for cod or haddock during the past five years will only be asked the first question, then will be directed to skip to the last section of the survey (Section D) to answer demographic questions. Please see Part A, Question 2 that explains the need for collecting a brief subset of information from anglers not targeting the species of interest. As shown in Table 2, the overall NARFS I completion response rate was approximately 33% with eligible respondents comprising about 39% (516/1,326*100) of the completed surveys.

Table 2. NARFS I Survey Disposition by State

State	Survey Invites	Eligible Completes	Ineligible Completes	Eligible and Ineligible Completes
Maine	726	63	148	211
Massachusetts	2,058	256	484	740
New Hampshire	1,216	197	178	375
Total % of total invites	4,000	516 12.9%	810 20.3%	1,326 33.2%

As the NARFS III will be an update to the NARFS I, administered in the same states, focused on the same species, using the same sampling methodology, we are expecting a similar overall and eligible respondent survey completion rate. Thus, we are expecting a completion rate of approximately 33%, with eligible respondents (those that have fished for cod or haddock in the previous five years) comprising about 39% of the completed surveys.

Identical to the NARFS I, we intend to send out 4,000 survey invites allocated across states according to the proportion of angler trips targeting/catching the species of interest. The most recent data from the National Marine Fisheries Service’s Marine Recreational Information Program (MRIP) show about 305 thousand angler trips in Maine, New Hampshire, and Massachusetts targeted and/or caught North Atlantic cod or haddock. The breakdown of angler trips by state in 2021 was virtually identical to 2019, so we intend to send out the same number of invites by state, as was done for NARFS I (Table 2). Thus, we are expecting 1,326 completed surveys. Approximately 39% (516) of the completed surveys are expected to be from anglers targeting and/or catching cod or haddock.

Implementation of NARFS III will follow procedures suggested by Dillman et al. (2009). These procedures have been followed extensively in survey research to increase response rates for mail and web surveys. Repeated contacts with respondents through one or more mediums (phone, email, or mail), allow flexibility regarding how potential respondents choose to respond and serve to maximize the overall survey response rate. As stated above, for NARFS III we plan to use email and mail surveys. The implementation schedule for NARFS III is described below.

Track A: Mail push-to-web invitation letter for respondents that have an email address on file.

This will be the initial invitation track for those with email addresses. Approximately 65% (181,402/280,170*100) of the saltwater recreational license holders across Maine, New Hampshire, and Massachusetts provided email addresses in 2021 (Table 1). We expect similar percentages for the 2022 license frames that we will use for NARFS III. The mail push-to-web letter provides an opportunity to explain the survey’s purpose and elicit cooperation. The letters will introduce the survey, demonstrate its relevance, encourage web participation, and assure confidentiality. Subsequent contacts with Track A participants who do not respond after receiving the mail push-to-web letter are as follows:

• An email invitation, which reinforces the importance of the survey and reminds those who did not respond about the survey. The reminder email will contain a web link to the survey, in the hopes that that will make it easier for participants to respond online.

• A reminder email, which reinforces the importance of the survey and reminds those who did not respond about the survey. The reminder email will contain a web link to the survey.

• A reminder letter, which reinforces the importance of the survey and reminds those who did not respond about the survey. The reminder letter will contain a web address to access the survey online.

• A mail package that includes the paper version of NARFS III and a cover letter explaining the importance of the survey.

Track B: Mail push-to-web invitation letter for respondents without an email address on file.

Track B is very similar to Track A, with the exception that these participants will not receive email contacts. Any subsequent contact after the first mail push-to-web contact will be made by mail.

A version of the NARFS III survey instrument and associated cover letters are included as an attachment.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical methodology for stratification and sample selection

A stratified random sampling strategy will be used to focus efforts on the population of interest and reduce implementation costs. Participants will be drawn from each state license database ( ) in proportion to that state’s contribution to the total number of recreational fishing trips taken during 2021 that caught or targeted Atlantic cod or haddock. Using publicly-available MRIP data, we estimated the total number of trips to be about 305 thousand; of these trips, Maine, New Hampshire, and Massachusetts accounted for 18%, 31%, and 51%, respectively. If we denote these proportions as , and the proportions from the realized sample as , we can use stratification weights equal to during estimation of the economic model to obtain consistent parameter estimates (Louiviere et al. 2000).

Estimation procedure

The most critical input for estimating the angler behavioral model are the data collected from Section B of NARFS III, the discrete Choice Experiment (CE). Each of the six questions in this section presents respondents with three options—two hypothetical cod and haddock fishing trips that vary in catch levels, probabilities of keeping a given number of fish, and cost, and the option to not go recreational saltwater fishing—and asks them to indicate which of the three options would be their first if they were presented with these alternatives in the real world. We use these data to estimate a vector of parameters that represent risk and ambiguity attitudes, and marginal utilities of fishing trip attributes included in the discrete choice experiment (see Part A, Table 1 for a summary of attributes).

We will analyze the discrete choice experiment data collected by NARFS III using random utility maximization (RUM) models, which decompose utility into its observable and unobservable components (McFadden 1973). RUM models assume that when faced with multiple alternatives, individual n will select alternative (from among other alternatives j) that maximizes utility, .

Partitioning into its two component parts, the choice of alternative is such that

where and are indirect utilities derived from alternatives j and i, respectively, and and are their respective error terms. Unlike standard specifications, which typically assume that is linear in parameters, , we are interested in eliciting angler’s risk and ambiguity preferences in catch space (Foster and Just, 1989, Leggett, 2002, de Palma and Picard 2004, Holzer and McConnell 2017). Thus, we assume a quasilinear specification in which risk preferences associated with the random catch are represented by a von Neumann-Morgenstern utility function with constant absolute risk aversion (CARA),

2. 

where k indexes the possible catch outcomes , is the probability that outcome k occurs, denotes a vector of non-random site attributes, is income net of trips costs, r denotes the Arrow-Pratt coefficient of absolute risk aversion, and and are parameters to be estimated. Function is given by

2. 

and combining both expressions, we write

2. 

that relates observed attributes to utility. In our choice experiments, while the catch is deterministic, the keep (essentially determined by whether those fish are legal size) is stochastic.

Because the ’s are stochastic, it is not possible to determine absolute levels of utility; however, probabilistic inference about individuals' choices can be made under the standard assumption for logit models that these terms are independently and identically distributed Type I extreme values. The probability that angler selects fishing alternative is

Train (2003) calculates this probability for a multinomial logit (MNL) model as

Next, to specify the contribution to the likelihood from the choices involving ambiguity, we use the α-Maxmin Expected Utility (α-MEU) model with fixed priors (Gilboa et al. 1989, Ghirardato et al. 2004). To illustrate, assume that angler n is presented with an hypothetical trip i that is characterized by three possible outcomes in terms of the number of legal-sized summer flounder kept, denoted , and , with known probability for outcome given by . If the probabilities associated with outcomes and are not specified, the angler faces uncertainty (i.e. unknown objective probabilities). In this case, we specify his/her utility as follows (Potamines and Zhang 2012, Ahn et al. 2014, Gneezy et al. 2015).

2. 

where and . Thus, above the unknown probabilities are skewed using weights and for the low and high keep, respectively. The parameter α represents the attitude towards ambiguity: indicates preferences that are ambiguity averse indicates preferences that are ambiguity seeking, and if we have the standard subjective expected utility specification. The probability of choosing the trip option with unknown probabilities can be expressed as

2. 

The probability of each individual in the sample choosing the alternative they were observed to actually choose is

where if an individual n is observed to choose alternative i in the choice experiments only involving risk, and zero otherwise. Similarly, if an individual is observed to choose alternative i in the choice experiments involving ambiguity, and zero otherwise. We will jointly estimate, using maximum likelihood, parameters which maximize

Degree of accuracy needed for the purpose described in the justification

The number of completed surveys needed to estimate the behavioral model parameters with adequate precision is based on the experimental design of the CE. We evaluated the minimum sample size required for the ensuing CE economic model using the most widely accepted approach described by Orme (2010).

The approach suggested by Orme for determining the minimum sample size for CE modelling, is to set

where is the number of respondents, is the number of choice tasks per respondent, is the number of alternatives per choice task excluding the opt-out alternative, and is the largest number of levels for any one attribute for a main effects model. While the value of 500 refers to the number of times each main effect level of interest should be represented across the design to have ample stability in the ensuing parameter estimates, Orme (2010) notes that this value “was intended to be a minimum threshold when researchers cannot afford to do better. It would be better, when possible, to have 1,000 or more representations per main-effect level”. When its right-hand side is set to 500, solving Equation (12) for based on our main-effects experimental design ( = 6¹, = 2, = 6) yields 250; when the right-hand side of Equation (12) is set to 1,000, = 500. Taken together, the general rule of thumb provided by Orme (2010) suggests that the minimum sample size for the NARFS III should be between 250 and 500.

Given results of the Orme (2010) approach used to determine the minimum sample size required for NARFS III, we seek to obtain at least 500 surveys completed by eligible anglers. Meeting this objective conditional on an expected 33% overall survey response completion rate, with eligible survey respondents comprising 39% of the completed surveys, requires distributing a minimum of 3,885 surveys across the three states. See Question 1 above for a description of the expected survey response rates. Distributing 4,000 surveys in the first NARFS resulted in 516 completed surveys, which exceeds the minimum sample size range suggested by the Orme (2010) test. As we expect the response rate to be at least as high as the first NARFS, our sample size of 4,000 should be more than sufficient to provide representative population estimates.

Unusual problems requiring specialized sampling procedures

The proposed stratified random sampling design of saltwater recreational license holders provided the necessary data for the first NARFS and we don’t anticipate any changes for NARFS III.

Any use of periodic (less frequent than annual) data collection cycles to reduce burden

NARFS III will be a one-off data collection effort conducted at a single point in time. Respondents will be asked to respond to a single questionnaire with no follow-up data requested.

2. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

As was done for the first NARFS, we will be offering a $2 incentive and have instituted survey design features to improve unit and item non-response. The findings from the $2 incentive experiment on survey response in the first NARFS showed that sampled anglers who received the incentive were significantly more likely to respond to the questionnaire than respondents that did not receive the incentive (38.46% versus 25.31% respectively; chi-square = 56.45, p < 0.001). Based on these findings and other research conducted on small monetary prepaid incentives, discussed under A9, NARFS III will include a $2 incentive for all survey invites. By providing the $2 incentive in all survey invitations, rather than to a subset of the sample as was done for the first NARFS, we expect the NARFS III survey response rate could be even higher than the first NARFS.

Additionally, the survey design features implemented in the first NARFS resulted in low unit and item non-response so will adopt these same features. In the first NARFS, 1,134 sampled license holders accessed the web questionnaire either by going to the survey website and entering their unique passcode or by clicking on a unique link provided in a subsequent email contact. Approximately 92% (1,039) of the sampled license holders that accessed the web questionnaire ultimately completed. The high completion percentage speaks favorably to the clarity of questions and instructions in the questionnaire as well as the screen presentation of these elements on desktop and mobile devices. It is not uncommon to see respondents breakoff when presented with CE questions given the cognitive demand required to evaluate the hypothetical scenarios. The CE questions were optimized for appearance on both desktop and mobile devices to ensure a positive user experience and we will follow the same design for NARFS III. Measures were also taken to mitigate item non-response within the CE questions such the use of soft validations with messages tailored to each preference question. A hide/reveal of the second preference question was implemented to improve data quality such that respondents could not pick the same choice twice within a given scenario. Only 19 of the 1,039 web respondents skipped one of the CE questions, meaning 98% of web respondents fully completed the most important section of the questionnaire. The measures taken to optimize user experience, improve data quality, and minimize item nonresponse coupled with clearly and concisely written questions and instructions contributed significantly to this successful completion rate.

We statistically tested for survey nonresponse bias in the first NARFS using data collected from a telephone survey. We found that survey non-response bias had no impact on the estimation results. Given the high cost, additional burden on society, extremely low telephone contact rate achieved in the first NARFS (6.9%), and the finding that non-response bias was not a factor in the NARFS I, we do not plan to conduct a nonresponse survey for NARFS III. We anticipate the $2 monetary incentive will help mitigate survey nonresponse bias by attracting participation from those who otherwise might not respond to the survey. We will also adopt the same survey design features that were so successful in the first NARFS, which will help maximize survey completion rates, improve unit and item nonresponse, and promote a better respondent experience. Additionally, we will use responses to a series of demographic questions contained in Section D of the questionnaire to assess relative sample representativeness by comparing the characteristics of our sample to the characteristics of the population of recreational anglers at large, which are available in NOAA-sponsored nationwide angler expenditure reports.

Data collected from NARFS III will improve our ability to understand and predict how changes in management options and regulations may change fishing mortality and the number of trips anglers take for North Atlantic cod and haddock. This data will allow fisheries managers to conduct updated and improved analysis of the socio-economic effects to recreational anglers and to coastal communities of proposed changes in fishing regulations. The recreational fishing community and regional fisheries management councils have requested more species-specific socio-economic studies of recreational fishing that can be used in the analysis of fisheries policies. This survey will address that stated need for more species-specific studies.

2. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

We tested and refined the proposed survey instrument prior to conducting both the NARFS I and the NARFS II. This survey will use the implementation and methodological lessons learned from both of those surveys.

Prior to conducting the NARFS I, we conducted focus group sessions with recreational cod and haddock anglers in the study region during November of 2018. Using the license databases described in B1, we recruited focus groups participants who differed in terms of gender, age, and Atlantic cod/haddock fishing experience to obtain feedback from a diverse mix of anglers. The first two of these focus groups were held in Braintree, MA, and the second two groups were held in Portland, ME. We used feedback received from those focus groups to (a) conform survey language to regional differences in dialect and ensure consistent interpretation of survey questions across the study region, and (b) design contextually realistic and straightforward choice experiment questions.

During the focus groups conducted for NARFS II, we tested the design of the improved approach to display uncertainty in the number of fish kept (i.e. pie charts with probability labels) with undergraduate students at the University of Maryland. The respondents overwhelmingly preferred this approach to the alternative of describing probabilities using frequency bar graphs. Then, to test the efficacy of the proposed survey instrument, we conducted focus group sessions with recreational anglers in the study region during March of 2019. Using 2018 license frames, we recruited focus group participants who differed in terms of gender, age, education, income level, and summer flounder/black sea bass fishing experience to obtain feedback from a diverse mix of anglers. Both focus groups were held in Cranford, New Jersey. After a brief introduction to provide context on the main purpose of the survey, focus participants were asked to answer the survey. Then, we facilitated the discussion on each of the sections in the survey, with particular emphasis on the CE questions and the pie charts describing uncertainty in the keep of summer flounder and black sea bass.

We used feedback received from these focus groups to (a) design contextually realistic choice experiment questions, (b) discuss the most intuitive design of choice experiments involving probabilistic outcomes, including the appropriate number of random outcomes, and (c) discuss the need to incorporate questions in which the probabilities are unknown a priori (i.e. hence the need to include scenario to deal with ambiguity regarding the catch).

2. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Design, analysis, report

Andrew Carr-Harris (co-P.I.)

Northeast Fisheries Science Center

Phone: 914-330-7881

Scott Steinback (co-P.I.)

Northeast Fisheries Science Center

Phone: 508-274-8067

Sabrina Lovell (co-P.I.)

Office of Science and Technology

Phone: 301-427-8153

Data collection

Gustavo Rubio

ECS Federal, contracting company

Phone: 301-427-8180

REFERENCES

Bekker-Grob, Esther W. de, Bas Donkers, Marcel F. Jonker, and Elly A. Stolk. 2015. “Sample Size Requirements for Discrete-Choice Experiments in Healthcare: A Practical Guide.” Patient 8 (5): 373–84.

Dillman, Donald A., Jolene D. Smyth, and Leah M. Christian. 2009. Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 3rd ed. New York: Wiley.

Groves, Robert M. 2006. “Nonresponse Rates and Nonresponse Bias in Household Surveys.” Public Opinion Quarterly 70 (5): 646–75.

Lee, Min-Yang, Scott Steinback, and Kristy Wallmo. 2017. “Applying a Bioeconomic Model to Recreational Fisheries Management: Groundfish in the Northeast United States” 32 (2).

Louiviere, Jordan J., David A. Hensher, and Joffre D. Swait. 2000. Stated Choice Methods: Analysis and Applications. Cambridge University Press.

McFadden, Daniel. 1973. “Conditional Logit Analysis of Qualitative Choice Behavior.” In Frontiers in Econometrics, 105–142. New York.

Train, Kenneth. 2003. Discrete Choice Methods with Simulation. New York: Cambridge University Press.

1This was the number of choice tasks per respondent in the first NARFSS (t = 6). Additionally, feedback obtained from the NARFS II focus groups’ indicated that participants were able to properly answer six CE questions, but a larger number of questions became burdensome.

4