Supporting Statement B

Status: Original and derived artifacts are available for this document.

Download: docx | pdf | html

SUPPORTING STATEMENT
U.S. Department of Commerce
National Oceanic & Atmospheric Administration
Marine Recreational Information Program (MRIP) Fishing Effort Survey 
OMB Control No. 0648-0652


    B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS 

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved. 

The Fishing Effort Survey (FES) is a monthly, cross-sectional mail survey designed to estimate the total number of private boat and shore-based recreational, saltwater fishing trips taken by residents of coastal states.  For each monthly survey administration, the FES utilizes address-based samples (ABS) covering Hawaii and 15 coastal states along the Atlantic coast (Maine through Florida). The sample frame is derived from the USPS Computerized Delivery Sequence File (CDS) and includes all full-time (non-seasonal), residential addresses, except for group quarters and PO boxes that are not flagged as the only way to get mail.  Sampling is stratified both geographically and by angler license status.  Within each state, sampling is stratified into coastal and non-coastal sub-state regions defined by geographic proximity to the coast.  Generally, counties with borders that are within 25 miles of the coast are in the “coastal” stratum and all other counties are in the “non-coastal” stratum. Rhode Island, Connecticut, Delaware, and Florida are not geographically stratified due to relatively consistent rates of fishing among counties.

Within the geographic strata, addresses are matched to the National Saltwater Angler Registry (NSAR), which consists of state lists of licensed saltwater anglers.  This creates two additional strata within each geographic stratum; license matched (households with one or more licensed anglers) and license unmatched (households that cannot be matched to NSAR).  Within each stratum, addresses are selected in a single stage using simple random sampling. 

Table 1 provides the sample universe (estimated number of occupied households), annual sample sizes (estimated number of sampled households), and estimated number of completed household surveys, overall and for each state.  The sample size for each state and month is expected to result in estimates of total fishing effort with coefficients of variation of 0.20 or less. Within each state and month, samples are allocated using a Neyman allocation, which distributes sample among strata in proportion to the product of the population size (number of households) and the standard deviation of total fishing activity, which is estimated from historical FES data.  


Table 1.  Estimated size of the sample universe, annual sample sizes, expected response rates and estimated number of completed household surveys.  

State
Estimated Number of Households
Estimated Number of Sampled Households
Expected Response Rate (%)
Estimated Completed Surveys
CT
1,433,635
30,231
27.0
8,109
DE
402,334
21,145
28.0
6,012
FL
8,815,441
16,660
24.0
4,029
GA
4,050,612
48,190
20.0
9,503
HI
494,826
30,183
33.0
9,897
ME
594,358
10,478
31.0
3,266
MD
2,370,494
23,443
25.0
5,837
MA
2,795,534
41,218
27.0
11,295
NH
555,737
15,453
29.0
4,556
NJ
3,516,978
31,837
24.0
7,749
NY
7,760,709
46,312
20.0
9,156
NC
4,263,652
32,874
27.0
8,888
RI
446,249
26,269
28.0
7,403
SC 
2,123,915
24,379
30.0
7,245
VA
3,351,434
26,116
27.0
7,054
Overall
42,975,908
424,788
26.0
110,000

*The denominator for the calculation of response rates includes all addresses, including those returned by the postal service as non-deliverable.  Based upon historical FES administrations, approximately 5% of addresses will be returned as non-deliverable.  In 2024, the overall response rate, excluding non-deliverable addresses, was approximately 27.1%.


2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden. 

2.1. Statistical Methodology and Data Collection Procedures

The FES is a self-administered mail survey utilizing an overlapping reference period design. The survey questionnaire asks respondents to report fishing activity for each of the two complete calendar months (reference months) immediately preceding the survey administration month (e.g., the March survey administration asks about fishing activity during January and February). This design generates two independent estimates for each reference month (M1,m and M2,m), which are subsequently combined using an optimized composite weighting approach to improve precision.  

As described in Question 1, sample selection is based on stratification of the target population by geography and angler license status. Data collection procedures have been extensively tested through several pilot studies (Andrews et al. 2010, 2014; Brick et al. 2012a; Andrews 2025).  Each year, the survey is conducted for twelve independent administration months.  The data collection period for each month begins one week prior to the end of the month with an initial survey mailing.  The timing of the initial mailing is such that survey materials are received prior to the end of the reference month.  The initial mailing is delivered by regular first-class mail and includes a cover letter stating the purpose of the survey, a survey questionnaire, a business reply envelope (BRE), and a $2.00 prepaid cash incentive (as described in section A.9).

One week following the initial mailing, a thank you/reminder postcard is sent via first class mail to all sampled addresses.  Approximately three weeks after the initial survey mailing, a follow-up mailing is delivered to all sample units that have not responded to the survey.  The follow-up mailing is delivered via first class mail and includes a nonresponse conversion letter, a second questionnaire and a BRE.       

2.2. Estimation Procedures

The FES estimates fishing effort (angler trips) by residents of sampled states.  An adjustment to account for non-resident fishing activity is derived from the MRIP Access-Point Angler Intercept Survey (APAIS, OMB Control No. 0648-0052).

FES weights for each survey administration are calculated in stages.  In the first stage, base weights ) for each sampled address within a given stratum are calculated as the inverse of the inclusion probabilities

where πi is the probability that unit i is included in the sample.

In the second stage, base weights are adjusted to compensate for unit nonresponse (e.g., when households fail to mail back the completed survey).  The sample is partitioned into nonresponse adjustment cells, or weighting classes, by state, sub-state region (coastal or non-coastal), license match (matched or unmatched) and boat registration match (e.g., whether the sampled address could be matched to database of state registered boats).  The base weights of the respondents in each adjustment cell () are divided by the response rate for that cell () to calculate the adjusted weight ()



where  ,
is the sum of the base weights of each respondent within adjustment cell c, and
 is the sum of the base weights of each nonrespondent within adjustment cell c.

In the third stage, nonresponse weights are further adjusted through a process known as raking, which adjusts weights so that the separate or marginal distributions for select variables in the sample data conform to corresponding distributions from independent data sources (Brick and Kalton 1996).  For the FES, auxiliary variables are derived from the American Community Survey, Current Population Survey and National Health Interview Survey, and include households with seniors, households with children, household tenure (own/rent), households with three or more household members, and wireless-only households.  Raking is an iterative procedure that sequentially adjusts weights to force sample distributions to match marginal distributions for each auxiliary variable.  The weights are repeatedly adjusted until the sample marginal distributions match the auxiliary marginal distributions for all selected variables.    Raked weights are calculated as 


where Rs is a generalized raking adjustment in state s.

During the fourth stage, raked weights are post-stratified to account for incomplete coverage of the target population.  Post-stratification is commonly used to make respondent data conform to target population totals from other sources independent from the survey (Brick and Kalton 1996).  The most recent estimates of the number of residential households available from the American Community Survey are used as population control totals.  Raked weights are post-stratified to household-level control totals within state and coastal and non-coastal strata.  The resulting post-stratified weight () of address  in stratum  is calculated as 


where the adjustment factor is equal to the ratio of the control total (, from the American Community Survey) to the estimated total based upon the sum of raked weights ().

Following the post-stratification stage, a weight trimming process is applied to mitigate the impacts of extreme values on the precision of survey estimates. Highly variable weights can result in large sampling variances, so it is desirable to minimize the frequency and size of extreme weights, though a tradeoff exists between increasing precision and biasing estimates through the trimming procedure. The FES utilizes an estimated Mean Square Error (MSE) trimming procedure to identify an optimal trimming level that minimizes the sum of sampling variance and the square of the estimated bias.

The process begins with the untrimmed effort estimate ( using the post-stratified weights ( and establishes an initial weighting threshold equal to the maximum weighted value ( within each specific estimation domain defined by state, reference month and fishing mode.  The procedure then reduces the weighting threshold by increments of 5%. At each step, any weighted value that exceeds the current threshold is “trimmed” by reducing its weight such that the revised weighted value is exactly equal to that threshold. The trimmed portion of the weights - the "residual weight" - is redistributed among the remaining untrimmed sample cases within the same estimation domain. This redistribution ensures that total weight sums remain constant and continue to match the target population control totals established during the post-stratification stage.   

For each threshold, the MSE for the trimmed estimate ( is estimated using the formula: 



Where  is the effort estimated using untrimmed weights,  is the effort estimated using trimmed weights, and  and represent their respective estimated variances. This procedure is repeated until a minimum MSE value is identified, indicating that the optimal level of trimming has been reached. Trimming is performed separately for each state, reference month and fishing mode, resulting in the final, trimmed weights ( used for estimation.
 
For each survey administration, reference month and fishing mode, estimates of resident fishing effort (, as well as associated estimates of variance , are calculated in SAS Version 9.4 using the survey means procedure.  For each state, survey month and reference month, total fishing effort is calculated as a weighted sum over the sample


	


where  and  are the final, trimmed weight and reported number of recreational fishing trips, respectfully, for address i in stratum h.

Variance is estimated using the Taylor series linearization 





The overlapping reference period design results in two independent measures of fishing effort for each month (m) - one measure from the survey administration immediately following the reference month (M1), and a second measure collected during the subsequent survey administration (M2).  An inverse variance-weighted compositing approach is used to combine the independent measures into a single estimate for each month (Hartley 1962).  The resulting estimate for each reference month is effectively a weighted average of the two independent trimmed estimates, with the weights optimized to minimize the variance of the composite estimate.

Let  and  represent the independent effort estimates for month  from administrations  and , and let  and  represent their respective variances. The composite weight  for the first administration is calculated based on the relative variance of both measures:




Using this optimized weight, the final composited point estimate  for month  is derived as follows:



Variance for the composited monthly effort estimates is estimated using a stratified bootstrap replication approach (Wolter 2007). This method is specifically selected to account for the additional uncertainty introduced by the compositing process itself, where the weighting factor (𝝀) is a random variable derived from sample data. For each monthly survey administration, 100 bootstrap replicates are generated by resampling households with replacement within strata. Within each replicate, weights are recalibrated to match known household population totals, and independent monthly estimates are produced. A unique, optimal compositing factor is then recalculated for every replicate based on the relative variances of its specific internal estimates. By calculating the final variance as the variability among these 100 composited replicate estimates, the model captures not only the sampling error of the underlying trip data but also the "variance of the variance" inherent in the dynamic weighting.

Adjustments to account for fishing activity by non-resident anglers are estimated from the APAIS.  For each coastal state and month, resident effort is adjusted by the inverse of the estimated proportion of fishing trips taken by resident anglers () to estimate total effort (



and



where the proportion is estimated from APAIS data as the weighted mean of an indicator variable.





and





3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied. 

Previous administrations of the FES resulted in response rates ranging from 20-35%.  We expect similar response for future administrations of the survey.

The expected response rates will be achieved by using standard mail survey protocols (Dillman et al, 2008).  An initial mailing will include an introductory letter stating the purpose of the survey, the survey questionnaire, a business reply envelope (BRE), and a prepaid, $2.00 cash incentive.  During testing of the FES design, a $2.00 incentive was found to be optimal in terms of maximizing response and minimizing data collection costs.  A thank-you/reminder postcard will be administered to all sample units one week following the initial mailing.  A final mailing, including a second questionnaire, a nonresponse conversion letter, and a BRE will be sent to all nonrespondents three weeks after the initial mailing.  

We will minimize nonresponse bias by using a questionnaire that maximizes responses by the entire sample population, including both fishing and non-fishing households.  Testing of the FES design included two versions of the survey instrument, a fishing-specific version and a more general version that included non-fishing questions.  The FES will utilize the more general “Weather and Outdoor Activity Survey” instrument, which provided the most representative sample of the general population during testing.

FES testing also included a nonresponse follow-up study to assess nonresponse bias in the data collection design.  Each wave, 400 nonrespondents were sampled for the follow-up study.  Data collection for the nonresponse study was initiated six weeks after the final contact for the FES with the delivery of an advanced letter via regular first-class mail.  Five days later, a survey packet, including a cover letter, questionnaire (the same questionnaire used in the FES), post-paid return envelope, and a $5.00 cash incentive was delivered via FedEx (USPS Priority Mail was used where FedEx is unavailable).  A thank you/reminder postcard was delivered eight days after the FedEx.  

The nonresponse follow-up study achieved a 40% response rate, and respondents to the nonresponse follow-up study were not significantly different from FES respondents in terms of recreational fishing activity.  These findings suggest that nonresponse bias in the FES is minimal. A second nonresponse follow-up study was implemented in 2020 and found similar results – follow-up study respondents were not significantly different from FES respondents with respect to fishing activity (Andrews 2021).

Finally, the FES will use information from the sample frame to define weighting classes for nonresponse weighting adjustments (as described above).  Weighting classes are defined such that response rates and fishing activity are similar within classes.  Andrews et al. (2010) and Andrews (2021) describe the effect of weighting adjustments on reducing nonresponse bias.  

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

Web-Push Study: NOAA Fisheries previously tested a web-push survey design in 2018-2019 that encouraged response via an online questionnaire prior to sending a traditional paper and pencil mail survey request.  Results from the study (Andrews 2019) demonstrated the willingness of respondents to report via an online questionnaire - >70% of responses were submitted online - as well as the associated cost savings – online response reduced postage and mailing costs by approximately 15%.  Despite the potential benefits of the web-push design, total response rates were 7-11 percentage points lower than the traditional FES mail survey design, increasing the overall data collection cost on a per-complete basis, as well as the risk for nonresponse bias.

More recent studies (Patrick et al. 2020, Heimel et al. 2024) demonstrate that sequential web-push designs have begun to outperform paper and pencil mail surveys with respect to response rates, resulting in additional cost efficiencies.  The web-push follow-up study will be conducted in parallel to the FES in 3-4 states for 3 reference months with a total sample size of 25,000 addresses.  We anticipate a response rate of 30% resulting in approximately 7,500 completed surveys. The study will utilize FES sampling and estimation procedures and the survey materials developed for the 2018-2019 web-push study, which are included as a supplementary document.  The web-push design will be evaluated with respect to; 1) overall response rates, 2) response rates among demographic sub-groups that are likely to differ with respect to their internet usage, 3) sample representativeness, 4) percentage of sample responding via the online instrument, 5) reported fishing activity, relative to the mail survey design, 6) timeliness of survey response, and 7) cost per completed survey.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency. 

Statistical support was provided by the following:
Dr. J. Michael Brick, Westat, 301-294-2004

Rob Andrews, Fisheries Biologist, NOAA Fisheries Service, Office of Science and Technology, 301-427-8105 is the point-of-contact for the Agency.
John Foster, Chief, Recreational Fisheries Statistics Branch, NOAA Fisheries Service, Office of Science and Technology, 301-427-8130.

References

Andrews WR (2021). Evaluating nonresponse bias in the mrip fishing effort survey. https://apps-st.fisheries.noaa.gov/pims/main/public?method=DO WNLOAD_FR_DATA&record_id=2018.

Andrews, W.R., J.M. Brick and N.A. Mathiowetz (2013).  Continued Development and Testing of Dual Frame Surveys of Fishing Effort (Testing a Dual-Frame, Mixed Mode Survey Design). 
https://apps-st.fisheries.noaa.gov/pims/main/public?method=DOWNLOAD_FR_PDF&record_id=931.

Andrews, W.R., J.M. Brick, N.A. Mathiowetz (2014).  Development and Testing of Recreational Fishing Effort Surveys (Testing a Mail Survey Design).
https://apps-st.fisheries.noaa.gov/pims/main/public?method=DOWNLOAD_FR_PDF&record_id=1179.

Andrews, W.R., J.M. Brick and N.A. Mathiowetz (2010).  Pilot Test of Dual Frame Two-Phase Mail Survey of anglers in North Carolina.
https://apps-st.fisheries.noaa.gov/pims/main/public?method=DOWNLOAD_FR_PDF&record_id=455.

Brick, J.M., W.R. Andrews and N.A. Mathiowetz (2012a). A Comparison of Recreational Fishing Effort Survey Designs
https://apps-st.fisheries.noaa.gov/pims/main/public?method=DOWNLOAD_FR_PDF&record_id=462.
Brick, J.M., W.R. Andrews, P.D. Brick, H. King, and N.M. Mathiowetz (2012b).  Methods for Improving Response Rates in Two-Phase Mail Surveys.  Survey Practice 5(4).
Dillman, D.A., J.D. Smyth, and L.M. Christian (2009). Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. New York: Wiley and Sons.

Heimel, S.K., R.L. Medway, and R.T. Horwitz (2024). Should We Offer Web, Paper, or Both? A Comparison of Single- and Mixed-Response Mode Treatments in a Mail Survey. Journal of Survey Statistics and Methodology 12: 593-623.

Lohr, S. (2009). Multiple Frame Surveys. Chapter 4 in Pfeffermann, D. (Ed.) Handbook of Statistics: Sample Surveys Design, Methods and Applications (vol. 29A). Elsevier, Amsterdam.

National Academies of Sciences, Engineering, and Medicine (2021). Data and management strategies for recreational fisheries with annual catch limits. https://www.nationalacademies.org/projects/DELS-OSB-19-02/publication/26185. 

National Research Council (2006). Review of recreational fisheries survey methods. Retrieved from: https://www.nap.edu/catalog/11616/review-ofrecreational-fisheries-survey-methods.

Patrick, M.E., M.P. Couper, M.J. Parks, V. Laetz, and J.E. Schulenberg (2020). Comparison of a Web-Push Survey Research Protocol with a Mailed Paper and Pencil Protocol in the Monitoring the Future Panel Survey. Addiction 116: 191-199.

Wolter, K. M. (2007). Introduction to Variance Estimation (2nd ed.). Springer-Verlag.