SS MRIP Part B Final

SUPPORTING STATEMENT

MARINE RECREATIONAL INFORMATION PROGRAM

PART B

OMB CONTROL NO. 0648-XXXX

2. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

1.1. MRIP Fishing Effort Survey

The MRIP Fishing Effort Survey (MFES) is bi-monthly (wave), cross-sectional mail survey designed to estimate the total number of individuals who participate in marine recreational fishing and the total number of private boat and shore-based recreational fishing trips taken by anglers in the study states. The survey consists of two independent components; 1) the Resident Angler Survey (RAS), which estimates saltwater fishing effort by residents of coastal states, and 2) the Nonresident Angler Survey (NAS), which estimates saltwater fishing effort by residents of non-coastal states. The RAS is an address-based sample (ABS) that covers all residential addresses within the study states. The NAS is a list-based sample that covers individuals who are licensed to participate in saltwater fishing in the study states but reside in a different state.

1.2. Resident Angler Survey

The sample frame for the RAS is a list of all residential addresses that are serviced by the United States Postal Service (USPS) within the study states. Sampling is stratified by state and geographic proximity to the coast. Specifically, counties with any border that is within 25 miles of the coast are in the coastal stratum, and all other counties are in the non-coastal stratum¹. This stratification serves two purposes. First, residents of coastal counties are more likely to participate in recreational saltwater fishing than residents of non-coastal counties – historical estimates from the Marine Recreational Fisheries Statistics Survey (MRFSS) demonstrate that 65-90% of recreational saltwater fishing trips in the study states are taken by residents of coastal counties within those states. Stratification provides an opportunity to sample at different rates among strata and subsequently increase the efficiency of data collection. Secondly, the coastal resident stratum is consistent with the coverage of the Coastal Household Telephone Survey, which will allow for direct comparisons between the two surveys.

Each wave, a representative sample of addresses is selected within each stratum in a single stage. The sample size is sufficient to permit sub-sampling, as described below. Addresses are selected from a comprehensive list of residential addresses maintained by a vendor licensed to distribute the USPS Computerized Delivery Sequence File. Following selection, sampled addresses in each state and stratum are matched, by address and telephone number, to databases of anglers licensed to participate in saltwater fishing in the respective state. Databases of licensed anglers are provided to NMFS by state natural resource agencies approximately one month prior to the beginning of data collection for each wave. Prior to matching, addresses within the license databases are formatted to conform to USPS postal addressing standards, and duplicate angler records, as well as records for individuals less than18 years of age are identified and removed.

Matching addresses to license databases screens the ABS sample to identify households with (matched) and without (unmatched) licensed anglers, effectively stratifying the sample into matched and unmatched strata (Lohr, 2009). Stratification provides an opportunity to optimize sampling among strata - previous studies (Andrews et al., 2010, Brick et al., 2012a) have demonstrated that residents of households that match to license databases respond to fishing surveys at a higher rate and are more likely to have fished during the reference wave than residents of unmatched households. The survey instrument collects information about the recent saltwater fishing activity for all residents of each sampled address (i.e. each address is a cluster of individuals who reside at the address).

Table 1 provides the sample universe, initial ABS sample sizes, final target sample sizes and estimated number of completed household interviews for each stratum within a given reference wave. Initially, sample will be distributed among strata such that the expected yield of completed household interviews is uniform among states. However, allocations will be reassessed following each wave. The final target allocation is achieved by retaining all matched addresses in the sample and sub-sampling unmatched addresses. Target sample sizes are expected to result in a completed number of household surveys that are optimally allocated among strata and achieve a coefficient of variation of 15% on estimates of total fishing effort² for each state and wave. Sampling requirements are based upon results from previous MRIP pilot studies.

Table 1. Estimated size of the sample universe, initial and final sample sizes, expected response rates and estimated number of completed household interviews per wave for the resident angler survey.

State	Geographic Stratum	License Stratum	Estimated Number of Households3	Initial ABS Sample Size4	Estimated Final ABS Sample Size56	Expected Response Rates7	Estimated Completed Household Surveys
FL	Coastal	Matched	737,818	1,325	1,325	57%	759
FL	Coastal	Unmatched	6,754,869	12,131	2,930	43%	1,270
MA	Coastal	Matched	53,612	1,290	1,290	57%	665
MA	Coastal	Unmatched	1,856,226	44,664	2,262	43%	882
MA	Noncoastal	Matched	43,023	198	198	53%	94
MA	Noncoastal	Unmatched	585,613	2,695	1,055	41%	388
NC	Coastal	Matched	199,839	1,290	1,290	57%	665
NC	Coastal	Unmatched	557,660	3,600	2,262	43%	882
NC	Noncoastal	Matched	222,650	198	198	53%	94
NC	Noncoastal	Unmatched	2,812,924	2,502	1,055	41%	388
NY	Coastal	Matched	75,957	1,290	1,290	57%	665
NY	Coastal	Unmatched	4,434,800	75,318	2,262	43%	882
NY	Noncoastal	Matched	35,537	198	198	53%	94
NY	Noncoastal	Unmatched	2,650,522	14,768	1,055	41%	388
Total			21,021,050	161,467	18,671	48%	8,116

1.3. Nonresident Angler Survey

Non-resident anglers are sampled from lists of individuals who are licensed to participate in saltwater fishing in each study state. The sample frame for each state consists of anglers who were licensed to fish in the state (license state) during the wave but reside in another state. Databases of licensed anglers are provided to NMFS by state natural resource agencies approximately one month prior to the beginning of data collection for each wave. Prior to sampling, addresses within the license databases are formatted to conform to USPS postal addressing standards, and duplicate angler records, as well as records for individuals less than18 years of age are identified and removed.

Each wave, a simple random sample of licensed anglers is selected from each state’s license frame. The survey instrument collects information about recent saltwater fishing activity for the sampled angler, as well as any other individuals who reside at the same address as the sampled angler; each sampled angler represents a cluster of anglers who reside at the same address. Table 2 provides the sample universe, sample size, expected response rates and estimated number of completed surveys for each state within a given reference wave.

Table 2. Estimated size of the sample universe, initial and final sample sizes, expected response rates and estimated number of completed interviews per wave for the nonresident angler survey.

State	Estimated Number of Nonresident Anglers8	Sample Size	Expected Response Rate9	Estimated Completed Surveys
FL	443,711	285	60%	171
MA	73,195	285	60%	171
NC	159,743	285	60%	171
NY	14,555	285	60%	171
Total	691,204	1,140	60%	684

A resident of a study state who is also licensed to fish in one of the other study states could be sampled for both the RAS and the NAS. However, given the sampling rates, it is extremely unlikely (less than 1/10 of 1%) that the same individual would be sampled from both frames. Each wave, sample from each frame will be cross-checked against the other sample to identify any duplicates. If this situation were to occur, the NAS sample will be withheld and treated as a special case of nonresponse.

1.4. Experimental Tests

Previous MRIP pilot studies (Andrews et al. 2010, Brick et al. 2012a) demonstrated that addresses that match to angler license databases respond to fishing surveys at a higher rate and are more likely to have participated in saltwater fishing than unmatched addresses. These studies accounted for this differential nonresponse through nonresponse weighting adjustment.

This study will include an experiment to test two versions of the RAS questionnaire. The two versions will be evaluated in terms of overall response rates and the degree of differential nonresponse and reported fishing activity between anglers (matched addresses) and nonanglers (unmatched addresses). One version of the questionnaire (Version 1) utilizes a “screen out” approach that quickly identifies anglers (and non-anglers) and encourages participation by minimizing the number of questions, particularly for non-anglers. The second version (Version 2) utilizes an “engaging” approach that encourages response by broadening the scope of the questions to include both fishing and non-fishing questions.

During the first two waves of data collection, sampled addresses within each stratum will be randomly allocated into treatments defined by the questionnaire. Tables 3 and 4 provide the sample sizes for each treatment group, as well as the expected detectable differences in response rates and fishing incidence rates, respectively, between experimental treatments. The version that minimizes differential nonresponse between matched and unmatched addresses will be utilized for the subsequent six waves.

Table 3. Expected detectable differences in response rates for questionnaire experiment

RAS Questionnaire	Sample Size10	Expected Response Rate	Expected Detectable Difference in Response Rates11
Version 1 (Screen Out)	16,804	48%
Version 2 (Engaging)	16,804		1.55%

Table 4. Expected detectable differences in fishing incidence for questionnaire experiment

RAS Questionnaire	Expected Responses	Expected Fishing Incidence	Expected Detectable Difference in Fishing Incidence11
Version 1 (Screen Out)	8,066	22%
Version 2 (Engaging)	8,066		1.86%

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

2.1. Data Collection Procedures

The RAS and NAS are both single-phase, self-administered mail surveys, and data collection procedures for the two survey components are identical. These data collection procedures have been extensively tested through previous MRIP pilot studies (Andrews et al. 2010, Brick et al. 2012a). The surveys are administered for eight independent, two-month reference waves. The data collection period for each wave begins one week prior to the end of the wave with an initial survey mailing. The timing of the initial mailing is such that materials are received prior to the end of the reference wave. The initial mailing is delivered by regular first class mail and includes a cover letter stating the purpose of the survey, a survey questionnaire, a post-paid return envelope and a prepaid cash incentive (as described in section A.9).

One week following the initial mailing, a follow-up thank you/reminder contact is initiated. For sample units with an attached landline telephone number (sample units for which a landline telephone number can be found through a lookup service), an automated voice message is delivered to remind sample units to complete and return the questionnaire. Previous studies have demonstrated that varying the delivery mechanism, for example, switching from regular first class mail to telephone or special mail, may improve response rates in mail surveys (Brick et al., 2012b). For sample with no associated landline telephone number, a thank you/reminder postcard is sent via regular fist class mail. We expect to identify landline telephone numbers for approximately 50% of sampled addresses.

Three weeks after the initial survey mailing, a follow-up mailing is delivered to all sample units that have not responded to the survey. The follow-up mailing is delivered via first class mail and includes a nonresponse conversion letter, a second questionnaire and a post-paid return envelope.

2.2. Estimation Procedures

Final sample weights for both the RAS and the NAS are calculated in stages. In the first stage, base sample weights within each stratum are calculated as the inverse of the selection probability ( , where π_i is the probability of selecting unit i for the sample). In the RAS, base weights for addresses that cannot be matched to an angler license database (sample units in the unmatched strata), are adjusted to account for subsampling by multiplying the base weight by the inverse of the subsampling rate.

In the second stage, base weights (or adjusted base weights in unmatched RAS strata) are adjusted to account for nonresponse. Specifically, the weights of nonresponding units are increased by the inverse of the weighted response rate within nonresponse adjustment cells

where

and and are the sums of base weights in cell c for respondents and nonrespondents, respectively. Weights for all individuals who reside at a sampled address are equal to the final sample weight for the address.

In the RAS, nonresponse adjustment cells will be defined by state or residence, coastal/non-coastal county, matched/unmatched designation, and whether or not the address was successfully matched to a landline telephone number. In the NAS, adjustment cells will be at the stratum level (license state). Other potential criteria for defining nonresponse adjustment cells will be examined after each wave of data collection and may include demographic information and type of recreational fishing license.

Estimates of total fishing effort, as well as associated estimates of variance, are calculated in SAS Version 9.3 using the surveymeans procedure. For a given coastal state and wave, total effort is the sum of resident angler effort (from RAS) and nonresident angler effort (from NAS), both of which are calculated as weighted sums

where and are the final weight and reported number of recreational fishing trips, respectfully, for unit j at address i of stratum h.

Variance of the total effort estimate is estimated using the Taylor series method

where

For estimating total fishing effort, we expect stratification to be more effective than simple random sampling due to the oversampling of coastal and licensed households. Gains in efficiency will be offset somewhat by weighting effects, which will increase the variance of total effort estimates. Given these two factors, we expect a design effect of approximately 1.1.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

The expected response rates for the RAS and NAS are 48% and 60%, respectively. Previous MRIP pilot studies utilized similar data collection procedures and achieved similar response rates.

The expected response rates will be achieved by using standard mail survey protocols (Dillman et al, 2008). An initial mailing will include an introductory letter stating the purpose of the survey, the survey questionnaire, a business reply envelope, and a prepaid cash incentive. Incentive levels of $0, $1, $2 and $5 will be evaluated. Either a thank-you/reminder postcard or automated voice message will be administered to all sample units one week following the initial mailing. A final mailing, including a second questionnaire, a nonresponse conversion letter, and a business reply envelope will be sent to all nonrespondents three weeks after the initial mailing.

We will minimize nonresponse bias by using a questionnaire that maximizes responses by the entire sample population, including both anglers and non-anglers. Experimental testing of different versions of the survey questionnaire is described in Question 1, Section 1.4.

We will assess nonresponse bias in three ways. First, we will compare early and late responders with respect to reported fishing activity. This analysis will identify differences in respondents based upon the level of effort required to solicit a response. Previous studies (Brick et al., 2012) demonstrated that early and late responders are similar in terms of reported recreational fishing activity.

The second approach will utilize information from sample frame to define weighting classes for postsurvey weighting adjustments. Weighting classes will be defined such that response rates and fishing activity are similar within classes. Nonresponse bias will be measured by comparing unadjusted estimates to estimates that have been adjusted to account for differential nonresponse among weighting classes. Previous studies identified differential nonresponse and reported fishing activity between households with and without licensed anglers and demonstrated that nonresponse weighting adjustment decreased estimates of fishing effort by 25% over unadjusted estimates (Andrews et al., 2010).

Finally, we will conduct a nonresponse follow-up study that includes a more intensive effort to contact nonrespondents. In each of the six waves, 400 nonrespondents will be sampled for the follow-up study. Data collection for the study will be initiated six weeks after the final contact for the RAS and the NAS with the delivery of an advanced letter via regular first-class mail. Five days later, a survey packet, including a cover letter, questionnaire (the same questionnaire used in the RAS and NAS), post-paid return envelope and a $5.00 cash incentive will be delivered via FedEx (USPS Priority Mail will be used where FedEX is unavailable). A thank you/reminder postcard will be delivered eight days after the FedEx.

We expect a response rate of 25% for the follow-up study. Respondents to the follow-up study will be compared to RAS and NAS respondents in terms of reported fishing activity and demographic characteristics.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

No additional testing is planned.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Statistical support was provided by the following:

Dr. J. Michael Brick, Westat, 301-294-2004

Dr. Nancy A. Mathiowetz, University of Wisconsin-Milwaukee, 414-229-2216

Rob Andrews, Fisheries Biologist, NOAA Fisheries Service, Office of Science and Technology, 301-427-8105 is the point-of-contact for the Agency.

References

Andrews, W.R., J.M. Brick, N.M. Mathiowetz, and L. Stokes (2010). Pilot Test of a Dual Frame Two-Phase Mail Survey of Anglers in North Carolina. Retrieved from http://www.countmyfish.noaa.gov/projects/downloads/Final_Report%20NC%202009%20Dual%20Frame%20Two%20Phase%20Experiment.pdf.

Brick. J.M., W.R. Andrews, and N.M. Mathiowetz (2012a). A Comparison of Recreational Fishing Effort Survey Designs. Retrieved from https://www.st.nmfs.noaa.gov/mdms/doc/08A_Comparison_of_Fishing_Effort_Surveys_Report_FINAL.pdf.

Brick, J.M., W.R. Andrews, P.D. Brick, H. King, and N.M. Mathiowetz (2012b). Methods for Improving Response Rates in Two-Phase Mail Surveys. Survey Practice 5(4).

Dillman, D.A., J.D. Smyth, and L.M. Christian (2009). Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method. New York: Wiley and Sons.

Lohr, S. (2009). Multiple Frame Surveys. Chapter 4 in Pfeffermann, D. (Ed.) Handbook of Statistics: Sample Surveys Design, Methods and Applications (vol. 29A). Elsevier, Amsterdam.

1 Florida is not stratified due to the relatively high rate of fishing across the state.

2 Total fishing effort includes fishing by both resident (RAS) and nonresident anglers (NAS).

3 Estimated number of households in the matched stratum is based upon the number of unique addresses in state databases of licensed saltwater anglers as of 8/29/2012. Estimated number of households in the unmatched stratum is the difference between the estimated number of total occupied housing units (Census 2010) and the number of unique addresses in the state license databases.

4 Estimated amount of ABS sample required to achieve final sampling targets for the matched strata.

5 Final ABS sample sizes after subsampling from the unmatched strata. All matched addresses are retained in final sample.

6 Approximately 10% of addresses will be returned by USPS as undeliverable, reducing the total sample for each wave to 16,804.

7 Response rates estimated from previous MRIP pilot studies.

8 Based upon participation estimates from the Marine Recreational Fisheries Statistics Survey

9 Estimated from previous MRIP pilot studies.

10 Sample sizes have been adjusted to account for an estimated 10% ineligibility rate.

11 The detectable difference is the difference between experimental treatments.

9