Nonresponse follow-up

Marine Recreational Information Program

Fishing Effort Survey

Nonresponse Follow-up Study

9/26/2013

The MRIP Fishing Effort Survey (MFES) was implemented in Massachusetts, New York, North Carolina and Florida in October, 2012 to test a revised data collection design for monitoring marine recreational fishing effort. The survey, which collects information for two-month reference waves, included a follow-up study to assess nonresponse bias in the MFES. We also assessed nonresponse bias by comparing survey measures between early and late responders. Details of these assessments are provided below.

Nonresponse Follow-up Study

Each wave, 400 total nonrespondents, 320 from the Resident Angler Survey (RAS) and 80 from the Non-Resident Angler Survey (NAS), were sampled for the Non-Response Follow-Up study (NRFU). Data collection for the study was initiated six weeks after the final contact for the MFES with the delivery of an advanced letter via regular first-class mail. Five days later, a survey packet, including a cover letter, questionnaire, post-paid return envelope and a $5.00 cash incentive was delivered via FedEx. A thank you/reminder postcard was delivered eight days after the FedEx. The NRFU survey instruments were identical to the instruments used for the MFES. To date, four waves of the NRFU have been completed (Wave 5, 2012 – Wave 2, 2013).

Table 1 provides the initial sample sizes, number of completed interviews and response rates for the NRFU. Overall, 474 nonresponse interviews were completed for the RAS and 124 for NAS, resulting in unweighted response rates (AAPOR RR1) of 37% and 38.8% for the respective samples.

Table 1. Sample sizes, completed interviews and response rates by wave for the RAS and the NAS.

	Resident Angler Survey			Non-Resident Angler Survey
State	Sample Size	Complete Interviews	Response Rate	Sample Size	Complete Interviews	Response Rate
	(n)	(n)	(%)	(n)	(n)	(%)
MA	293	119	40.6%	80	35	43.8%
NY	270	88	32.6%	80	26	32.5%
NC	359	149	41.5%	80	35	43.8%
FL	358	118	33.0%	80	28	35.0%
Overall	1280	474	37.0%	320	124	38.8%

We assessed nonresponse bias by comparing estimated fishing prevalence (percent of households that reported fishing during the wave) between the initial MFES and NRFU samples. Differences between MFES and NRFU estimates would suggest that MFES and NRFU samples are different with respect to recreational fishing activity, resulting in biased MFES estimates.

Table 2 shows that differences in estimated fishing prevalence between initial samples and NRFU samples are neither significant nor systematic for either the RAS or NAS, demonstrating that MFES respondents and nonrespondents are not significantly different with respect to saltwater fishing activity. This suggests that nonresponse is not a significant source of bias in the MFES.

Table 2. Estimated fishing prevalence for the full sample and nonresponse follow-up sample for the (a) Resident Angler Survey and the (b) Non-Resident Angler Survey.

(a)

	Estimated Prevalence
State	Full Sample (RAS)		NRFU Sample (RAS)		p-value
	(%)	(n)	(%)	(n)
MA	9.4%	6424	8.2%	119	0.667
NY	7.2%	4864	13.9%	88	0.230
NC	10.5%	7921	7.1%	149	0.100
FL	20.9%	6767	23.3%	118	0.682

(b)

	Estimated Prevalence
State	Full Sample (NAS)		NRFU Sample (NAS)		p-value
	(%)	(n)	(%)	(n)
MA	55.3%	745	63.3%	35	0.322
NY	43.5%	649	30.1%	26	0.342
NC	29.5%	609	44.2%	35	0.472
FL	43.5%	589	37.1%	28	0.418

Notes – Comparisons between full sample data and NRFU include four waves of data collection, wave 5, 2012 – wave 2, 2013.

Early vs. Late Responders

We also assessed nonresponse bias by comparing final prevalence estimates, generated from complete sample data¹, to preliminary prevalence estimates, derived from survey data collected within three weeks of the conclusion of each wave.

Table 3 shows that there are no significant differences between preliminary and final estimates for either the RAS or NAS, verifying the results from the NRFU.

Table 3. Final and preliminary fishing prevalence estimates for the (a) Resident Angler Survey and the (b) Non-Resident Angler Survey.

(a)

	Estimated Fishing Prevalence (RAS)
State	Final Estimate		Preliminary Estimate		p-value
	(%)	(n)	(%)	(n)
MA	10.1%	7982	9.8%	5811	0.610
NY	8.1%	6183	7.9%	4532	0.689
NC	11.1%	9839	11.0%	7413	0.944
FL	22.0%	8342	22.6%	6197	0.384

(b)

	Estimated Fishing Prevalence (NAS)
State	Final Estimate		Preliminary Estimate		p-value
	(%)	(n)	(%)	(n)
MA	47.6%	905	47.4%	699	0.944
NY	32.4%	802	32.8%	615	0.920
NC	47.0%	760	45.8%	580	0.667
FL	52.3%	723	50.9%	526	0.631

Notes – Comparisons between preliminary and final estimates include 5 waves of data collection, wave 5, 2012 – wave 3, 2013.

Nonresponse will result in biased estimates if respondents and nonrespondents are different with respect to survey measures. In the MRIP Fishing Effort Survey, estimates of fishing prevalence will be biased if respondents are more or less likely to participate in recreational fishing than nonrespondents. We tested for nonresponse bias in the MFES by comparing preliminary and final survey data and by conducting a nonresponse follow-up study. Neither assessment demonstrated that MFES estimates are biased as a result of nonresponse.

1 Complete sample data includes surveys returned within 12 weeks of the end of the reference wave.