NORE_ssfmt_final_revised_11_16_11_part_b_v2_clean

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

Potential Respondent Universe

The potential respondent universe includes the U.S. civilian, non-institutionalized population, aged 18 years and older, who participate in ocean recreation. According to results of the National Survey on Recreation and the Environment (NSRE), conducted by the U.S. Forest Service and NOAA’s National Ocean Service in 1999-2000, about 43% of U.S. households (i.e., 44.4 million households) participated in ocean recreation during the year. This participation rate seems somewhat high, particularly because the NSRE survey appeared to sample only coastal states. A major purpose of the proposed data collection is to provide participation rate estimates by region that are suited to the breadth of recreation covered by this survey and can serve as a basis for expanding activity and expenditure estimates from the sample to the population. The proposed data collection plans to sample both coastal and non-coastal (inland) states.

Sampling Frame

The sample frame for this study is a research panel recruited and maintained by Knowledge Networks (KN) that includes approximately 32,804 U.S. households (KN, personal communication, 6/9/10). This sample frame includes cell-phone only households, Spanish-speaking households, and households who did not previously have internet access. A comparison of KN’s research panel membership relative to demographic characteristics (i.e., benchmarks) from the U.S. Census is shown in Table B.

Table B. Characteristics of samples drawn from KN’s research panel
compared to Census demographic benchmarks

(Knowledge Networks 2010a)

Demographic characteristic		Proportion of research panel members*	Proportion of U.S. population**
Gender	Male	47.3	48.3
	Female	52.7	51.7
Age	18-24	10.4	12.6
	25-34	17.7	17.8
	35-44	19.1	18.1
	45-54	18.9	19.6
	55-64	18.3	15.3
	65 and over	15.7	16.7
Race	White	79.5	81.2
	Black (African American)	12.4	11.8
	American Indian, Alaskan Native	1.1	0.8
	Asian	1.8	4.6
	Hawaii or Pacific Islander	0.4	0.3
	2 or more races	4.7	1.3
Hispanic Ethnicity	Hispanic	14.0	13.8
	Non-Hispanic	86.0	86.2
Employment Status	In the labor force	67.4	67.6
	Not in the labor force	32.6	32.4
Marital Status	Married	53.4	55.5
	Not married	46.6	44.5
Housing Ownership	Own	72.9	71.0
	Rent or other	27.1	29.0

* Weighted percent of KN’s adult panel members; weighted for non-response and non-coverage.
** Percent of the U.S. civilian, noninstitutionalized population, ages 18 years and over (June 2009 Current Population Survey (CPS)).

Demographic data related to panel members sampled for this study, whether or not they choose to participate in this study, will be made available to NMFS. This information will be used to evaluate the representativeness of each sample selected for the pretest and for each wave of the survey. Each sample that is drawn will be compared with U.S. Census benchmarks such as age, gender, race, ethnicity, educational attainment, employment status, and household income. This demographic data is collected by KN as part of their initial panel recruitment process and is collected independent of this proposed data collection. Demographic data for panel members who are selected to participate in the pretest and/or the actual survey but do not participate (i.e., non-respondents) will be provided to NMFS. This will allow NMFS to evaluate non-response bias in terms of possible systematic differences between panel members who are selected for this study and participate and those who are selected but do not participate.

Recruitment of Knowledge Networks’ research panel (sample frame)

Households are recruited to become part of KN’s research panel using probability-based random sampling of residential addresses, using the U.S. Postal Service’s Delivery Sequence File as a sampling frame. This address-based sampling (ABS) method has reportedly increased the demographic representativeness of KN’s research panel, particularly for populations that were difficult to recruit using random digit dial (RDD) methods (Knowledge Networks 2010). ABS sampling to recruit panel members has been used by KN since 2009. Prior to 2009, KN used RDD landline telephone recruiting methods. Table C provides a comparison of difficult-to-recruit population proportions in KN’s research panel compared with proportions estimated by the U.S. Census.

Table C. Characteristics of samples drawn from KN’s research panel

compared to Census demographic benchmarks

(Knowledge Networks 2010b)

Difficult-to-recruit population	RDD*	ABS*	Census estimates**
Ages 18-24	6.4	9.4	12.7
Ages 25-34	13.5	18.9	17.9
Racial minority	20.0	24.0	18.7
Hispanic ethnicity	7.7	11.2	13.5
No high school diploma	6.0	8.5	14.0
High school diploma	18.4	21.5	31.7
Family income, > $10,000	3.9	6.1	5.9
Family income, $10,000 - $24,000	9.7	14.3	15.6

* Unweighted percent of KN’s adult panel members.

** Percent of the U.S. civilian, noninstitutionalized population, ages 18 years and over.

In addition, the recruitment rate for households participating on KN’s research panel has increased using ABS sampling methods. Of the eligible households that were contacted by KN to become part of their research panel using ABS methods, 14% positively responded to KN’s mail invitation, indicating interest in the panel. Of the households that indicated interest, 75% become participating panel members. According to KN, these recruitment rates are higher than the 50% achieved using RDD methods (Knowledge Networks 2010). Lastly, the increasing number of cell phone- only households has increased coverage error associated with RDD sampling of telephone landlines, as an adequate and reliable sample frame of cell phone numbers is not available for recruiting those households. ABS allows sampling of these households, increasing the representativeness of KN’s research panel.

Sampling or Other Respondent Selection Methods

Stratified random sampling of the sample frame will occur with replacement for each wave of the survey period – six waves over 12 months. Households that have participated in previous waves are eligible to participate in all subsequent waves.

The sample itself is stratified by geographic region. Five of the six geographic strata approximate the jurisdictional boundaries of the regional fishery management councils (FMCs) established by the Magnuson-Stevens Fishery Conservation and Management Act (MSA). The sixth region includes inland states not covered by the MSA and is included here to account for participation in coastal recreation by inland households. The delineation of the five coastal regions used in this survey is intended to facilitate comparison with the 2011 National Marine Recreational Fishing Expenditure Survey (NMRFES), which is expected to provide regional estimates that follow FMC boundaries. Table D describes the regions covered by this data collection and their correspondence to the FMCs.

Table D. Comparison of strata used for this data collection relative to FMC regions

Stratum or region	FMC region
Pacific	Pacific
	Western Pacific (HI)
	North Pacific (AK)
New England	New England
Mid-Atlantic	Mid-Atlantic
South Atlantic	South Atlantic
Gulf of Mexico	Gulf of Mexico
--	Caribbean*
Non-coastal states**	--

*Includes Commonwealth of Puerto Rico and U.S. Virgin Islands, which are outside the purview of this survey.

**Non-coastal states are generally not within the geographic research and management

purview of NMFS.

Table E describes the regions of interest, the states included in each region, the number of households in KN’s sample frame that reside in each region, and the proportion of households in each region relative to proportions found in the U.S. Census (Census 2000). The regional distribution of KN’s frame closely follows the Census distribution.

Table E. Definition of regions (strata) used for this study

Stratum or Region*	States, including District of Columbia	# of households, KN research panel**	% of households, KN research panel	% of households, 2000 U.S. Census
Pacific	AK, WA, OR, CA, HI (5)	5,708	17.4%	14.9%
New England	ME, NH, VT, MA, RI, CT (6)	1,567	4.8%	5.1%
Mid-Atlantic	NY, NJ, PA, DE, MD, VA, DC (7)	5,481	16.7%	19.1%
South Atlantic	NC, SC, GA, FL (4)	3,771	11.4%	13.3%
Gulf of Mexico	AL, MS, LA, TX (4)	3,199	9.8%	11.2%
Inland states	ID, NV, UT, AZ, NM, MT, WY, CO, ND, SD, NE, KS, OK, IA, MO, AR, TN, KY, WV, MN, WI, IL, MI, IN, OH (25)	13,078	39.9%	36.4%
Total	51	32,804	100%	100%

*Regions defined for the Regional Fishery Management Councils by the Magnuson-Stevens Fishery Conservation and Management Act (MSA) were considered when defining these strata. However, due to the limited number of households in regions such as the North Pacific (AK) and Western Pacific (HI), these regions were combined with the Pacific Region. Inland states are not a region defined by the MSA but are included in this study.

** Estimates based on the membership and size of KN’s research panel as of 6/9/10.

Expected Survey Completion Rate and Comparison with Other Surveys

An 80% survey completion rate was suggested by KN as a conservative estimate (KN, personal communication, 10/25/100). This completion rate is based on their experience with a variety of web-based surveys. However, a more conservative completion rate of 70% was assumed for purposes of the proposed data collection.

Table F shows response rates for other national recreation surveys conducted by Federal agencies that did not meet the needs of the proposed data collection (see Question A4) but nevertheless collected at least some similar data (e.g., participation, recreation days, expenditures and/or demographics). Response rates reported here were taken from the literature or from agency websites. With regard to the 2000 NSRE, a marine recreation module was added to collect detailed information about marine recreation activities (rather than aquatic activities in general); however, the response rate for that module was not available. Instead Table F provides the response rate for the lifestyle module in 2000 (51-55% for their telephone mode, according to Green et al. 2006) and the average 2002-2008 response rate for all modules (19-20% for their telephone mode, according to Green et al. 2008).

Table F. Comparison of survey completion rates for national recreation surveys

Survey	Year	Federal agency	Sample frame	Mode of data collection	Response rate
Saltwater Angler Expenditure Surveys	2006	NOAA/NMFS	Lists created from anglers intercepted at public access sites	In-person and telephone interviews, mail survey	40% (mail), 62% (in-person)
National Survey of Fishing, Hunting, and Wildlife-Associated Recreation	2006	USFWS	Census Bureau’s master address file (MAF) and the Current Population Survey (CPS)	In-person and telephone interviews	90%
National Survey of Recreation and the Environment (all survey modules)	2002-2008	USFS, NOAA/NOS	Phone list provided by Survey Sampling, Inc.	Telephone interviews	19-20%
National Survey of Recreation and the Environment (“lifestyle” module only)	2000	USFS, NOAA/NOS	Phone list provided by Survey Sampling, Inc.	Telephone interviews	51-55%

Number of Entities to be Sampled

In question 12 of Part A of this Supporting Statement, 58,939 completed responses were estimated for this data collection for all six waves. A sample size of 84,198 would be needed to yield these completed responses, assuming a 70% survey completion rate.

For allocating this sample across regions and waves, it was considered appropriate to take the total sample for each region and distribute it evenly to each wave. This was considered appropriate because wave by wave participation rates for all eight ocean recreation activity categories is not known.

We considered following the sampling protocol used by the 2006 and 2011 NMRFES. That is, the survey is not implemented in Wave 1 (January and February) in New England and the Mid-Atlantic, and in Wave 6, Maine is not sampled. This protocol was followed by the NMRFES due to historically low participation rates in marine recreational fishing in those waves and geographic areas, and due to limitations in funding. However, it is not clear whether participation in other ocean recreation activities will follow a similar pattern to marine recreational fishing. Therefore, sampling in each region and in each wave seemed to be the best option for understanding the level of participation in these activities throughout the year. Distributing this sample evenly across waves seemed appropriate because relative, wave by wave participation rates are not known.

The expected number of completed responses, the sample sizes needed to yield these responses, and the allocation of sample in each region is shown in Table G below. In addition, there were concerns regarding KN’s panel size in each region and whether the estimated sample sizes in each wave would exceed the panel’s size. For each wave and region, between 29% and 65% of KN’s panel will be sampled in each wave. This information is also shown in Table G.

Table G. Sample sizes for each region in each wave relative to KN’s panel size in each region.

Region	KN panel size1	Total completed responses, all waves combined	Sample size, all waves combined (assuming 70% completion)	Sample size, each wave	% of KN panel sampled, each wave
Pacific	5,708	12,803	18,289	3,048	53%
New England	1,567	3,245	4,635	773	49%
Mid-Atlantic	5,481	11,010	15,729	2,621	48%
South Atlantic	3,771	10,220	14,600	2,433	65%
Gulf of Mexico	3,199	5,931	8,473	1,412	44%
Inland states	13,078	15,730	22,472	3,745	29%
U.S. total	32,804	58,939	84,198	14,033	43%

¹ Knowledge Networks’ panel size estimates provided by KN on 6/9/10.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Sampling methodology

The sampling frame for the proposed data collection is the approximately 32,804 U.S. households who comprise KN’s web-enabled research panel. This sample consists of households randomly drawn from the U.S. population. The sample includes cell-phone only households, households without prior access to the internet at home, and households that are predominantly Spanish-speaking. The sample is being stratified into six geographic regions: Pacific, New England, Mid-Atlantic, South Atlantic, Gulf of Mexico, and inland states – with the intent of producing region-specific estimates of ocean recreational participants and recreational activity.

We are interested in two key variables related to participation in ocean recreation and associated expenditures. First, we are interested in the proportion of the U.S. and regional populations who participate in ocean recreation on an annual basis. Specifically, we are interested in regional and national rates of participation in all ocean activities combined, and national rates of participation in each of the eight ocean recreation categories. Second, we are interested in annual mean expenditures per participant per year for the U.S. for all ocean activities combined and for each ocean recreation category. Regional estimates of mean expenditures should also be possible for all ocean activities combined.

Annual participation in ocean recreation

To obtain estimates of annual participation in ocean recreation activities, questions in the survey will identify survey respondents as a participant in any ocean recreation activity and then in which particular ocean recreation category. The number of respondents in each stratum (region) needs to be sufficiently large to estimate the proportion of ocean recreation participants with reasonable accuracy. The equations [1] – [7] below were used to estimate the sample size (n) needed in each region to estimate the annual ocean recreation participation rate (p) with a maximum margin of error of e = 0.04 with 95% probability. The ocean participation rates reported by the 2000 NSRE were used for p.

n_Pacific = [p_Pac (1-p_Pac)] (z_0.025/e)² = [0.59 0.41] (1.96/0.04)² = 581 [1]

n_{New England} = [p_NE (1-p_NE)] (z_0.025/e)² = [0.65 0.35] (1.96/0.04)² = 546 [2]

n_{Mid Atlantic} = [p_MA (1-p_MA)] (z_0.025/e)² = [0.5 0.5] (1.96/0.04)² = 600 [3]

n_{South Atlantic} = [p_SA (1-p_SA)] (z_0.025/e)² = [0.58 0.42] (1.96/0.04)² = 585 [4]

n_{Gulf of Mexico} = [p_Gulf (1-p_Gulf)] (z_0.025/e)² = [0.37 0.63] (1.96/0.04)² = 560 [5]

n_{Inland states} = [p_Inland (1-p_Inland)] (z_0.025/e)² = [0.24 0.76] (1.96/0.04)² = 438 [6]

n_US = [p_US (1-p_U.S.)] (z_0.025/e)² = [0.43 0.57] (1.96/0.04)² = 588 [7]

The minimum number of responses needed (all waves combined) to calculate annual participation rates in any ocean recreation activity (all activities combined) are: 581 for the Pacific, 546 for New England, 600 for the Mid-Atlantic, 585 for the South Atlantic, 560 for the Gulf of Mexico, 438 for Inland States, and 588 for the U.S. (total across regions). Assuming a 70% rate of completion, the following sample sizes will be needed to yield these responses: 830 for the Pacific, 780 for New England, 857 for the Mid-Atlantic, 836 for the South Atlantic, 800 for the Gulf of Mexico, 626 for Inland States, and 840 for the total U.S. The sample sizes shown in question 5 above are larger than the sample sizes resulting from this precision analysis.

However, the participation rates used in the precision analysis above represent participation in any ocean recreation activity (all activities combined) and do not represent participation rates for the individual ocean recreation categories of interest. Below, Table H shows participation rates from the 2006 NMRFES, 2006 Fish-Hunt, and 2000 NSRE reports, as they relate to the ocean recreation categories for the proposed NORES data collection. The table shows that participation rates for individual activities, such as hunting (1%) and swimming (26%), are below NSRE’s national rates of participation in all ocean activities, used in equations [1] – [7] above.

Table H. Estimated and reported participation rates for ocean recreation categories nationwide.

NORES category	2006 NMRFES	2006 Fish-Hunt1	2000 NSRE2
Recreational fishing	8.3%	2.6%	10.32%
Recreational shellfishing	--	--	--
Hunting waterfowl or other animals	ND	0.77%, includes fresh and saltwater	0.33%
Viewing or photographing the ocean	ND	15%, marine mammals only	9.19%, viewing/photographing scenery 7.17%, bird watching 6.45%, viewing other wildlife
Beachcombing, tidepooling, or collecting items	ND	ND	30.03%, beach visitation 4.5%, visiting watersides besides beaches
Water contact sports	ND	ND	25.53%, swimming 5.07%, snorkeling 1.59%, surfing 1.35%, scuba diving 0.39%, wind surfing
Boating and associated activities	ND	ND	7.11%, motorboating 2.98%, sailing 2.57%, personal watercraft use 1.33%, kayaking 1.15%, water-skiing 1.05%, canoeing 0.53%, rowing
Outdoor activities not involving water contact	ND	ND	30.03%, beach visitation 4.5%, visiting watersides besides beaches

ND = no data for this activity is collected.

¹ Participation rates are not reported by the 2006 Fish-Hunt survey (U.S. DOI 2007). These participation rates are based on total participants reported by the FWS and 2006 U.S. Census population estimates (U.S. Census 2008). The FWS participation rate for hunting waterfowl and other animals includes both fresh and saltwater activities and is likely an overestimate of ocean-related hunting activities. The FWS participation rate for viewing or photographing the ocean includes only marine mammal viewing and photographing activities and is likely an underestimate of ocean-related viewing or photographing activities that would likely include bird and wave watching.

²All marine recreation categories reported by the NSRE were matched with a NORES category. However, NSRE’s “beach visitation” and “visiting watersides besides beaches” categories overlapped with two NORES’ categories and are listed twice.

To estimate sample sizes necessary for estimating participation in activities with lower participation rates, equation [8] below calculated the sample size needed to estimate U.S. participation in hunting waterfowl and other animals at the ocean or coast. A 1% probability of participation was assumed, with a maximum margin of error of e = 0.004 and 95% probability.

n_US = [p_US (1-p_U.S.)] (z_0.025/e)² = [0.01 0.99] (1.96/0.004)² = 2,377 [8]

To yield 2,377 responses, a sample size of 3,396 is needed assuming a 70% survey completion rate. The expected sample size for the U.S., shown in Table G, is adequate for accommodating this estimate.

Additionally, regional participation rates for ocean recreation categories are not known and could be smaller than the national estimates shown in Table H, as well as smaller than the participation rates for all ocean activities combined (proportions used to estimate equations [1] – [7]). Though not a primary goal of this data collection, it may be difficult to estimate regional rates of participation for individual ocean recreation categories if a very small proportion of the population participates.

Annual mean expenditures per participant

To estimate sample sizes needed to calculate mean expenditures in ocean recreation activities per participant per year for each region and nationwide, we looked at expenditure information available for marine recreational fishing reported by the 2006 NMRFES (Gentner and Steinback 2008). Total annual mean expenditures (durable goods, trip expenditures) per angler for residents and nonresidents of coastal states were reported. However, no expenditure information was available for inland states or for the U.S. No expenditure information for other ocean recreation categories were available from the NSRE or Fish-Hunt reports. Thus, expected mean expenditures in activities other than recreational fishing is not know; marine recreational fishing expenditures are considered a proxy for the other ocean recreation categories for the purposes of this precision analysis.

Mean expenditures for each region ranged from $237.62 for the Mid-Atlantic to $1,641.15 for the South Atlantic. Based on the estimates of mean expenditures related to marine recreational fishing, the minimum number of completed responses needed (n) to estimate the mean annual expenditures in an ocean recreation category per participant, with a maximum absolute error of e = 50 with 95% probability, was calculated in equations [9] – [15]. For inland states where marine recreational fishing expenditures were not reported by the 2006 NMRFES, the lowest regional estimate, for New England, was used. For the U.S. estimate of mean expenditures per participant also not reported by the 2006 NMRFES, the mean of the regional mean expenditures was used to calculate standard deviations.

n_Pacific = (z_0.025 /e)² = (1.96 733.89/50)² = 828 [9]

n_{New England} = (z_0.025 /e)² = (1.96 682.22/50)² = 715 [10]

n_{Mid Atlantic} = (z_0.025 /e)² = (1.96 237.62/50)² = 87 [11]

n_{South Atlantic} = (z_0.025 /e)² = (1.96 1,641.15/50)² = 4,139 [12]

n_{Gulf of Mexico} = (z_0.025 /e)² = (1.96 980.89/50)² = 1,478 [13]

n_{Inland states} = (z_0.025 /e)² = (1.96 682.22/50)² = 715 [14]

n_US = (z_0.025 /e)² = (1.96 821.84/50)² = 1,038 [15]

To yield the above completed responses and assuming a survey completion rate of 70%, the following sample sizes would be necessary (all waves combined): 1,183 for the Pacific, 1,021 for New England, 124 for the Mid-Atlantic, 5,913 for the South Atlantic, 2,111 for the Gulf of Mexico, 1,021 for Inland states, and 1,483 for the U.S. The sample sizes shown in Table G are larger than the estimates resulting from this precision analysis and should be adequate for estimating mean expenditures in an ocean activity per participant per year.

Regarding the sample frame and precision requirements

In Part B of the Supporting Statement, the relationship between the minimum sample sizes estimated for this data collection and KN’s panel size was not made very clear. Table G attempts to clarify this relationship and shows that given the minimum sample sizes estimated, between 29% and 65% of KN’s panel would be sampled in each wave of this data collection. Therefore, KN’s panel size should be adequate for meeting the estimated sample sizes necessary for this study.

The current funding available for this data collection will allow for larger sample sizes in each region and wave, relative to the precision analysis in question 7b above. If prevalence estimates in ocean recreation are lower than what was reported by the 2000 NSRE, we believe that national and regional level estimates of all ocean recreation combined and national estimates for each ocean recreation category will still be possible. Estimates of individual ocean recreation categories at the regional level may be difficult if participation levels are very low, as mentioned in question 7b.

As mentioned above, we believe that are sample sizes will be adequate for measuring participation and mean expenditures related to ocean recreation. However, when reporting results from this data collection, all information relevant for interpreting these results will be reported. That is, information such as sample sizes and observations obtained by wave, survey completion rates, nonresponse bias analyses, indications of panel conditioning, and other issues and concerns related to this data collection will be reported alongside estimated participation rates and mean expenditures per participant.

This is the first time that the NMFS has proposed to undertake a data collection of this scope (i.e., collecting participation and expenditure information about ocean recreation activities, in addition to recreational fishing) and mode of data collection (i.e., web-based). Therefore, clearly reporting the results of this research and providing information that aids the interpretation of these results, is essential for helping the NMFS evaluate the feasibility of undertaking this type of research in the future.

Pretest

As indicated in Question A1, a random sample of KN panelists will be drawn to pretest survey protocols and completion rates prior to full survey implementation. The pretest will hopefully occur as early as December 2011, January 2012, or February 2012 in the Pacific or Southeast regions, contingent on OMB approval. The Pacific or Southeast regions will be the focus of this pretest because milder weather is typical and therefore, greater ocean recreation opportunities may be available to allow for a broader range of ocean recreation-specific survey questions to be tested.

We will evaluate whether the survey completion rates of 70%, suggested by KN, are observed. Also, we are looking into the possibility of increasing the size of our pilot test to include more than one region. This would give us a better sense of relative participation and survey completion rates between regions, and how they compare with the NSRE participation and KN suggested survey completion rates.

Table I describes the number of KN panelists needed to obtain 250 completed surveys during the pretest.

Table I. Derivation of sample size needed for pretest in Pacific region

	Sample size
# KN panelists randomly selected for pretest	1,673
# randomly selected panelists who agree to participate in pretest*	1,171
# pretest panelists who recreated in past 12 months**	691
# pretest panelists who recreated in Wave 2***	250

* Assuming 70% survey completion rate.

** Assuming 12-month recreational participation rate of 59% for the Pacific region (Leeworthy and Wiley 2001).

*** Assuming that 36.2% of 12-month recreational participants in the Pacific Region recreate in wave 2 (the most likely wave for the pretest) (Gentner and Steinback 2008).

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Several steps are being taken to maximize response rates and address nonresponse bias.

Maximizing response rates

Developing an appealing and understandable survey instrument is important to achieve high response rates. Experts on survey design and, specifically, experts with experience designing recreational expenditure surveys were consulted and assisted in the design and testing of this survey. Focus groups and one-on-one interviews were conducted and were instructive for ensuring that key concepts and terms were understood, for determining a recall period that facilitates recall without unduly increasing the frequency of data collection, and for evaluating the overall design and format of the survey instrument. Additionally, the one-on-one interviews allowed for finer tuning of the survey instrument to ensure that familiar words were used and terms were adequately defined. These interviews were also helpful for ensuring that the time necessary to complete the survey was not burdensome for the respondent. More detailed information regarding these focus groups and one-on-one interviews is provided in Appendix B.

Specific design issues that were incorporated to increase overall and item response rates included: increasing the number of yes/no questions, where applicable (e.g., Q1 regarding whether or not the respondent participated in an ocean recreation activity within the last 12 months); providing discrete categories for multiple choice questions, where applicable (e.g., Q2 regarding all ocean recreation activities a respondent participated in within the last 12 months); using tables to increase visual interest and provide variety in the format of questions asked in the survey instrument, where applicable (e.g., Q9 regarding boat-related expenditures); and using a web-based mode that incorporates skip patterns so that respondents are not asked to manually navigate to questions based on responses to previous questions. The use of a web-based mode will reduce the time burden on respondents, decrease the likelihood that respondents will respond to questions that they are not eligible to answer, or miss questions that they are eligible to answer.

The implementation protocol that will be employed by Knowledge Networks is based on methods suggested by Dillman et al. (2009):

1. For each wave, a stratified random sample will be drawn, using KN’s web-enabled research panel.

2. Once selected, an advance e-mail will be sent to alert respondents that a survey will be made available to them online in a few days. This e-mail will identify the survey as a NMFS-sponsored study, emphasize the voluntary nature of the survey, and the importance of their participation.

3. An email notification will be sent to let respondents know when the survey becomes available to them. Respondents will again be reminded of who is sponsoring the survey, the voluntary nature of their participation, and the importance of their participation. Each potential respondent is provided a link to the survey from their personalized “home page”, a feature that each KN panel member already has. Surveys are self-administered and accessible any time of day for a two week period.

4. E-mail reminders will be sent to respondents who have not completed this survey during this two week period.

5. A phone reminder will occur approximately three days after the e-mail reminder, personally addressed to the respondent.

As discussed in Question A9, nonsurvey-specific incentives are used by Knowledge Networks in the form of “points”, internet service, and/or computers to access the internet. These incentives are provided for any survey that is completed by a member of KN’s research panel and are not specific to this data collection. That is, for households that were recruited to be part of KN’s research panel but did not previously own a computer and/or have internet access, KN provided this equipment as an incentive to participate on their research panel. When panel members are selected to participate in a survey, they accumulate points for every survey they complete. By providing some households with computers and internet service and points for completing surveys, KN is able to maintain a high degree of panel loyalty and low levels of attrition from their research panel. Survey-specific incentives will not be used for this data collection.

Nonresponse bias

KN will provide NMFS with demographic data for all panel members asked to participate in this survey, regardless of whether or not they choose to participate. These data are collected by KN as part of their initial panel recruitment process independently of this proposed data collection. This demographic information will be used to evaluate the representativeness of the sample that responds to the pretest as well as to each wave of the survey. Each sample will be compared with U.S. Census benchmarks such as age, gender, race, ethnicity, educational attainment, employment status, and household income. To the extent that any particular sample is found to be unrepresentative, Census benchmarks will be used to weight the data to ensure that estimates of the number of participants, recreational activity days, and expenditures derived from the survey are representative of the population.

For the NORES, a random sample of Knowledge Networks’ (KN) research panel is planned. This panel is reported to be representative of the general U.S. population. The ocean recreation participation levels of panel members are not known. For the NMRFES, a random sample of recreational fishermen will be selected from a sample frame(s) of known recreational fishermen. Due to the different composition of the two sample frames, it may be difficult to make inferences about differences noted between respondents from these two surveys.

Additionally, for the 2006 NMRFES, a sample of recreational fishermen who did not respond to the mail survey were contacted by phone. From these telephone interviews, demographic and marine recreational fishing-related participation and effort information were collected. Currently, it is not clear whether a similar follow-up with nonrespondents will be conducted for the 2011 NMRFES. Thus, a demographic comparison of nonrespondents from the NMRFES and nonrespondents from the NORES may not be possible.

Therefore, the following nonresponse bias analyses are proposed. The first is an approach often taken by researchers to test for nonresponse bias. The second is a proposed method for testing for possible panel conditioning as it may relate to nonresponse.

For the NORES data collection, Knowledge Networks (KN) will provide NMFS researchers with demographic information about both respondents and nonrespondents. For this data collection, a respondent is someone who, at minimum, answers yes or no to Q1 (12 month participation in ocean recreation). A nonrespondent is someone who either does not click on the weblink they are e-mailed when invited to take this survey, or who clicks on the weblink but does not continue from the first screen that introduces the survey to answer Q1 on the second screen.

In addition to demographic information, KN will be contracted to follow-up with nonrespondents by phone or e-mail. These nonrespondents will be asked about their ocean recreation activities over the last 12 months (Q1), which activities they participated in (Q2), which state most of these activities occurred in (Q3), and how many days in the last two months they recreated (Q21). They will also be asked why they chose not to take this survey.

Demographic information about respondents and nonrespondents, and ocean recreation participation information about nonrespondents, will allow NMFS researchers to test for nonresponse bias. This can be done by testing the following:

1. Compare demographic characteristics of survey respondents to nonrespondents and evaluate differences between these two groups in terms of characteristics such as age, education, ethnicity, and income;

2. Compare demographic characteristics of respondents to nonrespondents who indicate that they participated in ocean recreation in the past 12 months. That is, respondents who indicate “Yes” on Q1 of the online survey, and nonrespondents who indicate “Yes” in the follow-up interview;

3. Compare demographic characteristics of respondents to nonrespondents who indicate that they did not participate in ocean recreation in the last 12 months. That is, respondents who indicate “No” on Q1 of the online survey, and nonrespondents who indicate “No” in the follow-up interview; and

4. Compare the proportion of ocean recreation participants and nonparticipants in the respondent group, to the proportion of participants and nonparticipants in the nonrespondent group.

For numbers one through three above, we would assume that demographic characteristics between the comparison groups will be similar. For number four above, we would assume that the proportion of ocean recreation participants to nonparticipants will be similar when comparing the respondent and nonrespondent groups.

However, if there are differences between respondent and nonrespondent groups in terms of participation rates, we may question whether panel conditioning is occurring. That is, it is possible that KN’s research panel members “learn” that answering the first few questions of a survey in the negative will not lead to more questions, yet qualifies them for an incentive. That is, KN’s research panel members receive an incentive in the form of points to participate in surveys. They are e-mailed a link to a survey and can choose to participate or not. It is possible that panel members, based on past experience with our survey (we are sampling households with replacement in each wave) or with other surveys, learn that if they answer in the negative for the first few questions, their survey participation ends (i.e., they do not move on to more questions) and they receive their incentive. If this is the case, we might see a higher proportion of nonrespondents who participated in ocean recreation compared with the proportion of participants within the respondent group.

By taking the steps above, we will evaluate the presence of nonresponse bias and the possibility that panel conditioning maybe occurring. The results of these analyses will be reported alongside summary statistics and other information resulting from this proposed data collection.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

As indicated in Question B3, focus groups and cognitive interviews were conducted in the course of survey design. Focus groups were conducted to evaluate the information presented in the survey instrument and the material covered in each focus group varied, depending on feedback received. No more than nine members of the general public were included per focus group.

The survey instrument was further evaluated and revised using input from cognitive interviews (one-on-one interviews). Both self-administered and verbal protocols (talk aloud) interviews were conducted, each followed by debriefing by project team members. This qualitative testing was helpful for ensuring that survey questions and the information presented were easily understood and interpreted by the respondent as intended. No more than nine members of the general public were included in a set of cognitive interviews.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The following individuals were consulted on the statistical aspects of the design:

Rosemary Kosaka

Economist

NOAA Fisheries

Southwest Fisheries Science Center

831-420-3988

[email protected]

Scott Steinback

Economist

NOAA Fisheries

Northeast Fisheries Science Center

508-495-2371

[email protected]

Cindy Thomson

Economist

NOAA Fisheries

Southwest Fisheries Science Center

831-420-3911

[email protected]

Rosemary Kosaka, Scott Steinback, and Cindy Thomson are responsible for analyzing the data.

The following grant administrator and contractor are responsible for survey implementation:

Alex Miller

Economist

Gulf States Marine Fisheries Commission (grantee)

2404 Government St

Ocean Springs, MS 39564

228-875-5912

[email protected]

Bill Murphy

Consolidated Safety Services (contractor)

10301 Democracy Lane, Suite 300

Fairfax, Virginia 22030

703-877-3353

[email protected]

Knowledge Networks has been identified by the GSMFC and CSS as a possible subaward recipient and subcontractor, respectively, responsible for data collection:

Michael Lawrence

Vice President, Research Development

Knowledge Networks

9702 Schmidt Drive

Burke, VA 22015

202-370-6345

[email protected]

Michael Dennis, Ph.D.

Executive Vice President, Government & Academic Research

Knowledge Networks

1350 Willow Road, Suite 102

Menlo Park, CA 94025

650-289-2160

[email protected]

References

California State Parks. 2003. Public Opinions and Attitudes on Outdoor Recreation in California, 2002: An Element of the California Outdoor Recreation Plan. California State Parks: Sacramento, California, 114 p.

Cameron, T.A. and J.R. DeShazo. 2008. "Demand for Health Risk Reductions." ("Flagship" paper for project on valuation of health risk reductions; status: revise-and-resubmit, July 2009). Available at: http://pages.uoregon.edu/cameron/vita/wpabstracts.htm

Cochrane, W.G. 1977. Sampling Techniques, third edition. John Wiley & Sons: New York, NY. 428 p.

Dean Runyan Associates. 2009. Fishing, Hunting, Wildlife Viewing, and Shellfishing in Oregon: 2008 State and County Expenditure Estimates. Prepared for the Oregon Department of Fish and Wildlife and Travel Oregon. Dean Runyan Associates: Portland, Oregon, 68 p.

Dillman, D.A., Smyth, J.D., and L.M. Christian. 2009. Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method, third edition. John Wiley & Sons, Inc.: Hoboken, New Jersey, 499 p.

Gentner, B. and S. Steinback. November 2008. The Economic Contribution of Marine Angler Expenditures in the United States, 2006. U.S. Department of Commerce, NOAA Tech. Memo. NMFS-F/SPO-94, 301 p.

Green et al. Cordell, H.K., Betz, C.J., and C. DiStefano. 2006. Construction and validation of the national survey on recreation and the environment’s lifestyles scale. Journal of Leisure Research, 38(4): 513-535.

Green, G.T., Sharp, J., Cordell, H.K., and C.J. Betz. 2008. Special Report for the Society for American Archeology. Available at: http://www.saa.org/Portals/0/SAA/new/NSREsurvey%20data%20final%20report.pdf

Hansen, J. 2008. “Panel surveys”. Pp. 330-339 in The Sage Handbook of Public Opinion Research, W. Donsbach and M.W. Traugott, eds. Thousand Oaks, California: Sage, 640 p.

Holbrook, A.L., Krosnick, J.A., and A. Pfent. 2008. “The causes and consequences of response rates in surveys by the news media and government contractor survey research firms.” Chapter 23 in Advances in Telephone Survey Methodology, edited by J.M. Lepkowski et al. John Wiley & Sons, Inc.

Knowledge Networks, 2010a. KnowledgePanel: processes and procedures contributing to sample representativeness and tests for self-selection bias. Prepared by J.M. Dennis. Available at: http://www.knowledgenetworks.com/ganp/docs/KnowledgePanelR-Statistical-Methods-Note.pdf

Knowledge Networks. 2010b. Update: address-based sampling nets success for KnowledgePanel recruitment and sample representation. Prepared by C. DiSogra. Available at: http://www.knowledgenetworks.com/accuracy/spring2010/disogra-spring10.html

Leeworthy, V.R. and P.C. Wiley. 2001. National Survey on Recreation and the Environment 2000: Current Participation Patterns in Marine Recreation. U.S. Department of Commerce, 47 p.

Magnuson-Stevens Fishery Conservation and Management Act (P.L. 94-265), as amended through January 12, 2007 (P.L. 109-479). May 2007. Available at: http://www.nmfs.noaa.gov/msa2007/details.html

National Marine Fisheries Service. 2010. Fisheries Economics of the United States, 2008. U.S. Dept. of Commerce, NOAA Tech. Memo. NMFS-F/SPO-109, 177 p. Available at: http://www.st.nmfs.noaa.gov/st5/publication/index.html.

Office of Management and Budget, Office of Information and Regulatory Affairs, Statistical Policy Office. May 1986. Federal Longitudinal Surveys, Statistical Policy Working Paper 13. Subcommittee on Federal Longitudinal Surveys, Federal Committee on Statistical Methodology, 159 p.

Rice, J.A. 2007. Mathematical Statistics and Data Analysis, Third Edition. Thomson Brooks/Cole: Belmont, California.

Singh, R., Petroni, R.J., and T.M. Allen. 1994. Oversampling in Panel Surveys, No. 196. U.S. Dept of Commerce, Bureau of the Census. Presented at the American Statistical Association Meetings, August 1994. Available at: http://www2.census.gov/prod2/sipp/wp/SIPP_WP_196.pdf

Solomon, S., D. Quin, M. Manning, Z. Chen, M. Marquis, K.B. Averyt, M. Tignor, and H.L. Miller, eds. 2007. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA.

U.S. Census Bureau, Population Division. 2008. “Table 1: Annual Estimates of the Resident Population for the United States, Regions, States, and Puerto Rico: April 1, 2000 to July 1, 2008 (NST-EST2008-01).” Available at: http://www.census.gov/popest/states/NST-ann-est2008.html

U.S. Census Bureau. 2010. Table 1, Reported Internet Usage for Households, by Selected Householder Characteristics: 2009. Current Population Survey (CPS), October 2009. Available at: http://www.census.gov/population/www/socdemo/computer/2009.html

U.S. Department of the Interior, U.S. Fish and Wildlife Service and U.S. Department of Commerce, U.S. Census Bureau. 2007. 2006 National Survey of Fishing, Hunting, and Wildlife-Associated Recreation, 164 p.

Yeager, D.S., Krosnick, J.A., Chang, L, Javitz, H.S., Levindusky, M.S., Simpser, A., and R. Wang. 2009. Comparing the accuracy of RDD telephone surveys and internet surveys conducted with probability and non-probability samples. Available at: http://communication.stanford.edu/faculty/krosnick/