PSRS SS 090412 Part B_Rev2

SUPPORTING STATEMENT

PUGET SOUND RECREATIONAL SHELLFISH HARVESTING SURVEY

OMB CONTROL NO. 0648-XXXX

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

Potential Respondent Universe

The population of interest for this survey is Washington State residents who have recreationally harvested shellfish (clams and oysters) in Puget Sound in a preceding 12 month period and who are not members of Puget Sound Indian tribes. (Members of Puget Sound Indian tribes have treaty rights to harvest shellfish in tribal areas and certain non-tribal areas.  Tribal shellfish harvest is almost always commercial or considered ceremonial and subsistence harvest, not recreational.) The respondent universe also does not include out-of-state recreational shellfish harvesters. The decision to exclude out of state license holders from the respondent universe was made to reflect the information we have received from knowledgeable stakeholders.  This information suggests that people don’t travel very far for the sole or primary purpose of recreational shellfish harvesting in Puget Sound.  This is turns suggests that the number of potential respondents not covered by our universe is small.  By limiting our respondent universe to in-state license holders, however, we recognize that any conclusion drawn from the survey data will be representative of only this group of harvesters.  This does not necessarily produce bias in the estimation for this group, but does suggest that estimates such as the total economic value of shellfish harvesting in Puget Sound will be underestimates.

Our estimate of the size of this universe (approximately 300,000 at most) is based on the number of Washington State residents who held a license that allows the holder to recreationally harvest shellfish in Puget Sound in 2012. Licenses are issued by the Washington Department of Fish and Wildlife, who have provided us with data on individuals who held any of the following types of licenses: 1) an annual shellfish/seaweed license (shellfish-only license); or 2) an annual, 1-Day, 2-Day, or 3-Day combination fishing and shellfish license (combination license). About 33% of this universe consists of shellfish-only license holders, while 67% consists of combination license holders.

Sampling and Other Respondent Selection Methods

The sample for the pretest will be randomly drawn from the population described above. The full survey that follows will use a stratified sampling approach, using two strata defined by license type: shellfish-only license holders and combination license holders. We will employ information learned during the pretest regarding the eligible population sizes and cost of survey administration by stratum to develop the appropriate size of each stratum. This two-phase sampling design should serve to lower the variance of the resulting estimates, relative to a random sample or an ad-hoc stratification scheme.

Expected Response Rate

A survey covering recreational shellfish harvesting has not been conducted by NWFSC in Puget Sound nor, to our knowledge, by other parties. The NWFSC has conducted surveys of similar design and length for recreational fishing, however. These surveys also used a telephone and mail approach. We are basing our estimates of response rates in part on our experience with those surveys.

We propose to use a stratified sampling approach, however, because we expect response and other relevant survey rates to differ between the two types of license holders. In general, we expect shellfish-only license holders to be a more receptive population than combination license holders for a survey that focuses only on shellfish harvesting. Specifically, we expect the two groups to differ in the following ways:

• If a phone contact is made, a shellfish-only license holder will be more likely to respond to the telephone survey than a combination license holder

• The eligibility of a license holder for a mail survey (determined by whether the license holder has harvested shellfish in Puget Sound over the previous 12 months) will be higher for a shellfish-only license holder than a combination license holder

• If a mail survey is sent, a shellfish-only license holder will be more likely to return a completed survey than a combination license holder.

As noted above, we will use the results of the pretest essentially to test these hypotheses and refine our estimates of the response and other rates for each stratum.

Because our survey design includes a telephone pre-survey contact to screen potential respondents for the mail survey and other features that create “branches” in the survey administration, we first present a flow diagram that traces the logic of the survey (Figure 1). Table B1 lists the assumed response and other rates for the different branches and endpoints for the survey based on this diagram. Tables B2 and B3 provide a summary of the expected number of people in each stratum at each stage of the survey, as determined by the initial sample sizes (833 for the pretest and 7500 for the final survey) and the response and other rates listed above.

Table B1 Response and other rates for survey
Survey branch	Combination License Holders	Shellfish-only License Holders
1) Percent of pretest sample	67%	33%
1) Percent of full survey (assumed)	50%	50%
2A) Percent of sample with valid phone number	85%	85%
2B) Percent of sample without valid phone number	15%	15%
3A) Percent of sample with valid phone number (2A) where a contact is made but the person is not willing to respond	25%	15%
3B) Percent of sample with valid phone number (2A) where a contact is made and the person is willing to respond	45%	55%
3C) Percent of sample with valid phone number (2A) where no is contact made	30%	30%
4A) Percent of sample where a contact is made and the person is willing to respond (3B) where the person is not eligible for a mail survey (person has not harvested shellfish in Puget Sound in past 12 months)	80%	50%
4B) Percent of sample where a contact is made and the person is willing to respond (3B) where the person is eligible for a mail survey (person has harvested shellfish in Puget Sound in past 12 months)	20%	50%
Telephone survey response rate (contacted by phone, willing to answer telephone pre-contact survey = 3B/[3A+3B])	64%	79%
Mail survey response rate (sent a mail survey, returns a completed survey)	50%	60%

Table B2: Combination License Holders
Survey Stage	Group	Branch %	Number	Next action?	Burden group	Respon- dents
1	Combination License holders (67% of pretest sample, 50% of full survey sample)	--	4,721	Find valid phone number
2	2A) Valid phone number	85%	4,013	Send to stage 3
	2B) No valid phone number	15%	708	Send mail survey
	2R) No valid phone number, returns mail survey	50%	354	Completed mail survey	Mail only	354
	2NR) No valid phone number, does not return mail survey	50%	354	No further action
3	3A) Contact made, not willing to respond to telephone	25%	1,003	No further action
	3B) Contact made, willing to respond to telephone	45%	1,806	Send to stage 4
	3C) No contact made	30%	1,204	Send mail survey
	3R) No contact made, returns mail survey	50%	602	Completed mail survey	Mail only	602
	3NR) No contact made, does not return mail survey	50%	602	No further action
4	4A) Not eligible for mail survey	80%	1,445	No further action	Telephone only	1,445
	4B) Eligible for mail survey	20%	361	Send mail survey
	4R) Eligible for mail survey, returns mail survey	50%	181	Completed mail survey	Mail and telephone	181
	4NR) Eligible for mail survey, does not return mail survey	50%	181	No further action	Telephone only	181
Summary		Number of people contacted by phone (3A+3B)			2,809
		Expected response rate [3A/(3A+3B)]			64%
		Expected number of telephone responses			1,806
		Number of people sent mail survey (2B+3C+4B)			2,274
		Expected response rate (2R+3R+4R)/(2B+3C+4B)			50%
		Expected number of mail responses			1,137

Table B3: Shellfish-only License Holders
Stage	Group	Branch %	Number	Next action?	Burden group	Respon- dents
1	Shellfish-only License holders (33% of pretest sample, 50% of full survey sample)	--	4,444	Find valid phone number
2	2A) Valid phone number	85%	3,777	Send to stage 3
	2B) No valid phone number	15%	667	Send mail survey
	2R) No valid phone number, returns mail survey	60%	400	Completed mail survey	Mail only	400
	2NR) No valid phone number, does not return mail survey	40%	267	No further action
3	3A) Contact made, not willing to respond to telephone	15%	567	No further action
	3B) Contact made, willing to respond to telephone	55%	2,077	Send to stage 4
	3C) No contact made	30%	1,133	Send mail survey
	3R) No contact made, returns mail survey	60%	680	Completed mail survey	Mail only	680
	3NR) No contact made, does not return mail survey	40%	453	No further action
4	4A) Not eligible for mail survey	50%	1,039	No further action	Telephone only	1,039
	4B) Eligible for mail survey	50%	1,039	Send mail survey
	4R) Eligible for mail survey, returns mail survey	60%	623	Completed mail survey	Mail and telephone	623
	4NR) Eligible for mail survey, does not return mail survey	40%	415	No further action	Telephone only	415
Summary		Number of people contacted by phone (3A+3B)			2,644
		Expected response rate [3A/(3A+3B)]			79%
		Expected number of telephone responses			2,077
		Number of people sent mail survey (2B+3C+4B)			2839
		Expected response rate (2R+3R+4R)/(2B+3C+4B)			60%
		Expected number of mail responses			1703

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Stratification and Sample Selection

The sample for the pretest will be drawn randomly from the population of combination and shellfish license holders. As stated in Part B, Question 1, however, there may be significant differences in factors such as eligibility and response rates between the two license types that would affect the unit cost of a completed survey. If the pretest confirms this hypothesis, we will use the estimates of unit costs obtained through the pretest to determine the sample sizes within each license type for the full survey mailing, often referred to as 'optimal' or 'Neyman' stratification (see J. Neyman, “On the two different aspects of the representative method: The method of stratified sampling and the method of purposive selection,” Journal of the Royal Statistical Society, 1934, 97: 558-606; or P.S. Levy and S. Lemeshow, Sampling of Populations: Methods and Applications, 1991, Wiley, New York, pp. 132-36).

Desired Accuracy Needed for the Intended Purpose

Data collected through this survey will be used for the estimation of an economic model intended to support ongoing policy making in Puget Sound. While more accurate data are clearly preferred, standards do not exist regarding the accuracy of data required for estimation of an econometric model. Factors such as the minimization of model specification error also contribute to the quality of the empirical results obtained using survey data. It is not possible to state a level of accuracy that is required for all uses and applications of data collected by this survey..

Desired Precision and Response Rate

As noted above, data collected through this survey will be used for the estimation of an economic model to support ongoing policy making in Puget Sound. In these types of applications, error will arise not only from the representativeness of data used for model development, but also from model specification and estimation. Since it is not possible to completely avoid specification and estimation error in model development, there is good reason to desire a high response rate and degree of accuracy in the data collection process, but it is not possible to specify the precise rate and degree needed.

Survey Fielding

The PSRSHP will follow a modified Dillman Method protocol, which will consist of a telephone pre-survey contact, and up to five mail contacts:

1) Telephone pre-survey contact:

Upon receipt of the sample, the contractor will submit the entire sample to a national reverse directory search to verify telephone numbers, and to fill in any missing or incorrect telephone and address information. All sampled individuals with a valid telephone number will be contacted a minimum of eight times, with attempts rotated through day, evening, and weekend shifts. If a contact is made, the interviewer will administer a pre-survey of five to ten questions.

2) Mail survey:

All individuals contacted successfully through the telephone pre-survey who have harvested shellfish in Puget Sound within the last 12 months will be eligible for the mail survey (respondents will not be asked if they wish to participate in the mail survey).

In addition to individuals identified as eligible for the mail survey through the telephone per-survey, all sampled individuals that were not contacted during the telephone pre-survey will be included in the mail survey sample. This will include individuals for whom a valid phone number could not be found and individuals who had a valid phone number but could not be contacted as part of the telephone pre-survey effort.

All eligible individuals will be mailed a pre-notice letter and subsequently a survey packet, which will consist of a personalized cover letter, a survey booklet, and a business reply envelope. The cover letter will be personalized with the respondent’s name and mailing address and will be dated with the mail out date. In addition, each letter will be printed with a NMFS toll-free contact number, in case respondents have questions or comments. Surveys that are returned with forwarding or address correction information will be remailed, and the contact information for that record will be updated. Subsequent contacts will be made to enhance the response rate (see below for more details).

Expected Dates of Survey Implementation

January – March 2013

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Methods used to maximize response rate

The PSRSHP will follow a modified Dillman Method protocol, which will consist of a telephone pre-survey contact, and up to five mail contacts. This includes the following steps intended to maximize response rates:

1) Seven days after the initial mailing date, the contractor will send a postcard reminder to each participating respondent, whether or not the respondent has returned a completed survey. The postcard will reinforce the importance of the survey; it will thank respondents who have already responded and remind those who did not respond.

2) One week from the postcard mailing, a report will be prepared which lists all respondents scheduled to receive the second survey mailing. The list will include respondents who meet the following three criteria:

1. Respondent was mailed the first survey three weeks before the second mail date

2. Respondent has not returned a completed survey

3. Respondent’s first mailing was not returned as undeliverable.

As with the initial survey mailing, the second packet will be sent via first class mail. The steps involved in this mailing will be the same as the initial survey mailing. However, the cover letter for the second mailing will be different from the initial cover letter.

3) Two weeks from the second mailing, a report will be prepared that lists all respondents scheduled to receive the third survey mailing. The list will include respondents who meet the following three criteria:

1. Respondent was mailed a second survey packet

2. Respondent has not returned a completed survey

3. Respondent’s first or second mailing was not returned as undeliverable

Similar to the initial survey mailing, the third packet will be sent via first class mail. The steps involved in this mailing will be the same as the initial and second survey mailings. However, the cover letter for the third mailing will be different from the initial and second survey cover letters.

Methods used to deal with non-response bias

We will use two sources of information to analyze the possibility of non-response bias. First, the license database that will be used to draw a random sample has information on address and type of license, the latter of which may be a proxy for avidity or general interest in shellfish harvesting. We can compare these for respondents and non-respondents. A second source of information will come from questions asked during the telephone pre-contact survey, which will contain a small number of demographic and other questions that will enable us to make a similar comparison.

If respondents are found to be significantly different than non-respondents on a characteristic that is also contained in the license database or telephone pre-contact survey, weighting adjustments will be used to compensate for unit nonresponse in subsequent analysis of the data. Based on responses to similar surveys we've conducted in the past, we are not expecting significant levels of item nonresponse. If encountered, however, we will consider using imputation or, depending on the severity of the item nonresponse, treat the record as unit nonresponse.

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval.

We will conduct a formal pretest of the survey using all of the protocols that will be used in the final survey. The pretest will consist of 200 completed mail surveys. The purpose of the pretest is to determine whether the survey instrument provides the data needed, as well as to test survey procedures and protocol. If the survey needs revision, we will submit the revised instrument as part of a non-substantive change request.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Mark Plummer (co-P.I.)

Northwest Fisheries Science Center

2725 Montlake Blvd. East

Seattle, WA 98112-2097

Phone: 206-860-3492

Leif Anderson (co-P.I.)

Northwest Fisheries Science Center

2725 Montlake Blvd. East

Seattle, WA 98112-2097

Phone: 206-302-2403

Todd Lee (consultant)

Northwest Fisheries Science Center

2725 Montlake Blvd. East

Seattle, WA 98112-2097

Phone: 206-302-2436

Gilmore Research Group (survey contractor)

2101 4th Avenue

Seattle, WA 98121

Phone: 206-726-5555

2