1018-SSB refuge visitor survey

Supporting Statement B for

Paperwork Reduction Act Submission

OMB Control Number 1018-XXXX

Survey of National Wildlife Refuge Visitors

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Respondent Universe

The respondent universe for this information collection is adults (18 years of age or older) who visit a national wildlife refuge having at least 25,000 annual visitors as per the Fish and Wildlife Service (FWS, we) 2008 Refuge Annual Performance Plan (RAPP) data. There are 234 refuges that meet this criterion. The minimum criterion of 25,000 annual visitors represents the median visitation for each refuge in the RAPP database. It is also the minimum visitation number at which USGS believes they can effectively survey a sufficient number of visitors during the sampling periods (e.g., these refuges have visitor centers and have concentrated visitor use in identified areas that can be sampled). Sampling efforts will occur biennially (every other year), pending funding for out years.

National Sampling effort: USGS will survey visitors biennially at 50 randomly selected refuges of the 234 eligible refuges across the country. For each biennial sampling effort, new refuges will be selected. USGS based its decision to sample 50 refuges on the minimum number of refuges needed to ensure the sample represents different types of refuges within the National Wildlife Refuge System (NWRS) and the budget and time allowances for the study.

The table below summarizes the respondent universe and sample for the National Sampling Effort.

Respondent universe and sample for National Sampling Effort.

# Refuges with >25,000 visitors (universe)	Total # visitors in universe	Total # visitors to be sampled	Total # visitors sampled at each of 50 refuges
234	35,928,900	10,000	200

Individual Refuge Sampling effort: Refuges not selected to be in the national sample may desire to conduct this information collection if they have funding and personnel available to do so. We anticipate that up to 25 additional refuges may participate. These refuges will not be included in the national sample because they will not be chosen randomly. Only individual refuges that have the financial and personnel resources will choose to survey their visitors. USGS will work directly with these additional refuges to implement the survey. The additional 25 refuges will follow the sampling protocol outlined in this supporting statement and use the survey instrument being approved in this information collection.

Expected Response Rate

USGS anticipates a response rate of 80 percent by providing a well-designed survey, having a team of trained surveyors at each refuge to contact visitors, and offering a token incentive. An 80-percent response rate will result in approximately 200 responses per refuge. For each visitor who is asked to participate, surveyors will record the date and acceptance or refusal to participate. When reporting response rate, we will report the refusal rate (% of people contacted who did not agree to participate) as well as the response rate (% of people agreeing to participate who complete the survey). This surveying effort will produce sample sizes that are considered robust at both the national and individual refuge level with acceptable margins of error of ≤ ±7% and ≤ ± 2% at the 95% confidence level, respectively.

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

USGS will conduct this surveying effort biennially, with visitors to 75 refuges every other year (50 for National sampling effort and 25 for the Individual Refuge sampling effort; see Item 1 above). Onsite, surveyors will ask 250 visitors at each refuge to participate in the survey. This number will result in an effective sample size so that results can be generalized at both the national level and the individual refuge level. Based on similar studies conducted by the USGS, an 80 percent response rate is expected. For each biennial sampling effort, this will result in 200 responses per refuge and 10,000 responses for the national sample.

Surveyors will intercept visitors at low, medium, and high visitation time periods on approximately 15 sampling days. This may include up to two visitor seasons, representing both consumptive and nonconsumptive uses at each refuge. For purposes of this study, consumptive use represents hunting and fishing activities, whereas nonconsumptive use represents wildlife observation, photography, environmental education and interpretation. For example, a sampling timeframe for one refuge may consist of 6 days during a 2-week period when higher or more concentrated use occurs (e.g., 2 weeks in November during deer hunting season, representing consumptive users) and another 9 days during a 3-week period when lower or more dispersed use occurs (e.g., 3 weeks in May, representing bird watchers and other nonconsumptive users).

While onsite, surveyors will ask visitors to participate in the survey. Surveyors will tell visitors that their participation is voluntary. Surveyors will complete a record log for every visitor contact, noting date, acceptance or refusal. Surveyors will record names and addresses, and their preference for completing the survey (online or paper) of willing participants on a data sheet and USGS will enter these names into a database. Surveyors will thank participants for their cooperation and give them a token incentive.

USGS will train surveyors via written correspondence and web training on proper techniques for intercepting visitors to ensure a random sample. Surveyors will follow a systematic sampling method involving a random start and then proceed with the selection of every k^th visitor from then onwards (for example, contacting every 10^th visitor). The surveyors will ensure that visitation is not overrepresented by certain groups such as bus tour groups. Additionally, surveyors will be onsite on randomly assigned weekdays (Monday – Thursday) and weekends (Friday – Sunday), with two weekdays and one weekend day chosen for each week. The same procedure will be followed for times of day, which is divided into morning (8:00 a.m. to 12:00 p.m.), midday (12:00 p.m. to 4:00 p.m.), and evening (4:00 p.m. to 8:00 p.m.) shifts. Visitor intercept will occur within the designated timeframe following the described technique until the target of 250 visitors for each refuge is met. If necessary, additional days will be added to the sampling timeframe in order to meet this target.

Surveyors will submit record sheets to USGS on a daily basis. In order to ensure quality control with sampling, USGS will check in with surveyors after the first few days of sampling and review record sheets as they are received.

Following Dillman’s Tailored Design Method (2007), USGS will contact visitors who agreed to participate via mail approximately 1 week after they were contacted onsite. They will have the option to fill out a paper survey (to be sent to them and mailed back) or fill out the survey online. For visitors who do not complete the survey online within 1 week of receiving the postcard invitation, USGS will mail a paper survey to them the following week (2 weeks from initial contact). The survey package will contain a postage-paid, business-reply mail or self-addressed stamped envelope for returning the completed survey. The questionnaire will be 11 pages in an easy-to-read format. USGS will mail a follow-up reminder postcard to visitors who do not respond within 3 weeks of the initial onsite contact. USGS will mail another full packet (cover letter, survey, return envelope) within 5 weeks of the initial onsite contact.

Mailing Schedule Following Dillman’s Tailored Design Method (2007).

Initial onsite contact.

1 week from onsite contact – Postcard with web link for online survey and reminder of agreement to participate.

2 weeks from onsite contact – Full packet to those who have not completed the online survey.

3 weeks from onsite contact – Reminder postcard with web link.

5 weeks from onsite contact – 2^nd full packet to all who have not responded.

The sample size for the national sampling effort will result a ≤+/- 2% margin of error (at the 95% confidence level). The sample size for each individual refuge will result in a ≤+/- 7% margin of error (at the 95% confidence level). This degree of accuracy will meet the needs of this study.

Unusual problems are not anticipated and periodic data collection in order to reduce burden will not occur.

3. Describe methods to maximize response rates and to deal with issues of nonresponse. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

USGS will take several steps to maximize response rate and ensure an accurate and reliable sample both at the national sample level and the individual refuge level.

Multiple Options for Completing the Survey

Respondents can fill out an online or paper survey. It is becoming increasingly accepted to use an online survey as an alternative to other survey modes such as mail or telephone (Couper, 2000).¹ Researchers often use online surveys to decrease costs, increase the speed of data collection, increase response rates by providing additional modes for response, and decrease the amount of non-response error (Dillman, 2000; Schaefer and Dillman, 1998)².

The Tailored Design Method (Dillman, 2007) to Ensure High Response Rate and Representative Sample

USGS will follow The Tailored Design Method for mail and internet surveys to help ensure a high response rate and representative sample. Surveyors will record names and addresses of visitors agreeing to participate in the survey on a data sheet onsite and USGS will enter them into a database. USGS will use this contact information for all mailings, as outlined by the Tailored Design Method (Dillman, 2007). See item 2 above.

Token Incentive

Surveyors will provide a token incentive onsite when visitors are initially contacted. Dillman (2007) indicates that there is an improvement in response rate for mail and online surveys when incentives are included. Results of a study that implemented Dillman’s method, including a material incentive, produced an increase in response rate of 8 percent over those respondents who did not receive an incentive (Boynton, 1995)³. Therefore, we believe the inclusion of a de minimis incentive will have a similar effect for this survey. The incentive will be an FWS magnet, notepad, or other item available to the agency displaying an FWS or a National Wildlife Refuge visual identifier. These items are routinely used as a part of FWS outreach activities and are commonly given to visitors. There is no increased burden on refuge personnel or the Government by providing this type of incentive.

Special Training for Personnel Who Will Intercept Visitors Onsite

USGS will train surveyors to intercept visitors onsite. This will include training on the protocol to increase the effectiveness of communications with visitors. Using a well-trained team will help decrease nonresponse error by effectively communicating the survey’s purpose and importance and the FWS’s desire for input from all visitors. See item 2.

Addressing Potential Nonresponse Bias

USGS will conduct a final followup with all nonrespondents to test for nonresponse bias. This nonresponse mail-back survey will contain a small subset of six key questions from the original survey designed to be completed in 5 minutes. For both the National Sampling Effort and the Individual Refuge Sampling Effort, USGS will send this nonresponse survey via mail to all nonrespondents in anticipation of a completed sample of at least 15 responses per refuge, assuming a 30% response rate. This response rate is based on previous research by USGS scientists (Sexton, Stewart, and Koontz, 2008).

USGS will statistically compare results from the nonresponse follow-up questionnaire to the initial respondent sample to identify potential nonresponse bias. USGS will report any differences found in the final report.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

USGS pretested the survey with seven National Wildlife Refuge Friends’ Group members, each from a different FWS region. For the pretest, USGS mailed each participant a hard copy version of the survey and asked them to review and provide comments concerning the overall structure, sequence, and clarity of questions. Pretesters estimated the time to complete the survey as 25 minutes. Comments and actions taken to address the comments are summarized below.

Section	Pretester Comments	Actions taken to address Pretester Comments
1-6	Reword and consider reordering some of the questions for clarity throughout the survey.	Made adjustments for clarity and ordering of items.
1	Section 1 is confusing when questions jump back and forth between the most recent visit and trips within the last 12 months.	Reordered questions in Section 1 to reduce confusion.
1	Uncertain how to respond to questions about Visitor Centers if the refuge does not have one.	No change. According to the 2008 RAPP data, approximately 80% of refuges with a minimum criterion of 25,000 annual visitation have a visitor center.
2	Add lines immediately following the importance/satisfaction items so respondents can comment.	Added lines.
2	There is room for clarity regarding transportation-related items. Should also ask about directional signs on trails as well as signs on roads.	Clarified that certain questions specifically ask about roads on the refuge. Added a question regarding direction signs on trails.
3	Section 3 is burdensome when asking to recall the amount spent in the local area and elsewhere.	Deleted some questions or combined them with other categories to reduce the burden.
3	“Local area” needs to be clarified; it may be as far away as 50 miles in some rural areas.	Defined local area as within 50 miles of the refuge.
3	Uncertain how to answer question regarding appropriateness of fee if a pass was used to access the refuge.	Included a response to indicate use of a pass.
4	Add lines immediately following the importance/satisfaction items so respondents can comment.	Added lines.
5	Include an “opt out” for the climate change questions for individuals who don’t have enough information to respond adequately.	No change. An “opt out” is likely to encourage people to skip a section that is important to FWS for strategic planning purposes.
6	Some demographic questions (e.g., income, ethnicity) are of questionable utility.	No change. Demographic questions are important for identifying diversity of refuge visitors.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Statistical consultants:

Dr. John B. Loomis

Colorado State University

Department of Agriculture and Resource Economics

Professor, Agriculture and Resource Economics

B310 Andrew G. Clark

Office Phone: (970) 491-2485

Email: [email protected]

Lynne Koontz and Natalie Sexton are the principal investigators for the Survey of National Wildlife Refuge Visitors. They are trained in univariate and multivariate statistical analysis methods; and are responsible for designing the survey questionnaire and sampling procedure and for collecting and reporting resulting information.

Collection and analysis agency:

Policy Analysis and Science Assistance Branch

Fort Collins Science Center

U.S. Geological Survey

2150 Centre Ave, Building C

Fort Collins, CO

Phone: 970-226-9313

1 Couper, M. P. 2000. Web surveys: a review of issues and approaches. Public Opinion Quarterly, 64(4), 464-494.

2 Dillman, D.A., 2007, Mail and internet surveys-The tailored design method, 2nd ed: Hoboken, NJ, John Wiley & Sons, Inc.

Schaefer, D., and Dillman, D.A. 1998. Development of a standard e-mail methodology: Results of an experiment. Public Opinion Quarterly, 62(378-397).

3 Boynton, M., 1995. Results of a mail survey of department of licensing customers on service satisfaction and preference. Data Report No. 9559.

6