Supporting_Statement_Part_B

Part B - Supporting Statement for OMB 0596-NEW

Trends in Use and Users in the Boundary Waters Canoe Area Wilderness, Minnesota

June 2007

Supporting Statement for OMB 0596-NEW

Trends in Use and Users in the Boundary Waters Canoe Area Wilderness, Minnesota

June 2007

B. Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sam­pling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corre­sponding sample are to be provided in tabular form for the uni­verse as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The following plan provides the details of sampling for the Boundary Waters Canoe Area Wilderness social science research to be conducted by the U.S. Forest Service Aldo Leopold Wilderness Research Institute, with cooperators at the University of Minnesota and the U of Montana in 2007. This design is based on previous studies conducted in 1969 and 1991, as well as by current knowledge about distribution of recreation use in the Boundary Water Canoe Area Wilderness, state-of-the-art methods, and input from study cooperators.

The population of interest for the trend/change/management study includes current adult visitors (> 15 years old) to the Boundary Water Canoe Area Wilderness during the peak season of May to September 30. Total visitation per year estimated at over 200,000; while modeling of the permit data suggest that at least 130,000 permitted (both allocated and self-issue) day and multi-day visits occur during the peak period. The population of interest in 1969 and 1991 included only overnight visitors during the peak season, while the current study will include all permitted recreation visitors, including self-permitted day users (permits are available for self-issue at launch points for day use, non-motorized). The sample of visitors will be sub-divided according to the type of trip they were on when contacted for this study (either day use or overnight use), and a separate survey instrument will be developed for each of these trip types. Sufficiently large samples of day and overnight users will be required for each of the survey instruments.

The current front-end form could be completed either before or after the trip and includes several questions about trip characteristics and one question about the experience of the individual party members. This form provides limited ability to test for non-response bias, but it is easy to administer and could be answered by one member of the group if other members were unavailable for the interview.

In 1969, visitors were contacted on-site as they finished their Boundary Water Canoe Area Wilderness trip and asked to either complete a questionnaire at that time or provide contact information for later mailing of a questionnaire. In 1991, visitors were contacted on-site as they began their trip and asked to complete a short on-site interview to collect the information on the front-end form for later mailing of a questionnaire.

In 1991, approximately 400 people were intercepted at entry points to the Boundary Water Canoe Area Wilderness and at permit distribution locations. They were asked to provide their contact information for a mail-back questionnaire. Contacts were made on-site at the busiest entry points as visitors began their trips, and low use sites were targeted through the central distribution locations. The sampling was partitioned by sample day, with a different location chosen for each sample day. There were 36 sampling days that were determined according to how they were distributed in 1969 – 18 weekday/ 18 weekend distributed during specific weeks across the peak season. The entry locations were distributed across sampling days to roughly correspond with their estimated distribution of use. This intercept method, using a mail-back questionnaire, obtained a 74 percent response rate. There are two mail back questionnaires – one for overnight visitors and another for day visitors (with only some mention of camping removed).

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the pur­pose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The proposed sampling design for the current study is based on an example laid out in 1991. It also builds on that example to develop a sampling design more representative of the current population. Visitor population estimates by entry point and type of use have been made using self-issue permit data from 2004 and allocated permit data from 2005. This model of the population distribution was used to develop a sampling schedule that includes interviews with parties at the busiest 17 entry/exit points that account for more than 70 percent of use by the population of interest for the trend study. The overall sampling goal is to obtain two representative samples from relatively large populations of visitors – (1) overnight users and (2) day users. It is generally desirable to obtain a sample of at least 250 from a large population to provide the appropriate power for statistical analysis, and assuming a 75 percent response rate, this requires 666 visitor intercepts for the two samples.

Sampling at busy entry points will be for half days, alternating between entry hours and exit hours of the day. The other half day will be used to sample visitors prior to entry at permit distribution centers. This method of visitor contact may be less ideal for being able to talk to all group members than on-site contacts, but it will be more efficient for reaching visitors to low use entry points. Five centralized communities, each having both Forest Service and private concessionaire permit distribution points, will be used for centralized sampling on alternating mornings and afternoons. Allocated permits may be picked up by group leaders or their designees on the day before or the day of the trip, so sampling should be effective throughout the day at these locations. The centralized location closest to the primary sampling point for that day will be used during the alternate half day.

The schedule calendar is constructed to provide two independent sample schedules at the 17 primary sampling points with each occurring randomly across 25 percent of the days in the peak season. The schedule uses two interviewers, each working 38 days out of the 153-day peak season, for a total of 76 sample days. This will require each interviewer to make an average of 2.2 visitor contacts per day to reach the target of 666 total intercepts.

For each interviewer, 19 random days were chosen and then the day before or the day after was alternately added to form sampling blocks of at least two days. Each of those days was then randomly assigned to one of the primary sampling locations, with distribution of sampling across entry points adjusted by level of use. Specific site sampling plans have been developed to guide the interviewers on each day of the sampling season. For each day, a morning or afternoon sampling unit was randomly selected (7:30 – 11:30 am for on-site, 7-11:00 am for permit distribution centers; 2 – 6 pm for on-site locations, 1 – 5 pm for permit distribution centers) and then an afternoon location and time period assigned to avoid bias toward selection based upon prevalent times of departure and arrival. In most cases, entry is limited to one or two specific launch points or trailheads and that is where on-site sampling will occur. However, a few of the points have numerous entry locations that may differ by type of use. These have been identified and randomly chosen for on-site sampling.

The following table shows the 17 entry points that will be sampled along with estimates of their types and levels of use during the peak season.

The next table shows the central locations that will serve as the secondary sampling points at Forest Service and commercial permit distribution centers along with distances and driving times between them. Following this table is the calendar of sampling dates and locations. This information provides a sense of the logistics involved in conducting the sampling.

Analysis of quantitative data will begin with descriptive statistics to display the current responses from visitors. For selected variables, parametric and nonparametric tests of comparability for categories of subjects, such as party leaders and party members or for permit compliers and non-compliers, will be presented and discussed.

To accomplish the stated objectives of determining trends, the data is subjected to a series of comparative analyses. This process of trend analysis will vary slightly across areas. For example, for a previous study at the Desolation Wilderness conducted under a previous OMB approval, due to progress made in increasing day use permit compliance rates between 1972 and 1990, it was believed that appropriate trend analysis would necessarily proceed in a stratified manner. All tests of significance at Desolation were conducted on comparisons of 1972 day-user responses to 1990 day-user responses. Likewise, 1972 overnight-user responses were compared to 1990 overnight-user responses, as were party-leader responses. The major question addressed was whether or not there were differences in user characteristics, visitor behavior, or visitor preferences between the sample years. Next, we asked how accurately these data reflected the true situation at the time of the 1990 study using a party-member sample and a check on permit non-compliers. For the Boundary Waters Canoe Area Wilderness, we will be interested in understanding how adding day use visitors and better coverage of low use entry points makes our overall estimates of population characteristics more accurate. We are also interested in why observed differences are evident and will examine those variables that exhibited some degree of change. User characteristics, reported behavior, and preferences (and changes in these) will be cross-tabulated by selected independent variables such as mode of travel, length of stay, experience, and socio-demographic characteristics.

Specific hypotheses can be tested for each item in the questionnaire as well as for relationships typically studied in wilderness use research. The following hypotheses are examples of how trends will be tested for significance, to allow comparison to baseline information.

• Hypothesis 1: The percentage distribution of visitors across the primary methods of wilderness travel is not different across the three study years. Variables to be tested: Study year and method of travel. Appropriate analysis method: Chi-square.

• Hypothesis 2: The percentage distribution of visitors across the various education level categories is not different across the three study years. Variables to be tested: Study year and education levels. Appropriate analysis method: Chi-square

• Hypothesis 3: The amount of experience the visitors have at the study area is not different for the the study years. Variables to be tested: Independent variable – study year; Dependent variable – number of previous visits to the site. Appropriate analysis method: ANOVA with adjustment for non-normality if needed.

• Hypothesis 4: The relationship between the visitor’s expectations for social encounters and reported trip evaluation is not a significant one. The comparison of expected with experienced encounter levels will serve as the independent variable and the evaluation will be considered a single dependent variable in an effort to explain variation in the reported evaluation through a Chi-square analysis. The change in this relationship over time will be the trend of interest in this analysis.

Trend data of this sort is of national interest to many agencies, organizations, and individuals interested in how wilderness visitors have changed and are changing. Forest Service Research Papers summarize the descriptive trend data and review major findings. Additionally, more interpretive aspects of the results, such as changes in visitor preferences, will be presented at national and international symposia and published in the International Journal of Wilderness and other journals.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sam­pling, a special justification must be provid­ed for any collection that will not yield "reli­able" data that can be generalized to the universe studied.

A multi-stage cluster sampling design has been suggested for this type of social research. The primary sampling unit is actually blocks of time (essentially visitors to the area during that block of time). Before the blocks of time are selected, a stratification scheme is employed to define weekend clusters (Friday through Sunday) and weekday clusters (Monday through Thursday). The first stage cluster sample draws random clusters from each strata per month of sampling. This is 2 weekend and 2 weekday clusters chosen randomly from the possible clusters for each month. In a 7-month use period, the sample is 28 total clusters, 7 pairs of weekdays and 7 pairs of weekends. The second stage of this sampling procedure is to select smaller clusters within each of the 14 pairs of clusters. Visitors to a specific trailhead on a particular day are a subdivision of a cluster of days. The third stage involves additional time clusters, with time shifts for sampling. The likely clusters of time to be randomly assigned to each of the individual sampling locations on a particular day would be 2 of 4 possible 4 hour time periods, varying slightly depending upon when permit distribution centers open and close and when morning and evening arrivals and departures are most likely. All visitors entering or leaving through the chosen trailhead would be included in the sample.

In the selected sampling method, clusters are chosen through simple random sampling, but a ratio estimator is used as a measure of central tendency. Ratio estimators are quotients of two variables, each of which varies randomly from cluster to cluster. Ratio estimation is considered to be an efficient technique. The ratio estimator equals the sum of the cluster totals divided by the sum of the cluster sizes, where the sums range over all clusters in the sample. Ratio estimators may be biased and variances can only be approximated. However, the degree of bias is usually negligible for sample sizes likely to be encountered in practice. The ratio estimator is consistent. As is the ratio estimator of the population mean, this estimation is biased. The bias of the estimated variance is inversely proportional to the sample size, n, and a serious problem only for small sample sizes. Jaeger provides a method of approximation of the bias of the ratio estimator: the estimator is inversely proportional to the number of clusters sampled. The ratio estimator is unbiased if the mean per element within clusters is uncorrelated with sample size.

Response to the on-site contacts is expected to be very high. It is not uncommon for 100 percent of a sample of visitors contacted at trailhead locations to agree to participate in a study. Unfortunately, not all of those agreeing to participate will participate beyond the on-site contact. There are some who will not return the mail-back questionnaire. It is believed that the primary reason that some do not mail the questionnaire back is due to a belief that since visitors may not participate in recreation very often at that particular place, their opinions may not be very important. Follow-up mailings are used to convince them otherwise. Past response rate examples for similar surveys include the Boundary Waters Canoe Area Wilderness (74 percent response), Shining Rock Wilderness (75 percent response), Desolation Wilderness (83 percent response), and Gates of the Arctic National Park and Preserve (95 percent response).

Don A. Dillman, of Washington State University, published a book entitled Mail and Internet Surveys: The Tailored Design Method in 2000, which precisely documents the appropriate ways to assure high response rates in mail-back surveys in social research. Dillman’s methods have been used in many dispersed recreation visitor studies and have produced consistently high response rates. Dillman provided guidelines for writing initial and subsequent cover letters in which a justification of the information collection effort appears along with an appeal for response based upon the importance of each individual sampled to respond for a larger population of people represented. Following this approach, there would typically be an initial mailing of information, a postcard reminder, and two follow-up mailings of the questionnaire and appropriate cover letter.

Whether or not this minimum response rate of 70 percent is obtained using these methods, on-site responses for respondents and non-respondents will be compared. Enough basic information is being collected from all people to help us understand whether the respondents and non-respondents differ to a significant degree on basic demographic factors and area visitation patterns.

The chief statistical consultant for this study will be David Turner, Station Statistician, Rocky Mountain Research Station, 860 North 1200 East, Logan, UT 84321 (801) 755-3560. All sampling and surveys will conform to guidelines established by Watson, Cole, Turner, and Reynolds (2000, Appendix 7).

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separate­ly or in combination with the main collection of information.

While the test instrument is largely a replication of the survey used in 1991, some changes have been made due to review suggestions made by managers and peer reviewers. There have also been some additional items to capture some new issues since 1991, and of interest to managers there. Some Boundary Waters Canoe Area Wilderness visitors have already reviewed and completed the draft survey, and an additional 10 pilot test respondents will be identified at the beginning of the use season and will be asked to complete the survey on-site to allow analysis prior to actual initiating mailings to the larger sample.

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Dr. Joseph W. Roggenbuck

Professor, Virginia Tech, Retired

Blacksburg, VA

(540) 961-7553

[email protected]

Dr. William T. Borrie

College of Forestry & Conservation

32 Campus Drive

University of Montana

Missoula, MT 59812

(406) 243-4286

[email protected]

Dr. Ingrid Schneider

University of Minnesota

CFANS

Department of Forest Resources

115 Green Hall

1530 Cleveland Avenue North

St. Paul, MN 55108

(v) 612 624 2250

[email protected]

Carolyn Swan

Mathematical Statistician

USDA/NASS

Statistics Division/Methods Branch

(202) 690-8639

QUESTION JUSTIFICATION MATRIX (Overnight survey)

Study Objective	Key Wilderness Management Issues
	Resource Conditions	Visitor Management	Recreational Quality	Use Distribution	Trends in Use & Users
Visit characteristics	3b,11	2b,3c,13a,13b,13c,13d,13e	4,5,16,17,18,19,22,23	1,5,17,18,20,23,25	1,2a,3a,4,5
Visitor characteristics					28,29,30,31,32,33,34,35,36,37,38,39,40
Visitor preferences	7,11,26	7,12,13,14,15, 26,27	5,6,8,9,10,12,17,18,20,21,23,,24,26,27	9,10,25	5,6,7,8,9,10,12
Visitor knowledge	16,19,22