Response to OMB comments and questions

Response to OMB questions/suggestions received November 29, 2007

Alan Watson, Rocky Mountain Research Station, Aldo Leopold Wilderness Research Institute, Missoula, MT

January 2, 2008

Suggestion 1: In order to ensure that each individual is only surveyed once per season, OMB suggests that each individual be asked if they have previously completed a survey OR you put some sort of crosschecking system into the process.

In the Supporting Statement, item 2.e., this was the stated intent:

5. How frequently will the information be collected?

This information will only be collected once for each visitor contacted during the 2007 use season, representing one trip to the Boundary Waters Canoe Area Wilderness.

To assure this happens, the on-site contact sheet has been changed to contain a reminder, with the question about previous participation included for confirmation upon initial contact:

Previously interviewed this season? NO ____ YES____ (See attached on-site contact sheet)

Action item 1: Please send copies of all letters and cards associated with the surveys that will be handed out or sent to individuals

Attached as requested: Cover letter 1, cover letter 2 and post card reminder

Action item 2: Day Survey Item 6, change "preferred" on scale to "expected" (see yellow highlighted portion of attached Day Survey)

Changes made (see Day use survey and Overnight survey)

Action item 3: Day Survey Item 38. Condense income levels to 4 categories, or provide rationale for use of 11 different categories

We would like to keep this more precise categorization scheme, if possible. We know, from past studies, that wilderness users represent a broad segment of the spectrum on income. There will be many students and young people in very modest income categories, but there will also be many highly educated, professionals in very high-income categories, particularly those on guided or outfitted trips and accessing the Boundary Waters with motorized boats. Previous OMB clearances have asked us to be more accurate in the understanding we develop about what segments of the American society are using wilderness, in order to address some perceptions of elitist user groups, particularly the wealthy leisured. We feel the need to both accurately reflect the income categories of users that can be compared to U.S. Census reports for the region in which most users originate, as well as provide trend analysis from comparison to previous studies. Trying to accomplish both of these tasks leads us to as precise a categorization scheme as possible, without probing for exact numbers.

Action item 4: Day Survey Item 45. Change "check all that apply" to "please mark one or more" (see green highlighted portion of attached Day Survey)

Changed on both the day use and overnight visitor forms (see attached)

Question 1: What is the timeline for issuing the report once the data is collected and reviewed? Six months? One year?

Managers will receive summary statistics within 3 months of final data collection (there is need to allow time to receive final mail-outs from reminders). After discussion of descriptive results, and confirmation that descriptive statistics have been tabulated correctly, more complex analysis and trends will be conducted to answer the basic questions driving this study, plus those asked by managers after seeing descriptive statistics. A final report will be provided to the management unit within 6 months of receipt of feedback and review. Over the following year, additional analyses will be conducted for peer review journal articles and conference presentations.

Question 2: Trends analysis - what differences do you anticipate you will be able to detect?

From the trend analyses, we hope to detect whether there are differences in perceptions of social conditions, perceptions of resource conditions and attitudes towards specific management actions that might address these issues. Managers need to know if distribution of method of access (canoe, paddle, and hike) is changing, whether emphasis on fishing is changing among visitors, and their level of past experience at this place. Managers are also interested in understanding how difficult it is for visitors to locate acceptable campsites and how the attitudes of current visitors about appropriateness of general wilderness management techniques might be different from those found in previous studies.

Question 3: Questions 13 and 14 (see blue highlighted portion of this document) use a fairly complex ranking system. Have you used this type of ranking system before? If so, were there any problems with it? Did you receive any comments about this ranking system in testing of draft? If so, what were they? If you receive negative comments during pre-testing phase, how will you modify the questions?

The format is complex, particularly for Question 13, and therefore has been examined in many ways before. We kept the number of these items at a very minimum in this Boundary Waters study, but felt it was important to include the ones we did. In one previous study using this response format, university cooperators from Virginia Tech did a generalization analysis to understand how timing influenced subject response to these social judgment scales. And although a great deal of research has been published that generally have tested these scales in a variety of social science contexts, we were most concerned about whether using them in a mail back format instead of on-site would influence the generalizability of the results. In fact, that research was accepted for publication in Forest Science, our more prestigious journal in the forest management field, and provided evidence that the results from the mail back format was conservative in results, but corresponds consistently to on-site results. Another factor to consider in replicating these social judgment scales is that a very complex analysis of the previous trend study in 1991 was heavily based on these items, with responses recorded in exactly the same way. The analysis was published in the Northern Journal of Applied Forestry. Comparison between past analysis and findings from this survey is anticipated to be some of the most important analysis coming out of this study and will help managers evaluate the long-term success of the permit quota system currently in place at the Boundary Waters Canoe Area Wilderness.

Question 4: Part B - item 2, page 4 (see pink highlighted portion of Supporting Statement Part B - attached) Minimal Sample Size - What is the plan if too few people respond for statistical analysis?

We certainly intend to monitor response both on-site and mail back as contacts are made to understand how our expectations for sampling success compare to what actually happens. If we have underestimated the success of our contact method, we will adjust quickly to meet our initial contact objectives. If response is low on mail back, we would be very surprised. We have established a minimum target of 70% based on many years of conducting this type of survey. We fully expect reaching that minimum, and we will conduct the non-response bias we described. If, due to unforeseen circumstances, the response rate were unacceptably low, it would merely limit us on our ability to sort out our analysis into all the most desirable stages. For instance, instead of subdividing respondents into fine categories based upon previous experiences (e.g., first time visitors, those who first visited over 3 years ago, those first visiting 4 to 10 years ago, and those over 10 years), though this is the most useful categories found in previous research focused on this question, we would go with a more simple approach (e.g., first time visitors, those under 10 years and those over 10 years). The back up plan would be for a second year of sampling.

Question 5: Part B - item 4, page 7 (see green highlighted portion of Supporting Statement Part B) Pre-testing - Did you allow more than 9 individuals (non-federal) to complete the draft pretest?

Only nine individuals will be sampled for pre-testing purposes. One advantage is that many of these questions have been asked previously at the Boundary Waters Canoe Area Wilderness. Several university studies conducted in recent years have followed our question content and format. We are also aware of some university studies following our sampling methods and aware of very good response rates. While none of these studies or any student projects at the Boundary Waters provides us with the information managers need at the Boundary Waters, we will look at them closely to give us additional insight into our own methods and questions to be asked.