Supporting Statement B - 1006-0028

Supporting Statement B

Recreation Visitor Use Surveys

OMB Control Number 1006-0028

Bureau of Reclamation

Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When the question “Does this ICR contain surveys, censuses, or employ statistical methods?” is checked "Yes," the following documentation should be included in Supporting Statement B to the extent that it applies to the methods proposed:

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Generally, respondents to Bureau of Reclamation (Reclamation) administered surveys will be customers directly engaged in recreational activities on Reclamation lands at the location where the survey is being administered. These respondents are considered to be the target population for information collection purposes. Visitors typically come from the local area where the surveys are being administered. However, many visitors come from large metropolitan areas within a 100 mile radius and from adjoining states. Visitors will be sampled and surveyed using probability sampling. Several of the surveys target specific users engaged in specific activities such as river rafting on a specific river, visitors camping at a developed campground, or visitors boating at a specific reservoir.

In 2009 and 2010, the Recreation Fee Survey was utilized at the New Melones Lake recreation area. Since there is now actual data regarding the Fee Survey, we are better able to estimate the more precise number of respondents who may actually fill out the Fee Surveys in the future. To further reduce the margin of error, staff at New Melones conducted well over 500 surveys in both years that the survey was performed. In 2009, there were a total of 692 surveys handed out to ensure at least an 80% response rate and 95% confidence level. In 2009, there were 692 surveys handed out and 649 completed. In 2010, there were 819 surveys handed out and 513 completed. These figures were then averaged to obtain the current estimated number of respondents: 581 (649 + 513 ÷ 2 = 581). Despite the disparities in number of respondents between these 2 years, the sample size was sufficient to meet the minimum requirement of at least 400 respondents needed to achieve a 95% confidence level with a ±5 percentage points sampling error. When factoring the number of surveys handed out with the number of surveys completed, a 77% completion rate was achieved.

Accordingly, a response rate of 80 percent is expected. It is anticipated that response rates will be high as most respondents, particularly those participating in intercept interviews, are highly involved and interested in the survey subject matter. In addition, surveyors will, for the most part, collect completed survey questionnaires on site which will increase the response rate. For the 11 surveys, it is anticipated that 348 survey questionnaires must be handed out per area to achieve the desired return of 278 surveys (80 percent response rate). Anticipating an 80 percent response rate for the remaining Fee Survey, at least 500 surveys will need to be handed out to achieve the desired return of at least 400 surveys.

2. Describe the procedures for the collection of information including:

2a. Statistical methodology for stratification and sample selection.

Surveys will be distributed to visitors at specific recreation areas within a Reclamation Federal water project. The sampling frame is a defined geographic area consisting of a reservoir and its immediate surrounding land base or a defined river corridor that is an integral part of a Reclamation project area. These geographic areas/sampling frames are relatively stagnate and will likely remain stable during the 3-year OMB approval period. However, Reclamation will modify and update the sampling frame if necessary to respond to each Reclamation office’s specific needs related to each survey collection. Any updates or changes will immediately be made available to all surveyors participating in the survey collection.

The approved surveys will be administered on site through the use of personal interviewing techniques. Completion of the surveys by respondents is voluntary. Under the supervision of a project leader, employees, contractors, or volunteers will ask the questions verbatim from the pre-approved questionnaires and will record responses or hand out questionnaires and wait for individual respondents to return them immediately upon completion. For those individuals staying in the area, arrangements may be made for surveyors, camp hosts, or volunteers to pick up completed surveys at a later time or have the individuals deposit them in a predetermined location at a campground, entrance station, visitor center, or survey drop box.

The sampling design that will be used is called Stratified Cluster Sampling (Sample Survey). This technique is used where you do not know in advance who is in the population (e.g., you do not have a list of names from a phone book from which to randomly select from). Stratified Cluster Sampling is especially useful when sampling recreation areas where there is no way to determine who will be visiting. This allows sampling at different field locations based upon time blocks, which include days of the week and hours of the day, that is further based upon an estimate of the percent of visitation at each of the locations. The time blocks are then randomly selected and assigned for survey sampling. Visitors are then surveyed in these locations during these time blocks. Typically, sampling times will include an 8-10 day period with a random start in June and three, 4-day periods randomly selected during July and August. Note: If the recreation area has a longer recreation season than Memorial Day to Labor Day, then the time period for administering the survey may be established to correlate to the proper length of the recreation season. Appropriate time blocks will be established for those areas. Whatever the length of recreation season, sample periods will be selected to ensure a balance of weekend and weekday periods and an appropriate distribution across the entire summer season.

In order to get input from a representative sample of all users to individual Reclamation water projects, every fifth group consisting of one or more individuals will be surveyed. For groups of more than one individual, the surveyor will solicit input from the person that will have the next birthday of those that are in the group. In all cases, those individuals surveyed will be 16 years of age or older. For the purposes of this survey collection process, Reclamation has not attempted to differentiate between a personal group and an organized user group. Valuable information can be obtained from both types of groups. Therefore, both types of groups will be surveyed.

On selected sampling days, interviewers will introduce themselves as researchers working for the Bureau of Reclamation. They will alert the potential respondents that participation is voluntary and that their anonymity will be protected. The surveyor will mention the length of time that it will take to fill out the survey and briefly describe what the survey is about. Visitor contacts will occur based on a pre-designed systematic schedule established by the project leader. Sampling will begin with the first available group during the sample time. Once a surveyor has finished with one group, he/she will move to the next eligible group that arrives at the survey site. If a group refuses to participate, the surveyor will then contact the next eligible group, adhering to the established sampling schedule of intercepting every fifth group.

There may be visitors that refuse to participate in the survey collection (e.g., a day use visitor that arrives late in the day and wishes to immediately participate in his or her activity then return home shortly after arrival). All surveyors will keep a log and record the disposition of every contact, including refusals and observable characteristics of both the non‑participants and the participants for use in a non-response bias analysis. Some of the items or characteristics that can be entered in the log for both respondents and non‑respondents are the date, time, and place where the survey was handed out; group size; activity that the respondent was participating in; approximate age of respondent; and whether the respondent was a male or female or an adult or child.

Private contractors, managing partners, or universities will also use the survey instruments to collect data for Reclamation. This will typically be completed under the terms of an acquisition document. The collected information will be used to supplement studies and analyses of Reclamation sites or resources. Contractual requirements will limit the use of the data to only support Reclamation programs and research efforts.

All individuals administering the approved survey questionnaires will receive orientation and training in the use of survey techniques by the project leader. In this manner, they will be sensitized of the need to assist respondents, ensure accuracy of the information being solicited, the need to reduce the burden on respondents, and the value of following the sampling plan.

Once the surveys are collected, the data are entered into a Statistical Package for Social Science (SPSS) program. Statistical programs are run on the data using SPSS. In all cases, non-parametric statistics are run. These include frequencies, means, and standard deviations. Depending on the research questions to be answered, parametric statistics are run. There are many possibilities, but typically a test of significance or analysis of variance such as a t test or chi-square would be run. The statistics are then interpreted and the findings documented in a findings report and presented to mangers for decision-making purposes.

More detailed instructions for administering surveys and a summary of the information collection process can be found in the Code of Conduct and Instruction for Surveyors document contained in this OMB package.

2b. Estimation procedure.

In most cases there will be a need to employ a statistical methodology to identify the actual sample size once a survey is determined to be necessary. For the purpose of this information collection, a statistical methodology has been applied to determine the sample size based on a number of parameters. It is assumed that there are approximately 700,000 visitors annually at the site where the Fee Survey will be administered and 30,000 average visitors at the remaining areas that might also be surveyed (source of information is Reclamation’s Visitor Use Data Report, which is a database that, among other things, tracks annual visitation to Reclamation recreation areas.). The average sample size of the Recreation Fee Survey, after averaging the actual number of respondents who completed the survey in 2009 and 2010, is 581. The average sample size for the remaining 10 surveys yet to be performed is 278, as determined by statistical formula. These sample sizes will achieve the desired confidence level of 95 and 90 percent respectively with a sampling error of ±5 percentage points.

Sample Size Methodology Used

(1) Selected a desired confidence level (i.e., probability that the results will be for a given level of precision):

Confidence Level Standard Deviations from the Mean

80% 1.28

90% 1.64

95% 1.96

(2) Determined an acceptable level of accuracy for results in terms of standard errors (e.g., Reclamation wanted to be accurate within 5% at the 95% confidence level):

Standard Error =

Accuracy

Standard Deviation with Confidence Level of Interest

= 0.025 (or 2.5%)

=

0.05

1.96

(3) Approximated the variance in the population: Because the variance was unknown, Reclamation assumed the highest possible proportional variance that occurs when 50% of the sample possesses the characteristic of interest and 50% do not.

(4) Calculated sample size with a 95% confidence level: n = sample size and p = proportion of population having characteristic of interest (i.e., 50%/50% from #3 above). When surveying people, the approximate size of the population must be determined that in most instances is over 1,000 people. With the exception of the Recreation Fee Survey, the sample size rarely exceeds 400 respondents; therefore, the following formula was used because the population (i.e., number of visitors to the area) being sampled is equal to or greater than 1,000.

(a) n =

p (1-p)

(Standard error)²

n =

=

= 400

0.5 (0.5) 0.25

(0.025)² .000625

The example above only includes the calculations for a 95% level with a ±5 percentage points sampling error. The calculations for a 90% confidence level were determined the same way.

As we noted above, the Recreation Fee Survey at the New Melones Lake achieved a 77% completion rate for the years 2009 and 2010. Despite the disparities in number of respondents between these 2 years, the sample size was sufficient to meet the minimum requirement of at least 400 respondents needed to achieve a 95% confidence level with a ±5 percentage points sampling error.

Overview of Proposal

The purpose of this proposal is to submit a request to the Office of Management and Budget (OMB) to reauthorize Reclamation to sponsor surveys for the collection of recreation data from users of Reclamation lands, rivers, and reservoirs, ensuring compliance with the requirements of the Paperwork Reduction Act. These data collections would be pursued on an as needed basis over a 3-year period to aid in proposed project evaluations and would be paid for using project funds. To meet its needs for the collection of visitor use data, Reclamation is requesting OMB to reauthorize a set of 11 pre-approved recreation visitor use surveys to be administered in their entirety and as approved by OMB.

The approved surveys will be administered on site through the use of personal interviewing techniques. Completion of the surveys by respondents is voluntary. Under the supervision of a Project Leader, employees, contractors, or volunteers will ask the questions verbatim from the pre-approved questionnaires and will record responses or hand out questionnaires and wait for individual respondents to return them immediately upon completion. For those individuals staying in the area, arrangements may be made for surveyors, camp hosts, or volunteers to pick up completed surveys at a later time or have the individuals deposit them in a predetermined location at a campground, entrance station, visitor center, or survey drop box.

A stratified-cluster sampling technique will be used to both identify potential respondents and obtain survey information. The stratified cluster sampling technique allows sampling at different field locations, days of the week, and hours of the day based upon the percent of use at each of the locations and times. All people are then surveyed at these locations during these times.

Private contractors, managing partners, or universities will also use the survey instruments to collect data for Reclamation. This will typically be completed under the terms of an acquisition document and the collected information will be used to supplement studies and analyses of Reclamation sites or resources. Contractual requirements will limit the use of the data to only support Reclamation programs and research efforts.

All individuals administering the approved survey questionnaires will receive orientation and training in the use of survey techniques. In this manner, they will be sensitized of the need to assist respondents, ensure accuracy of the information being solicited, and of the need to reduce the burden on respondents.

Once the surveys are collected, the data are entered into a Statistical Package for Social Science (SPSS) program. Statistical programs are run on the data using SPSS. In all cases,

non-parametric statistics are run. These include frequencies, means, and standard deviations. Depending on the research questions to be answered, parametric statistics can be run. There are many possibilities, but typically a test of significance or analysis of variance such as a t test or chi-square would be run. The statistics are then interpreted and the findings documented in a findings report and presented to managers for decision-making purposes.

2c. Degree of accuracy needed for the purpose described in the justification.

A 95 percent confidence level is required for the Recreation Fee Survey. The remaining surveys require a 90 percent confidence level. All surveys have a sampling error of ±5 percentage points with an anticipated response rate of at least 80 percent.

2d. Unusual problems requiring specialized sampling procedures.

There are no unusual problems requiring specialized sampling procedures. .

2e. Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

The surveys will be used by different field offices throughout the 17 Western States. However, the surveys will not be conducted annually at the same location but periodically used at different locations on an as needed basis.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Participation in the recreation user surveys is always voluntary. In our experience, recreation users who are directly engaged in the experience for which information is being solicited are enthusiastic about the opportunity to render opinions affecting future management, custodial responsibilities, and care of the area. Therefore, voluntary compliance is most often high while non-response rates are low.

Response rates will be maximized through careful consideration to detail in the development and implementation of the survey process.

In general, our experience with surveying recreationists has shown non-responsiveness to be a minor issue. However, if a respondent is not willing to participate in filling out a survey, Reclamation will not assume a zero value for those respondents to prevent a non-response bias to be integrated into the survey results. Instead, Reclamation will track the total number of surveys distributed against the number of completed surveys. The final report will describe the nonresponse bias as the number of surveys that were not completed or returned. If certain questions within a survey instrument are not answered, professional judgment will be used to explain why an answer to a specific question was left blank.

In addition to tracking the number of surveys that were not completed or returned by respondents, the surveyor will also attempt to ask the respondent why he/she refused to participate. If the respondent provides a reason for not participating in the information collection, Reclamation will record the responses in a log to be kept by the surveyors.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The current proposed surveys are based on the surveys conducted in 2009 and 2010. Successful pretesting occurred prior to OMB approval of the surveys in 2008.

5. Provide the names and telephone numbers of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The recreation-related surveys are based on information in use by the Corps of Engineers that were originally developed, tested, and improved in cooperation with experts in survey research. Experts assisting the Corps included: Dr. Chilman, Associate Professor, Forestry Department, Southern Illinois University; Dr. William Hammitt, Professor, PRTM Department, Clemson University; and Dr. Daniel Stynes, Professor, Parks and Recreation Department, Michigan State University.

Reclamation secured the services of Dr. Glenn Haas, Dr. Robert Aukerman, and Dr. Marcella Wells of Aukerman, Haas and Associates, LLC to review and provide feedback on the clarity of the questions, relevance, and format of the survey questionnaire. Dr. Haas and Ackerman are both Professors’ Emeritus at Colorado State University, Fort Collins, Colorado, and have extensive experience in the field of recreation and tourism. Dr. Wells is a former professor at Colorado State University in the field of recreation and tourism specializing in interpretive planning.

Dr. Robert Aukerman

729 Duke Square

Fort Collins, CO 80525

970-484-2028

Dr. Glenn Haas

P.O. Box 105

Como, CO 80432

970-691-2641

Dr. Marcella Wells

3403 Green Wing Court

Fort Collins, CO 80524

970-498-9350

The three consultants listed above believe that the burden hour estimates are a good assessment of the time required to complete each survey and had no further comments.

For further information regarding the development or re-approval of the survey questionnaires, please contact Jerome Jackson at 303-445-2712, or by email at [email protected].

8