Pt B - Suptng Stmnt

Supporting Statement for Paperwork Reduction Act Submissions
Recreation Survey New Melones Lake Project, Sonora, CA.
Bureau of Reclamation

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
The agency should be prepared to justify its decision not to use statistical methods in
any case where such methods might reduce burden or improve accuracy of results. When
Item 17 on the OMB Form 83-I is checked "Yes", the following documentation should be
included in the Supporting Statement to the extent that it applies to the methods proposed:
1. Describe (including a numerical estimate) the potential respondent universe and any
sampling or other respondent selection method to be used. Data on the number of entities
(e.g., establishments, State and local government units, households, or persons) in the
universe covered by the collection and in the corresponding sample are to be provided in
tabular form for the universe as a whole and for each of the strata in the proposed sample.
Indicate expected response rates for the collection as a whole. If the collection had been
conducted previously, include the actual response rate achieved during the last collection.
RESPONDENT UNIVERSE AND RESPONSE RATE
Phase I: On-Site Survey
The potential respondent universe for the on-site recreation survey is all visitors to New
Melones Lake Area. This universe is estimated at 700,000 visitors per year. This estimate was
extrapolated from a combination of vehicle count data and campground records. The confidence
interval for this survey will be ± 3 percentage points and the confidence level will be 95 percent.
The sample size therefore will need to be at least 1,100 surveys.
Respondents will be visitors to New Melones during the survey period who are over the age
of 18 and are willing to take our survey. Though the on-site survey is self-administered, the
surveys will be distributed by a pair of survey assistants. The assistants will try to approach all
groups of visitors during the survey period, but will only distribute surveys to visitors over the
age of 18.
A prior survey that used a similar survey instrument and techniques and was given at a
recreation area associated with a reservoir in the Central Valley of California in 2005 had a
response rate of 87.3%; we estimate that the response rate for our survey will be in a similar
range.
Phase II: Telephone Survey
The respondent universe to be sampled by the telephone survey will be dictated by the results
from the on-site survey. The on-site survey results will determine the primary market area for
1

the New Melones Lake Area. The primary market area for a recreation site is generally defined
as the area from which 70-75 percent of its visitors will originate. A review of zip code data
from similar recreation surveys in the Central Valley suggests that the highest patronage of a
recreation area comes from the counties in which the area is located. So for the telephone
survey, the universe has been assumed to be the two counties in which New Melones Lake Area
is located, Calaveras and Tuolumne Counties; the approximate population of these two counties
is 106,251 people. However some visitation to New Melones is known from camping records to
come from the nearby counties of Alameda, Contra Costa, Sacramento, and San Joaquin. Of
these counties, Alameda and Contra Costa counties have the largest populations which together
total 2,812,387 people.
To estimate sample size for the telephone survey, the respondent universe of 3 million
respondents was used to guarantee a sufficiently large sample size. The telephone survey will
have a confidence interval of ±5 percentage points, the confidence level will be 95 percent, and
the sample size approximately 500.
Respondents to the telephone survey will be private citizens, over the age of 18, who speak
English, and who answer a telephone number associated with the primary market area for New
Melones Lake. The telephone survey will be given orally by a trained surveyor who will ask for
a respondent who is over the age of 18. The telephone numbers will be randomly generated for
the market area that was established by the on-site survey.
A prior survey that used a similar survey instrument and techniques and was given at a
recreation area associated with a reservoir in the Central Valley of California in 2005 had a
cooperation rate of 76.2% percent. The cooperation rate was determined by dividing the number
of people surveyed by the number of eligible phone numbers called. We estimate that the
response rate for our survey will be in a similar range.
2. Describe the procedures for the collection of information including:
* Statistical methodology for stratification and sample selection,
* Estimation procedure,
* Degree of accuracy needed for the purpose described in the justification,
* Unusual problems requiring specialized sampling procedures.
* Any use of periodic (less frequent than annual) data collection cycles to reduce
burden.
Sample Selection
Phase I: On-Site Survey
The selection of the on-site survey sample during a sample day will occur every other hour
for 10 hours a day. The 10 hour sample period will alternate each day between starting at dawn
on 1 day to ending at sunset the next day. During this period the two survey assistants will
approach every group in their survey location for 1 hour and ask that one member of that group
complete the survey – though if more than one member of the group offers to complete the
survey that sample will also be taken. An emphasis will be put on approaching every potential
2

respondent group within a sample hour. Though surveyors will be fully equipped each sampling
morning, the intervening hour between sampling periods will allow supplies such as surveys and
pencils to be replenished, as well as time to assist respondents and collect completed surveys.
This will ensure that there will be no limitation of the respondents sampled due to survey
shortages. The surveyors will select a respondent by following a strict set of guidelines during
the survey period which are intended to keep surveyors safe and maximize response (see
surveyor instructions).
A series of four on-site surveys will be implemented at seven locations throughout the unit.
Surveys will be conducted through the recreation season in the spring (May), summer (July), late
summer (August), and fall (October). These four periods where chosen to sample both peak
visitation and off-peak visitation throughout the recreation season. Surveys will be conducted
during both weekdays and weekends.
Also a large number of locations will be surveyed in order to approximate a random sample.
The goal of the on-site design is to organize the survey so that if one recreates at New Melones
Lake during the sample period there is an equal likelihood of being asked to fill out a selfadministered survey as anyone else who is visiting New Melones irrespective of the type of
recreation one prefers. With a large number of sample locations the survey will minimize bias
for certain types of recreation. The surveys will be self-administered but distributed by survey
assistants who will be available to clarify questions, distribute surveys to eligible survey
respondents (adults over age 18 and who speak English), and collect completed surveys. With
these controls, we hope to increase the response rate and minimize bias in the on-site survey.
Phase II: Telephone Survey
The on-site survey will determine the market area from which a random survey selection of
telephone numbers will be obtained. Telephone calls will be made from these random numbers.
Those calls that reach businesses or fax machines will be thrown out, while those numbers that
indicate that it is a household will be called up to seven (7) times each in an attempt to make
contact with eligible survey respondents (adults over age 18 and who speak English). Calling
times will include daytime hours during weekdays, evening hours during weekdays, daytime
hours on weekends, and evening hours on weekends.
Sample Size Estimation
For the on-site survey and the telephone survey, the sample size was estimated by using the
following sample size calculation and the correction for finite populations.

3

Sample Size
Z * (p) * (1-p)
2

ss =
c2

where:
Z = Z value (e.g. 1.96 for 95 percent confidence level)
p = percentage picking a choice, expressed as decimal
(0.5 used for sample size needed)
c = confidence interval, expressed as decimal
(e.g., 0.04 = ±4 percentage points)

Correction for Finite Population
Ss
new ss =
ss-1
1+

Pop
where:
pop = population

Degree of Accuracy
Phase I: On-Site Survey
The confidence interval for this survey will be ± 3 percentage points and the confidence level
will be 95 percent. The sample size therefore will need to be at least 1,100 surveys. This degree
of accuracy was chosen because the results of the on-site survey will be used to generate a proxy
for visitation based on vehicle count information and to answer more specific questions about the
public recreation needs and wants at New Melones Lake Area.
Phase II: Telephone Survey
The telephone survey will have a confidence interval of ±5 percentage points, the confidence
level will be 95 percent, and the sample size approximately 500. The lower degree of accuracy
reflects the needs of the telephone survey to (1) describe general recreation trends, (2) identify
barriers that may prevent people from visiting New Melones, and (3) validate and reinforce the
data collected in the on-site survey. As Reclamation does not seek to increase visitation, this
survey will be used to identify possible barriers that might differentially affect certain population
4

segments or identify management changes that could create a higher quality of recreational
experience.
Unusual Problems Requiring Specialized Sampling Procedures
There are no known unusual problems that will require specialized sampling.
Any use of periodic (less frequent than annual) data collection cycles to reduce burden
This information collection is a one-time effort and no future collection cycles are planned.
3. Describe methods to maximize response rates and to deal with issues of non-response.
The accuracy and reliability of information collected must be shown to be adequate for
intended uses. For collections based on sampling, a special justification must be provided
for any collection that will not yield "reliable" data that can be generalized to the universe
studied.
Maximizing Response Rates and the Dealing with Non-Response
Phase I: On-Site Survey
To maximize response rate, the on-site survey has been designed to provide clear information
on the survey process and to maximize the non-monetary ‘rewards’to the respondent for
completing the survey. The respondent receives information about the survey in two ways;
directions are provided in writing in the on-site survey itself and the survey assistant provides
information on the survey process. The directions on the survey are as clear and brief as possible
and the structure of the survey from question Q1 to Q22 follow the typical top to bottom format
of a paper questionnaire. In addition, the survey instrument layout gives the respondent a sense
of progress by allowing a respondent to proceed quickly through its 7 pages and only 22
questions. Also each question provides a no opinion/don’t know response for each question and
at the end of the survey is a free form comment space. It is hoped that if the language within the
survey is difficult for the respondent to interpret, the respondent will utilize the no opinion/don’t
know response and comment space to complete as much of the survey as possible.
Another way to maximize response is to use a simple well organized survey process. For the
on-site survey process, a survey assistant will be utilize who will approach visitors, provide
information, and collect completed surveys. The use of a survey assistant maximizes survey
response by providing individual attention to the respondents. In this way, Reclamation provides
someone on the ground to encourage survey participation from a diverse range of visitors to New
Melones Lake opposed to just those visitors who might read a posted notice or be actively
interested in the New Melones Lake RMP process. Also the simple process of handing a visitor
the survey and providing a writing utensil increases survey response. Additionally the ability of
a survey assistant to provide information on confidentiality, the usefulness of the survey, who is
conducting the information collection, and clarification on the written instructions provides
additional incentive and guidance to some respondents. Reclamation will provide training and
instructions to survey assistants that emphasis being polite, helpful, and being neutral in any
5

interactions the survey assistant has with the public (see attached Survey Instructions). Finally
by collecting the survey at the survey location, the non-response encountered by respondents
who take surveys and never complete and/or return the survey is minimized.
The on-site survey has also been designed to maximize the non-monetary ‘rewards’the
respondent feels by completing the survey. Firstly the survey will be given in locations where
visitors are typically waiting to participate in their chosen recreation activity. The locations that
Reclamation is targeting include natural slow-down areas in which at least one person in a group
is waiting such as at boat launch facilities, fish cleaning stations, campground check-in, and
staging areas for trails and rock climbing. By targeting these locations, the respondents are not
being prevented from participating in their recreation activity. Furthermore, by providing these
respondents with the opportunity to participate in a valuable survey, the survey may be
especially rewarding to a respondent in contrast to the feeling of wasting time waiting.
Additionally the survey is relatively short and involves the respondents provide only basic
demographic data and express opinions; thus the survey increases the respondents sense of
importance and prevents frustration from complicated calculations or the suspicion that
accompanies the release of overly personal information. Also the survey process creates some
reciprocity with the respondent. The respondent is given a brochure which, though free to the
public, is only available from the campground host or at the visitor center. Moreover reciprocity
is created by the basic fact that the respondent recreates at New Melones Lake and this is a way
that the respondent can be a good steward to a place that they visit and use.
One non-response group, non-English speakers, will be tracked in the process of the survey.
The number of groups who did not include at least one English speaker will be recorded by the
survey assistant by writing Non-English speaker at the top of a blank survey and included with
the completed surveys. This non-English speaking non-response rate will be reported in the
survey report. It is hoped that at some future date there will be the resources to survey the nonEnglish speaking visitors at New Melones.
The final report will describe the non-response rate as the number of distributed surveys that
were not completed or returned. The distributed surveys will be tracked and the number and
location of those non-response surveys documented in the survey report. Though highly
unlikely, if the non-response rates is too high double sampling will be use to compensate for
non-response either from the telephone survey results or other surrogate survey. Finally, a
comparison of the on-site survey response with the responses obtained in the telephone survey
will be made. This check will verify that a broad-base of representative visitors to New Melones
were surveyed in the on-site phase of the information collection.
Phase II: Telephone Survey
To maximize response to the telephone survey a simple, logical, and short survey instrument
has been developed and repeated calling of eligible telephone numbers will be used. The
directions on the survey are as clear and brief as possible and the structure of the survey goes
from general questions that most respondents can answer to more specific questions about New
Melones Lake. The survey instrument has been designed to allow respondents to move quickly
through the survey especially those respondents unfamiliar with New Melones. Also each
6

question provides a no opinion/don’t know response for each question. It is hoped that if the
language within the survey is difficult for the respondent to interpret, the respondent will utilize
the no opinion/don’t know response option and complete as much of the survey as possible.
In addition to survey design, repeated calling will be used to maximize response. Those
numbers that indicate that it is a household will be called up to seven (7) times each in an attempt
to make contact with eligible survey respondents (adults over age 18 who speak English). Those
telephone numbers that reach businesses or fax machines will be thrown out. Calling times will
include daytime hours during weekdays, evening hours during weekdays, daytime hours on
weekends, and evening hours on weekends.
The calls that reached a respondent that did not speak English will be recorded by the survey
assistant by writing non-English speaker at the top of a blank survey and inclusion with the
completed surveys. This non-English speaking non-response rate will be reported in the survey
report. It is hoped that at some futures date there will be the resources to survey the non-English
speaking visitors at New Melones.
A non-response for the telephone survey will be reported in the final survey report and
will be considered not answering of the telephone after many phone calls. Also a cooperation
rate will be reported that will describe the number of respondents which refused to complete the
survey. The cooperation rate and the non-response rate will be used as a gauge of the success of
the survey. Though highly unlikely, if these rates are too high analysis of waves* will be used to
compensate for the high non-response rate.
*Analysis of Waves - Compensation through the analysis of waves uses the number of times that
respondents are contacted as a method of estimating the tendency to respond. This method uses a
regression so that the percent response for each wave is regressed on the value of the variable at
each percent rate. Then a line is fitted to the data that would predict the score of the variable if
the percent response were 100%. (January 2004. Nonresponse. P. De Michele, Responsive
Management, Report accessed February 2007 at:
http://www.responsivemanagement.com/download/reports/Nonresponse.pdf. ).
4.
Describe any tests of procedures or methods to be undertaken. Testing is
encouraged as an effective means of refining collections of information to minimize burden
and improve utility. Tests must be approved if they call for answers to identical questions
from 10 or more respondents. A proposed test or set of tests may be submitted for
approval separately or in combination with the main collection of information.
A test run under controlled conditions, using the current staff at New Melones was used to
test the procedures and methods to be undertaken in this survey. Since that pretest, Reclamation
in conjunction with the Department of Interior has done some fine tuning of the scales and
language used in the telephone and on-site survey. In addition to the pretest, much of the
sampling procedure has been adapted from a previous survey performed in the California Central
Valley by California State Parks for Folsom State Recreation Area RMP. As the survey at
Folsom State Recreation Area (FSRA) resulted in good response rates and cooperation rates,
7

information similar to that needed for the New Melones RMP, and statistically valid results;
Reclamation seeks to replicate the success at FSRA.
5. Provide the name and telephone number of individuals consulted on statistical aspects
of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s)
who will actually collect and/or analyze the information for the agency.
The following people were consulted during the initial development of the Recreation
Survey, New Melones Lake Project, Sonora, CA forms. It was not necessary to have any further
consultation concerning the statistical aspect of the design.
Name

Agency

Telephone no.

Kathleen Andereck, PhD

Arizona State University

(916) 989-7192

Richard Knopf, PhD

Arizona State University

(916) 989-7192

Mike Petrinovich

Reclamation

(916) 989-7276

Elizabeth Vasquez

Reclamation (Project Manager)

(916) 989-7192

8