SupportingStatementB_PassStudy_7-16-15

Supporting Statement B

A Survey of National Parks and Federal Recreational Lands Pass Holders

OMB Control Number XXXXXXXXXXXX

Collections of Information Employing Statistical Methods

The agency should be prepared to justify its decision not to use statistical methods in any case where such methods might reduce burden or improve accuracy of results. When the question “Does this ICR contain surveys, censuses, or employ statistical methods?” is checked "Yes," the following documentation should be included in Supporting Statement B to the extent that it applies to the methods proposed.

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The universe of potential respondents for this collection is based upon the number of individuals who have purchased a National Parks and Federal Recreational Lands Pass through the USGS website or call center in the previous 12 months. This listing is maintained by USGS, and potential respondents will be randomly sampled from the entire list.

The table below shows the approximate respondent universe, expected sample size, response rate, and the estimated final number of expected responses for each of the survey components.

Sample Frame	Respondent Universe	Sample Size	Response Rate %	Estimated Final Responses
Annual Pass Purchasers	17,000	1,850	57%	1,055
Senior Pass Purchasers	20,000	1,850	57%	1,055
Access Pass Holders	5,500	877	57%	500

Based on our estimations, we conservatively anticipate a response rate of 57% for the survey. This effort will produce sample sizes that are considered robust in the aggregate with acceptable margins of error between ±5% and ± 2% at the 95% confidence level for all aggregate samples and potential major sub-samples.

2. Describe the procedures for the collection of information including:

* Statistical methodology for stratification and sample selection,

* Estimation procedure,

* Degree of accuracy needed for the purpose described in the justification,

* Unusual problems requiring specialized sampling procedures, and

* Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

We will use a random sampling method to conduct the survey.

Degree of accuracy for the purpose described in the justification: Based on the results of similar surveys of park visitors in Yellowstone National Park (Duffield, Neher, & Patterson, 2006)¹ using similar methods of administration (repeat contact mail surveys), and similar question content (visitation patterns and trip characteristics), we expect a response rate of at least 57% resulting in 1,055 completed surveys for the Annual Pass and Senior Pass samples, and 500 completed surveys for the Access Pass sample. The expected standard errors associated with the simple survey results (proportions) would be + or – 0.025 at a 95% confidence interval (based on an estimated proportion of 0.5) or better.

Estimation Procedure: After the survey data collection is completed, the data will be cleaned, coded, and edited. Sampling weights will also be calculated if needed. The primary estimates of interest from the survey will be calculated using standard statistical software (SAS). Estimates of the distribution of pass-holder use across agencies will be generated using standard measures of central tendency and rules of variance. We will generate statistics to summarize and compare responses, response rates, and individual characteristics across groups (pass-holders of different types of passes). A post-stratification adjustment will also be generated to correct any detected non-response bias.

While no formal multivariate modeling is explicitly anticipated a priori, should the survey data suggest such modeling is appropriate to explore aspects of pass holder behavior or preferences, standard linear or logistic regression methods would be used in the analysis.

Unusual problems requiring specialized sampling procedures: None.

Any use of periodic (less frequent than annual) data collection cycles to reduce burden: This is a one-time data collection.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

A mail-back method will be used to administer this survey. We will use a modified Dillman method to maximize the response rate (Dillman 2014)². All potential respondents will be mailed an initial contact letter designed to inform them of the upcoming survey. A survey packet will include a cover letter, questionnaire, and a self-addressed stamped envelope. Ten days following the initial mailing of the survey packet, we will send a postcard reminder, and two weeks after that the final reminder (which will include another copy of the survey with a cover letter and a postage paid return envelope) will be mailed to all non-respondents.

Non-response testing

We will not use a separate mail back non-response survey. Our most recent experience with mail back surveys have shown that return rate is de minimis compared to the cost of printing, processing and mailing a non-response survey. Therefore, we will use the USGS data base that maintains the following information for all respondents in this sample:

• state/zip code,

• age (senior or not),

• type of pass purchased,

• time of year pass was purchased

We will use this information to determine if there are any statistically significant differences in the measures of central tendency between respondents and the entire population from which the sample was drawn. If statistically significant differences are found between survey respondents and the population of pass holders, survey responses will be weighted to make the respondent group mirror the entire population in terms of age, state of residence, type of pass, and time of year purchased (for example, season or quarter).

It is anticipated that the resulting survey responses from the sample (adjusted using any appropriate non-response calibration) will be statistically representative of the larger population of pass-holders from which it was drawn and thus will yield reliable data that can be generalized to the larger pass-holder population.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The draft survey instruments were reviewed NPS staff and others outside of the NPS. Suggestions on question form and content have been integrated into the draft survey instrument. These reviewers examined the surveys from both their perspectives as policy makers and as visitors familiar with recreating on federally managed lands in order to identify unclear or redundant questions, as well as to identify additional possible changes that could streamline the survey forms.

5. Provide the names and telephone numbers of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Person consulted on the statistical aspects of survey and sampling design	Dr. John Duffield Department of Mathematical Sciences University of Montana Missoula, MT 59812 (406) 243-5569
Principal Investigator for data collection and analysis	Dr. John Duffield Department of Mathematical Sciences University of Montana Missoula, MT 59812 (406) 243-5569

1 Duffield, J., Neher, C., & Patterson, D. (2006). “Wolves and People in Yellowstone: Impacts on the Regional Economy.” Yellowstone Park Foundation.

2 Dillman, D.A.and J.D. Smyth. 2014. Internet, Mail and Mixed Mode Surveys: The Tailored Design Method, forth ed. John Wiley and Sons, Inc., Hoboken, New Jersey.