1024-0224 Programmatic Review Form - ROMO

NPS Form 10-201 (Rev. 09/2016) OMB Control No. 1024-0224

National Park Service Expiration Date XX/XX/XXXX

PROGRAMMATIC REVIEW AND CLEARANCE PROCESS

FOR NPS-SPONSORED PUBLIC SURVEYS

The scope of the Programmatic Review and Clearance Process for NPS-Sponsored Public Surveys is limited and will only include individual surveys of park visitors, potential park visitors, and residents of communities near parks. Use of the programmatic review will be limited to non-controversial surveys of park visitors, potential park visitors, and/or residents of communities near parks that are not likely to include topics of significant interest in the review process. Additionally, this process is limited to non-controversial information collections that do not attract attention to significant, sensitive, or political issues. Examples of significant, sensitive, or political issues include: seeking opinions regarding political figures; obtaining citizen feedback related to high-visibility or high-impact issues like the reintroduction of wolves in Yellowstone National Park, the delisting of specific Endangered Species, or drilling in the Arctic National Wildlife Refuge.

SUBMISSION DATE: 10/21/2020

PROJECT TITLE: Survey of Rocky Mountain National Park Visitors to Understand Visitor Experiences under a Timed Entry Permit System.

ABSTRACT: (not to exceed 150 words)

The goal of this survey is to better understand how park visitors’ experiences are influenced or changed under the Timed Entry Permit System that was developed and put into place to reduce impacts and exposure from the COVID-19 pandemic in order to keep the park open and operating while providing the safest experience possible. This survey will take place during the fall of 2020 and will be conducted through an online survey of visitors that purchased reservations.

PRINCIPAL INVESTIGATOR CONTACT INFORMATION:

Name	Scott Esser
Title	Supervisory Ecologist
Affiliation:	National Park Service
Address:	1000 US HWY 36, Estes Park, CO, 80517
Phone	970-586-1394
Email:	[email protected]

PARK OR PROGRAM LIAISON CONTACT INFORMATION:

Name	John Hannon
Title	Supervisory Management Specialist
Affiliation:	Rocky Mountain National Park
Address:	1000 US HWY 36, Estes Park, CO, 80517
Phone	970-586-1365
Email:	[email protected]

PROJECT INFORMATION:

Where will the collection take place? Rocky Mountain National Park

Sampling Period Start Date: October 2020 Sampling Period End Date: November 2020

Type of Information Collection Instrument: (Check ALL that Apply)

Mail-Back Questionnaire Face-to-Face Interview Focus Groups

On-Site Questionnaire Telephone Survey

Other (List) Online Survey

Will an electronic device be used to collect information? No Yes – Type of Device: Respondents’ personal devices (including smartphones, laptops, desktop computers, tablets, etc.)

SURVEY JUSTIFICATION:

Social science research in support of park planning and management is mandated in the NPS Management Policies 2006 (Section 8.11.1, “Social Science Studies”). The NPS pursues a policy that facilitates social science studies in support of the NPS mission to protect resources and enhance the enjoyment of present and future generations (National Park Service Act of 1916, 38 Stat 535, 16 USC 1, et seq.). NPS policy mandates that social science research will be used to provide an understanding of park visitors, the non-visiting public, gateway communities and regions, and human interactions with park resources. Such studies are needed to provide a scientific basis for park planning and development.

Rocky Mountain National Park (ROMO) visitation has exceeded over 4.6 million visitors annually during the peak visitation months, July through September, that see over 35,000 visitors to the park. During this time park managers have noted shifting patterns in visitation and severe congestion on trails, roadways, in parking lots, entrance stations and visitor centers.

Due to the recent COVID-19 pandemic the park put into place a Timed Entry Permit System (TEPS) to help provide a safe experience to park visitors and staff. To address the information needs of park managers and ensure the safety of both staff and visitors, the park will implement a visitor use study/survey of park visitors who visited during the implementation of the TEPS. The purpose of this study is to capture visitor values, expectations for visits, and understand how visitors perceive their experience in the park under the TEPS. The results will better inform park managers on visitor perceptions of the TEPS visitor use management strategy.

These data will be used to understand park use, impression of park management, and provide insight and understanding on the experiences of visitors who obtained permits during the TEPS. These results could provide a comparison to previously recorded visitors’ experiences that came under periods without timed entry permits. Specifically, this study will collect the following data about the visitors using TEPS:

• Demographics: to develop a clearer profile of visitors who used the TEPS and visited during the COVID-19 pandemic.

• Values and Preferences: ROMO has limited information about visitor values and preferences and no information on visitor values and preferences under the TEPS.

• Expectations and Experiences: ROMO has limited perspective on the expectations of park visits and whether visitors are happy with their current experiences. The survey is intended to provide insight for what might improve visitor experiences in the future within possible change parameters.

SURVEY METHODOLOGY:

1. Respondent Universe:

The respondent universe will be limited to all people, 16 years and older, who have purchased a permit through the Recreation.gov system. Four groups of people will be missed in the respondent universe: 1) visitors who entered the park outside of the TEPS hours (5 p.m. to 6 a.m.), 2) visitors who visited the park during the TEPS but were not a permit holder, 3) visitors who would like to visit the park but could not obtain a permit, and 4) the non-visiting public. Doing an in-park survey is not possible due to COVID-19 restrictions and procedures.

2. Sampling Plan / Procedures:

This survey will be administered via email. Surveys will be sent out to two groups using email addresses provided when purchasing a TEPS reservation through Recreation.gov. Recreation.gov will automatically send a survey link to registered users that made a timed entry reservation. The TEPS was run for the period from June 4, 2020 to October 12, 2020. With an average of 3,900 reservations per day for the 131 period, the estimated number of permits sold during the sampling period is 510,900. Our goal will be to get a 1% (n=5,100) completed surveys. To ensure that we hit the 1% response rate we anticipate the research team will monitor responses as they are received. Reminder emails will be sent to sample participants. The survey will be open for X weeks for both sample groups.

Table 1: Sample Target Completed by Peak Survey Month

Peak Survey Months	Anticipated number of reservations	Target Number of Completed Surveys
June	115,800	1,158
July	115,800	1,158
August	115,800	1,158
September	115,800	1,158
October	46,800	468
Total	510,000	5,100

3. Instrument Administration:

The visitor survey will be hosted online with a link sent out through Recreation.gov. Recreation.gov will email a survey link through its on-line system. Visitors will receive the survey link via the email addresses provided when purchasing their permit. Survey respondents will have no time limit to open and reply. If a visitor has purchased multiple reservations throughout the year they will only be allowed to respond to the survey once. This will be achieved by only sending one survey link to each email in the list. For example, if an email address was used to purchase 14 permits only one survey link will be sent to that email address versus 14 separate survey links. Follow up reminders (emails) will be sent to all non-respondents. The survey will be hosted on Qualtrics or similar electronic survey software platform.

The survey platform will be live for a four-week period. A tracking page will be used to monitor survey results in real time, including survey starts, completions, and, to ensure that the survey is working as intended.

4. Expected Response Rate / Confidence Level:

We are expecting at least a 1% response rate which would be about 5,000 responses for the on-line survey.

Table 3: Estimated Response Rates by Platform Type

Platform Type	Initial Contacts	Completion 1%	Non-respondents 99%
Rec.gov	510,000	5,100	495,900

5. Strategies for dealing with potential non-response bias:

To address non-response bias our methods will include dividing the sample population into two groups. An initial email of surveys will be sent to half of the permit holders emails followed by a subsequent emailing to the remaining permit holders. Conducting a statistical comparison of these two groups responses should reveal if any non-response bias occurred. This is considered a standard test for non‐response bias through assuming the second group of respondents who return the subsequent mailings are a sample of non‐respondents to the first mailing and are a representative sample of that group.

Since this is an online survey there are few strategies, we can use to deal with non-response bias.

1. Send a follow up email(s) to respondents that have not completed the survey. (First/The) follow up email will be sent three after the initial survey invitation. The second response reminder will be sent one week before the end of the sampling period.

2. The survey population will be divided into two groups. Conducting a statistical comparison of these two groups responses should reveal if any non-response bias occurred. This is considered a standard test for non‐response bias through assuming the second group of respondents who return the subsequent mailings are a sample of non‐respondents to the first mailing and are a representative sample of that group.

3. Since the goal is to understand visitor experience during the implementation of the TEPS we will only be interested in responses of people that participated in the TEPS program. There will be no responses needed from the population that did not visit the park during the TEPS.

4. While higher response rates do not guarantee minimal nonresponse error, they do reduce the likelihood of nonresponse error and thus nonresponse bias Dillman et al. 2014). While we are assuming a conservative response rate of 1%, we will be implementing a strategy to improve response rates which includes sending follow-up reminders to all participants. Additionally, response rates are likely to be influenced by interest in the topic, or salience (Dillman and Bowker 2000).

5. Finally, nonresponse on email surveys may also stem from mechanical issues, frustration with questionnaire design, or incompatibilities between the respondent’s device and the questionnaire design (Dillman and Bowker 2000, Vicente and Reis 2010, Atif et al. 2012). To understand the ease of use of our survey we conducted internal pre-testing of the survey before the launch to help address the potential for mechanical issues.

6. Description of any pre-testing and peer review of the methods and/or instrument:

The survey questions were tested with less than 9 people within the local community and peer reviewed through extensive communication over several weeks with park staff and staff from the National Park Service Social Science Branch. The survey was pre-tested to ensure functionality on different screen configurations, operating systems, browsers, and partial screen displays. Clear, specific instructions will be provided for each question to ensure that respondents understand how to take the computer action necessary to respond to each question (Dillman and Bowker, 2000).

BURDEN ESTIMATES:

We plan to email all visitors that purchased reservations through Recreation.gov. Based on an estimation of 510,000 purchased reservations and our 1% response rate we expect about 5,100 people to complete the survey. The estimated respondent burden for each survey is estimated to be about 15 minutes. This includes reading the email, navigating to and completing the survey questions. The total estimated respondent burden for this collection is 1,312 hours.

Table 4. Burden Estimates

	Responses	Completion Time * (minutes)	Burden Hours (rounded up)
Completed questionnaire	5,100	15	1,275
Partial non-response survey	438	5	37
Total burden requested under this ICR:	5,538	-	1,312

REPORTING PLAN:

Results, including descriptive statistics (frequency distributions, duration of trip, visitor experiences, and impressions of safety), will be presented to ROMO leadership and published in the form of a Natural Resource Technical Report.

NOTICES

Privacy Act Statement

General: This information is provided pursuant to Public Law 93-579 (Privacy Act of 1974), December 21, 1984, for individuals completing this form.

Authority: National Park Service Research mandate (54 USC 100702)

Purpose and Uses: This information will be used by The NPS Information Collections Coordinator to ensure appropriate documentation of information collections conducted in areas managed by or that are sponsored by the National Park Service.

Effects of Nondisclosure: Providing information is mandatory to submit Information Collection Requests to Programmatic Review Process.

Paperwork Reduction Act Statement

We are collecting this information subject to the Paperwork Reduction Act (44 U.S.C. 3501) and is authorized by the National Park Service Research mandate (54 USC 100702). This information will be used by The NPS Information Collections Coordinator to ensure appropriate documentation of information collections conducted in areas managed by or that are sponsored by the National Park Service. All parts of the form must be completed in order for your request to be considered. We may not conduct or sponsor and you are not required to respond to, this or any other Federal agency-sponsored information collection unless it displays a currently valid OMB control number. OMB has reviewed and approved The National Park Service Programmatic Review Process and assigned OMB Control Number 1024-0224.

Estimated Burden Statement

Public Reporting burden for this form is estimated to average 60 minutes per collection, including the time it takes for reviewing instructions, gathering information and completing and reviewing the form. This time does not include the editorial time required to finalize the submission. Comments regarding this burden estimate or any aspect of this form should be sent to the Information Collection Clearance Coordinator, National Park Service, 1201 Oakridge Dr., Fort Collins, CO 80525.

RECORDS RETENTION - PERMANENT. Transfer all permanent records to NARA 15 years after closure. (NPS Records Schedule, Resource Page 1 of 7

Management And Lands (Item 1.A.2) (N1-79-08-1)).