Waco Mammoth National Monument Visitor Use Survey

NPS Form 10-201 (Rev. 09/2019) OMB Control No. 1024-0224

National Park Service Expiration Date 05/31/2023

PROGRAMMATIC REVIEW AND CLEARANCE PROCESS

FOR NPS-SPONSORED PUBLIC SURVEYS

The scope of the Programmatic Review and Clearance Process for NPS-Sponsored Public Surveys is limited and will only include individual surveys of park visitors, potential park visitors, and residents of communities near parks. Use of the programmatic review will be limited to non-controversial surveys of park visitors, potential park visitors, and/or residents of communities near parks that are not likely to include topics of significant interest in the review process. Additionally, this process is limited to non-controversial information collections that do not attract attention to significant, sensitive, or political issues. Examples of significant, sensitive, or political issues include: seeking opinions regarding political figures; obtaining citizen feedback related to high-visibility or high-impact issues like the reintroduction of wolves in Yellowstone National Park, the delisting of specific Endangered Species, or drilling in the Arctic National Wildlife Refuge.

SUBMISSION DATE: 6/07/2020

PROJECT TITLE: Waco Mammoth National Monument Visitor Use Study

ABSTRACT: (not to exceed 150 words)

The 2015 Presidential Proclamation establishing Waco Mammoth National Monument (WACO) called for development of a management plan “with full public involvement.” As one step in this effort, there is need to document visitor characteristics, understand visit motivations/behaviors, and identify perceptions and opinions about current visitor experiences and future facilities and programs. This project will provide data on these topics that will inform future decision making. An onsite survey and a follow-up online/mail survey of adult visitors to WACO will be used to collect the data. Seasonal sampling will occur spring/summer/fall 2021 and spring 2022. A final report will be presented to National Park Service management, Waco Mammoth Partnership member organizations, and other key stakeholders. The information gathered through this project will inform ongoing management decisions and allow managers to determine what future actions might be acceptable to visitors as they continue to consider options for the future development of WACO.

PRINCIPAL INVESTIGATOR CONTACT INFORMATION:
NAME: Christopher J. Wynveen
TITLE: Associate Professor
AFFILIATION: Baylor University
ADDRESS: One Bear Place #97313, Waco, TX 76798
EMAIL: [email protected]	PHONE:	254-710-4056

PARK OR PROGRAM LIAISON CONTACT INFORMATION:
NAME: Justin Bates
TITLE: Deputy Superintendent
AFFILIATION: Waco Mammoth National Monument / Lyndon B. Johnson National Historical Park
ADDRESS: P.O. Box 329, Johnson City, TX 78636
EMAIL: [email protected]	PHONE:	830-868-7128 x239

PROJECT INFORMATION:

Where will the collection take place?
Sampling Period Start Date: Spring 2021		Sampling Period End Date: Summer 2022
Type of Information Collection Instrument: (Check ALL that Apply)
X Mail-Back Questionnaire Only upon request as an alternative to online survey  Face-to-Face Interview	X On-Site Questionnaire  Focus Groups		 Telephone Survey  Other (List)
Will an electronic device be used to collect information?		 No X Yes – iPads for onsite survey instrument; Qualtrics website for online follow-up survey

SURVEY JUSTIFICATION:

Social science research in support of park planning and management is mandated in the NPS Management Policies 2006 (Section 8.11.1, “Social Science Studies”). The NPS pursues a policy that facilitates social science studies in support of the NPS mission to protect resources and enhance the enjoyment of present and future generations. This policy also mandates that social science research will be used to provide an understanding of park visitors, the non-visiting public, gateway communities and regions, and human interactions with park resources. Such studies are needed to provide a scientific basis for park planning and development. We are authorized to collect this information by the National Park Service Protection Interpretation and Research in System (54 USC §100702).

Waco Mammoth National Monument was established in 2015. Since 2015, visitation has increased five-fold, from 20,551 to 111,331 in 2019 (https://irma.nps.gov/STATS/Reports/Park/WACO). Given the relatively new nature of the unit, the programs and facilities are still in their early stages of development. Moreover, along with an increase in total number of recreational visitors, there has been a broadening in the set of visitor group types that are experiencing WACO. These realities along with other growing pains (e.g. parking congestion, perceived visitor impacts on resources and experiences, and increased tourism to City of Waco) create an urgent need to document visitor characteristics in more detail, understand visit motivations and behaviors, and identify perceptions and opinions about current visitor experiences and future facilities and programs. To date, little is known about visitation to WACO, except for the volume, broad age categories, and home city of the visitors. Additionally, WACO is unique in that it is one of a small number of NPS units managed as a partnership (NPS, City of Waco, Baylor University, and supported by the Waco Mammoth Foundation). WACO managers and other partnership leaders have requested this information to assess current operations and plan for future visitor use; planning efforts and programming decisions are currently challenged by the lack of data cited above.

Hence, this project aims to provide a scientific basis for park planning, operations and management, and interpretive activities at Waco Mammoth National Monument (WACO). This information is critical as the Waco Mammoth Partnership seeks to obtain baseline information to aid in decision-making processes and evaluate potential scenarios for development of the site and programs.

Specifically, park managers and Waco Mammoth Partnership leadership have four primary objectives for the project:

1. Identify the sociodemographic and other characteristics (e.g. group type and size, visitation length, and language preference) of visitors to WACO. This information will allow managers to gauge who is visiting WACO. Partners are strongly interested in increasing the share of local, repeat visitors and continuing to diversify the park’s visitor base. With this information, managers can identify underserved groups and better develop programs and facilities to attract the broadest array of the public possible. Because this is a relatively new park unit, understanding this information will also allow the park to study how visitor demographics and characteristics change over time in response to park development and programmatic changes.

2. Understand motivations for visitation and trip characteristics of visitors to WACO. This information will help managers better understand the reasons why people visit, the sources visitors use to plan their trip, and the synergies that exist between the motivation to visit WACO and other amenities and attractions in the City of Waco area. WACO is operated in partnership with the City of Waco, Baylor University, and the Waco Mammoth Foundation, and these entities are very interested in gathering this information through this effort. This will help the partnership better reach future visitors through target marketing program design, community engagement and outreach, and other activities.

3. Document visitors’ perceptions of their experience while visiting WACO. The visitor experience and public facilities at the Monument have remained largely unchanged since the City of Waco opened the site over a decade ago, even though visitation has dramatically increased. As the partnership works to further develop programs and facilities at this relatively new park unit, this information is necessary to understand strengths and deficiencies with the current visitor experience. This information, in turn, is necessary to inform partners’ discussions about how to improve the visitor experience and facilities at WACO; findings will highlight needed adjustments to current facilities and programs.

4. Identify visitor characteristics, perceptions, values, and individual uses and activities (those currently offered and those desired). This information will allow managers to better understand increasing visitation trends and changing visitor interests, identify the need for new or improved facilities and programs, and anticipate changes in the future.

In sum, this information will serve as a critical input as the park and its partners embark on development of a general management plan for this new park unit. This information will inform ongoing management decisions and allow managers to determine what future actions might be more or less acceptable to visitors as they continue to consider options for the future development of WACO.

SURVEY METHODOLOGY

1. Respondent Universe:

All adult park visitors who visit Waco Mammoth National Monument during the sampling period.

2. Sampling Plan / Procedures:

Phase 1: On-site Survey

The on-site visitor questionnaires will be administered for a total of 28 days. Throughout the duration of the study period, the sampling days occur for one-week (seven consecutive days including week and weekend days) such that sampling will occur one assigned week during each respective season (Table 1). By including an equal number of weekend days and weekdays in each season, this study is expected to capture mixed levels of use throughout the sampling periods while also ensuring that the intensive use associated with weekend visitation is represented.

Table 1a On-Site Sampling Days

	Summer 2021	Fall 2021	Winter 2022	Spring 2022	Total
Weekday	5	5	5	5	20
Weekend Days	2	2	2	2	8
TOTAL	7	7	7	7	28

During the sampling period, a systematic random sample will be conducted. Specifically, representatives of the research team will be on-site at WACO (stationed along the trail near the exit of the Dig Shelter) to collect data from 9am to 5pm (the operating hours of WACO) during each of the sampling days. Hence, most visitors will be contacted after experiencing much of the monument and many will be on their way back to their vehicles. Given that weekend visitation averages per day (n=410) are about double that of Monday-Friday per day visitation (n=227) (based on FY2018-2019 data collected by the City of Waco generated via tour ticket sales), we plan to collect data all seven days of each one-week sampling period. We anticipate contacting at least 784 individuals.

Table 1b. Estimated number of onsite visitor contacts

	Summer 2021	Fall 2021	Winter 2022	Spring 2022	Total
Weekday	114	113	113	114	454
Weekend Days	83	82	82	83	330
TOTAL	197	195	195	197	784

During each sampling period, a research team member will approach every nth visitor/visitor group to exit the dig

Shelter. The sampling interval will be based on visitation rate the day of the sampling. Visitation rate can be estimated by historical use reported by the National Park Service (https://irma.nps.gov/STATS/Reports/Park/WACO) and through the FY2018-2019 ticket sales recorded by the City of Waco) and will be confirmed via tour ticket sales on the days leading up to each sampling period.

Upon encountering a group, the team member will identify the person in the group who is 18 or older old with the most recent birthday and ask that individual to participate in the survey. Additionally, the questions outlined below (in section E) will be asked of all visitors who refuse to participate. Reasons offered for non- participation (e.g., lack of time, lack of interest, language difficulties) will also be recorded.

Phase 2: Online/Mail Back Survey

All respondents who complete the onsite survey will be asked if they are willing to participate in a follow-up survey. We anticipate that 80% of respondents completing the on-site survey will provide an email address in order to receive a link to an online survey. If respondents would rather complete a paper survey (that includes a unique identifying number so that the paper survey and onsite survey responses can be linked), they will be provided one to mail back at their convenience. Reminders for both the online version and the paper version will follow a modified Dillman (2014) method as outlined in section C below.

Table 1c. Estimated number of onsite visitor contacts accepting on-line survey

	Summer 2021	Fall 2021	Winter 2022	Spring 2022	Total
Weekday	73	72	72	72	289
Weekend Days	54	53	53	53	213
TOTAL	127	125	125	125	502

3. Instrument Administration:

Phase I - Onsite Survey:

Participants will complete the onsite survey on an iPad (this allows for more accurate data recording and minimizes data entry costs). Paper surveys will be available upon request or for use as a backup. Research team members will be trained to answer respondent questions. Upon completing the onsite questionnaire, respondents will be informed that they will receive an email within seven days with instructions to complete the on-line survey. Mail surveys will only be given to those respondents who specifically request paper questionnaires. See attached materials for contact scripts.

During each sampling day, the research team will record counts and refusal information. Information including time, group size, and presence or absence of children in the group will be collected.

COVID procedures:

While the COVID pandemic continues, interviewers will be given a COVID test prior to the data collection periods and only be allowed to collect data with a negative test and/or after the recommended quarantine period. All team members will be provided with and required to wear face masks and use hand sanitizer. Additionally, the following precautions will be taken:

• Participants will be provided disposable face masks and hand sanitizer.

• Electronic devices and pens will be wiped with sanitizer between uses.

• At least six feet of distance will be maintained between parties, except when exchanging survey materials/devices.

Phase II - Online/postal survey:

Within one-week of visiting WACO, respondents will be sent an email invitation to participate in the online survey. A unique link will be sent to each respondent that will direct them to the survey (hosted by Qualtrics, a provider of cloud based survey software). Respondents who do not complete the survey will receive 2 follow-up reminders to do so (within 5 and 10 days, respectively, from the initial email contact). Respondents who were given a paper survey will be mailed a reminder postcard within 5 days of their WACO visit. Respondents who do not return their survey will then receive another survey 10 days after their visit. Thank you notes will be sent to all. Respondents will be given multiple ways to contact the research team. See attached materials for contact scripts.

In order to facilitate the analysis, each respondents onsite survey responses will be connected with their online survey responses via their email address (mail-back surveys will have a unique identifying number to accomplish this linking of responses).

4. Expected Response Rate / Confidence Level:

A review of 17 visitor use surveys conducted at National Park Units over the past 15 years revealed that 80-90% of visitors approached agree to participate in an onsite survey. Of those, 40-80 percent of individuals who complete an onsite survey will also complete the follow-up questionnaire. The average response rate among these previous projects for the online/postal survey has been 61%.

Visitation to WACO has been about 100,000 people each year since fiscal year 2016-17. In FY 2018-19, 56,519 adults visited the site. Hence, the approximate population for this data collection effort is 50,000 adults who visit the site over the data collection period. Based on this figure, we calculate that the minimum sample size for the Phase II online survey is 375 adult visitors. To facilitate sub-group comparisons (e.g., age group or visitation group type) we will set a goal of 627 completed onsite surveys over the course of the data collection period. We expect that 80% (n= 501) will accept the invitation for the on-line survey and 75% (n= 375) will complete and return the Phase II online/postal surveys. This will provide data with an expected margin of error of +/-4 percentage points at the 95% confidence interval.

Table 4a. Anticipated Onsite Survey Response Rates

	Total Number of Visitor Contacts	Completed Onsite Surveys (80% of contacts)	Soft Refusals (20% of contacts)	Completed Non-Response Surveys (30% of soft refusals)	Hard Refusals (70% of soft refusals)
On-site Survey	784	627	156	47	109

Table 4b. Anticipated Online/Mail Back Survey Response Rates

	Completed Onsite Surveys	On-site visitors willing to participate in following up*	First wave returned Online Surveys**	Follow up returned Online Surveys	Total completed on-line responses
Online/mail back Survey	627	501	250	125	375

*80% of completed onsite surveys

**50% of On-site visitors willing to participate in following up

5. Strategies for dealing with potential non-response bias:

Phase I Onsite non-response evaluation:

During each sampling day, the research team will record counts and refusal information. Information including time, group size, and presence or absence of children in the group will be collected. Reasons offered for non-participation (e.g., lack of time, lack of interest, language difficulties) will also be recorded.

Additionally, we will ask the following questions of visitors willing to answer two questions:

1. Is this your first visit to Waco Mammoth National Monument?

 Yes  No

2. Please indicate your agreement with the statement, “Overall, I was satisfied with my visit to WMNM.”

1 Strongly Disagree

2 Disagree

3 Neither agree nor disagree

4 Agree

5 Strongly Agree

Non-respondents will be compared to respondents using the available data and any non-response bias and potential implications for the project objectives will be reported.

Phase II Online/mail non-response evaluation:

For those email address (unique identifier code for postal surveys) that do not return completed follow-up surveys, responses to the onsite survey will be used to assess and, if necessary, deal with non-responses. Specifically, demographic variables as well as visitation frequency and visit satisfaction will be compared between respondents and non-respondents to the online/mail survey. Appropriate weighting measures will be used if significant associations are identified between the two groups.

Oversampling in either phase:

If responses indicate an oversampling of weekend versus weekday visitors, a weighting procedure will be considered during statistical analysis of the data.

6. Description of any pre-testing and peer review of the methods and/or instrument:

The survey instruments were designed by an expert panel (research team members in partnership with National Park Service personnel and WMP representatives). The resulting surveys predominately use items adapted from the NPS' "Pool of Known Questions" (greater than 90%). Hence, most of the items have been assessed and successfully used in numerous past surveys. The readability of the final survey instruments were determined to be appropriate by a subset of the panel described above. Finally, the methods and instruments were evaluated by and deemed appropriate by an external reviewer (Dr. Gerard Kyle, Texas A&M University).

BURDEN ESTIMATES

The figures concerning public burden, contained in the table below, are based on the following assumptions:

• -The initial contact at WACO will take one minute to complete.

• -Reviewing instructions for the survey and/or answering respondent questions will take one minute.

• -The 14-item onsite survey will take five minutes or less to complete.

  • Onsite non-response survey will take two minutes or less to complete.

• -The 19-item online/mail follow-up survey will take about 15 minutes to complete.

Table 4. Burden Estimates

	Completed Responses	Completion Time * (minutes)	Burden Hours (rounded up)
On-site Survey*	627	6	63
On-site non-response survey*	47	3	2
Online survey	375	15	94
Total burden requested under this ICR:	1,049		159

* Initial contact time of one minute is added to the time to complete the surveys

REPORTING PLAN:

Data and the results of the analysis will be reported in an oral presentation given to the Waco Mammoth Partnership and an internal organizational report (that will be available upon request to external entities) will additionally be generated. The report will include details about each item in the questionnaires and a summary of findings reporting on the multivariate analysis conducted. At a minimum, univariate statistical distributions for each of the survey items will be reported and selected subgroup analyses will be completed. A complete record and documentation of the data collection and management procedures will be included in the report. Finally, results may be reported in presentations given at national/ international academic conferences and in academic journal articles. In all instances and in order to maintain the confidentiality of the respondents, the data will be reported in aggregate. No information will be reported that has the potential to identify individual respondents.

NOTICES

Privacy Act Statement

General: This information is provided pursuant to Public Law 93-579 (Privacy Act of 1974), December 21, 1984, for individuals completing this form.

Authority: National Park Service Research mandate (54 USC 100702)

Purpose and Uses: This information will be used by The NPS Information Collections Coordinator to ensure appropriate documentation of information collections conducted in areas managed by or that are sponsored by the National Park Service.

Effects of Nondisclosure: Providing information is mandatory to submit Information Collection Requests to Programmatic Review Process.

Paperwork Reduction Act Statement

We are collecting this information subject to the Paperwork Reduction Act (44 U.S.C. 3501) and is authorized by the National Park Service Research mandate (54 USC 100702). This information will be used by The NPS Information Collections Coordinator to ensure appropriate documentation of information collections conducted in areas managed by or that are sponsored by the National Park Service. All parts of the form must be completed in order for your request to be considered. We may not conduct or sponsor and you are not required to respond to, this or any other Federal agency-sponsored information collection unless it displays a currently valid OMB control number. OMB has reviewed and approved The National Park Service Programmatic Review Process and assigned OMB Control Number 1024-0224.

Estimated Burden Statement

Public Reporting burden for this form is estimated to average 60 minutes per collection, including the time it takes for reviewing instructions, gathering information and completing and reviewing the form. This time does not include the editorial time required to finalize the submission. Comments regarding this burden estimate or any aspect of this form should be sent to the Information Collection Clearance Coordinator, National Park Service, 1201 Oakridge Dr., Fort Collins, CO 80525.

RECORDS RETENTION - PERMANENT. Transfer all permanent records to NARA 15 years after closure. (NPS Records Schedule, Resource Page 1 of 9

Management And Lands (Item 1.A.2) (N1-79-08-1)).