1024-NEW SEM Supporting Statement B 2-25-25

Supporting Statement B

Socioeconomic Monitoring (SEM) Study of National Park Service Visitors

OMB Control Number: 1024-NEW

Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection methods to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Each year, the socioeconomic monitoring program will survey visitors at 30 National Park Service (NPS) sites; visitors to these sites represent the target population for this collection. This is inclusive of National Parks, Historic Sites, Memorials, Parkways, Seashores, and other similar units (we will use ‘parks’ or ‘units’ as a generic name for all units contained in this supporting statement). This collection will establish an ongoing, annual, survey of parks that collects information at the unit level and is consistent across all parks at a national level. Park selection is stratified across (1) size (based on visitation) and (2) type (Nature, Historic, and Recreation). A survey year consists of those parks surveyed during their peak visitation month within a calendar year.

The SEM program includes a two-phased survey approach within each selected park unit that includes a 1) intercept survey with eligible visitors on-site at the park unit, and 2) a mail-back survey (with online option) that is handed to visitors who participated in the intercept survey on-stie. Specific survey administration dates will be selected to target peak use at each park. Peak use will be considered those months within 10% of the peak month, based on a five-year average. Conducting survey administration during peak use at each park unit maximizes sampling efficiency and captures a broad range of use and users. This approach helps reach as close to a representative sample as possible and allows for standardization across parks. Months with the most visitors are also more likely to have enough survey participants that engage in various activities of interest and fit different group-type segments (e.g., non-local overnight, local day-use, etc.) to allow for statistically sufficient data analysis on these topics. Further, sampling during peak visitation should provide enough participants for two park types: 1) parks with low overall visitation, and 2) parks with seasonally low visitation. The latter category of parks can have extremely low or non-existent visitation across multiple months of undesirable conditions or inaccessibility.

To provide an example of this process, the 30 parks selected for the 2024 survey year are shown throughout this supporting statement. In 2022, a systematic sampling methodology was finalized to determine which park units would be sampled annually. This methodology is described in question two.

Using average annual (2013-17) NPS Visitation Statistics (https://irma.nps.gov/Stats/Reports/Park), total annual visitation to the targeted 30 SEM parks was calculated to be 45,519,921 (see Table 1.1). The five-year average from 2013-17 was used, as this was the time-period used for park group segmentation into their respective stratum. Each park will be targeted for sampling during its peak month or alternate months within 10% of the peak month visitation. Peak monthly average visitation volumes are shown in Table 1.1. Standard survey periods will cover consecutive ten-day periods within this peak month. Average daily visitation in peak months is calculated based on the number of days in the peak month (30 or 31). Respondent universe is then represented by the estimated number of visitors at each unit during the sampling period. Using this method, we estimate that the total annual respondent universe for the collection is 2,045,493. Calculations for each of the 30 parks in 2024 are shown below in Table 1.1. Given the systematic stratification of NPS units for annual sampling, while the respondent universe may fluctuate slightly from year-to-year, the number of intercepts, responses and annual burden will remain consistent.

Table 1.1: Respondent Universe

	NPS Unit	Park Size	Annual Visits (2013-2017)	Peak Month Visits (2013-2017)	Average Daily visits in peak month	Respondent Universe (10 days)
Park Type: Nature
1	Mammoth Cave National Park	Small	551,687	108,686	3,506	35,060
2	Organ Pipe Cactus National Monument	Small	229,771	33,852	1,092	10,920
3	John Day Fossil Beds National Monument	Small	192,129	33,110	1,068	10,681
4	Great Smoky Mountains National Park	Large	10,563,665	1,448,692	46,732	467,320
5	Grand Canyon National Park	Large	5,413,278	748,350	24,140	241,403
6	Hawai'i Volcanoes National Park	Large	1,802,632	177,386	5,722	57,221
7	El Malpais National Monument	Small	159,075	18,819	627	6,273
8	Arches National Park	Large	1,378,326	184,980	6,166	61,660
Subtotal			20,290,563	2,753,875	89,053	890,538
Park Type: Historic-Urban
9	Casa Grande Ruins National Monument	Small	70,550	13,037	421	4,205
10	Salem Maritime National Historic Site	Small	368,097	101,569	3,276	32,764
11	Roger Williams National Memorial	Small	61,451	9,697	313	3,128
12	Boston National Historical Park	Large	2,600,978	416,248	13,427	134,274
13	Minute Man National Historical Park	Large	1,008,422	117,084	3,777	37,769
14	Kennesaw Mountain National Battlefield Park	Large	2,234,387	243,666	7,860	78,602
15	Fort McHenry National Monument and Historic Shrine	Small	653,694	82,411	2,747	27,470
16	President's Park (White House grounds)	Large	950,497	240,735	8,025	80,245
Subtotal			7,948,076	1,224,447	39,846	398,457
Park Type: Historic-Non-Urban
17	Grant-Kohrs Ranch National Historic Site	Small	21,760	4,345	140	1,402
18	Fort Larned National Historic Site	Small	28,893	4,325	144	1,442
19	Homestead National Monument of America	Small	85,290	16,225	523	5,234
20	Vicksburg National Military Park	Large	528,230	63,679	2,054	20,542
21	Little Bighorn Battlefield National Monument	Large	301,110	71,870	2,318	23,184
22	Wilson's Creek National Battlefield	Large	173,881	21,747	702	7,015
Subtotal			1,139,164	182,191	5,881	58,819
Park Type: Recreation
23	Padre Island National Seashore	Small	592,340	99,138	3,198	31,980
24	Bluestone National Scenic River	Small	35,522	8,818	284	2,845
25	Amistad National Recreation Area	Small	1,158,294	135,443	4,369	43,691
26	Point Reyes National Seashore	Large	2,494,394	264,170	8,522	85,216
27	Indiana Dunes National Lakeshore	Large	1,746,390	326,270	10,525	105,248
28	Lake Mead National Recreation Area	Large	7,128,858	824,861	26,608	266,084
29	Saint Croix National Scenic Riverway	Small	623,674	205,659	6,855	68,553
30	Rock Creek Park	Large	2,362,646	282,188	9,406	94,062
Subtotal			16,142,118	2,146,547	69,767	697,679
Totals			45,519,921	6,307,060	204,547	2,045,493

Response Rate

For completed mail-back survey targets, there are two categories of parks: Normal Effort Units and Increased Effort Units. For 27 of the parks, the research team anticipates receiving approximately 400 returned mail-back surveys from each park to meet sample size needs; these are considered Normal Effort Units. Normal Effort Units will yield approximately 10,886 total mail-back surveys annually.

For the remaining three parks, we will target approximately 800 returned mail-back surveys per park with an estimated total of 2,411 mail-back surveys annually. These three parks are considered Increased Effort Units. The larger return goal is set to increase the precision and power to best estimate visitor segments within parks. The Increased Effort Units will be identified based on the historical understandings of the complexity of use and likely need for multiple visitor type segmentations. For example, potential units may include those like Natchez Trace Parkway (an Increased Effort Unit in the Pilot Phase II), in which the parkway covers an expansive 444-mile scenic drive across multiple states, or Golden Gate National Recreation Area, which cover numerous sub-units within the San Francisco Bay area.

Combined, the Normal and Increased Effort Units are expected to yield 13,298 mail-back surveys annually. This breakdown of 27 Normal Effort Units and 3 Increased Effort Units will remain the consistent from year-to-year.

Based on the Pilot Phase II (Socioeconomic Pilot Survey, Phase II; OMB Control #1024-0224; ex. 5/31/2023) results using the same methods, an 80% response rate is expected for the Intercept Survey. Combined, the Normal and Increased Effort Units will yield 46,170 visitor contacts and 36,936 completed intercept surveys. To achieve the goal of 400 mail-back surveys per Normal Effort Unit, 1,400 visitor contacts will be made on-site, per park. Of these contacts 1,120 visitors are expected to participate in the intercept survey per park. This yields an annual total of 30,240 completed intercepts, requiring 37,800 total contacts for Normal Effort Units. For the Increased Effort Units to achieve the goal of 800 returned mail-back surveys per unit, 2,790 visitor contacts will be made on-site per park. Of these contacts, 2,232 visitors are expected participate in the intercept survey per park. This yields an annual total of 6,696 completed intercepts, requiring 8,370 contacts per Increased Effort Units. 

Of those 20% (n=9,234) who do not agree to take the intercept survey, pilot studies suggest 90% (n=8,310) will answer the non-response questions (soft refusals) and 10% (n=924) will fully refuse (hard refusal). Further, of those taking the intercept survey, 90% (n=33,243) will accept taking the mail-back survey. Of those taking the mail back, 40% (n=13,298) will complete it – each of these components of response rate is described in further detail in Tables 1.2 and 1.3 below. 

Table 1.2: Anticipated Onsite Survey Response Rates

Effort Category	Number of Visitor Contacts	Completed Onsite Surveys (80% of contacts)	Refusals (20% of contacts)	Soft Refusal Completed   Non-Response (90% of refusals)	Hard Refusals (10% of refusals)
Normal Effort (27 parks)	37,800 (1,400/park)	30,240 (1,120/park)	7,560 (280/park)	6,804 (252/park)	756 (28/park)
Extra Effort (3 parks)	8,370 (2,790/park)	6,696 (2,232/park)	1,674 (558/park)	1,506 (502/park)	168 (56/park)
Total (All Parks)	46,170	36,936	9,234	8,310	924

 

The following estimates for the mail-back/on-line survey is assumed based on visitor contacts from the intercept survey. All visitors who are given a mail-back/online survey will have already taken the intercept survey, which includes non-response bias questions. Thus, there is no extra effort necessary to collect non-response bias responses from respondents who do not participate in the mail-back/online survey (Table 1.3). 

Table 1.3: Anticipated Follow-up Survey Response Rates 

Effort Category	Completed    Onsite Surveys   (see Table 1.2)	Accepted Follow-up Surveys   (90% of onsite completes)		Completed Follow-up Surveys   (40% of accepted surveys)	Follow-up Survey Non-respondents   (60% of accepted surveys)
Normal Effort   (27 parks)	30,240 (1,120/park)	27,216 (1,008/park)	10,887  (403/park)		16,329 (605/park)
Extra Effort   (3 parks)	6,696 (2,232/park)	6,027 (2,009/park)	2,411  (804/park)		3,616 (1,205/park)
Total   (All Parks)	36,936	33,243	13,298		19,945

 

2. Describe the procedures for the collection of information including:

• 2.1 Statistical methodology for stratification and sample selection,

• 2.2 Unusual problems requiring specialized sampling procedures

• 2.3 Estimation procedure, and degree of accuracy needed for the purpose described in the justification, and

• 2.4 Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Within the above key bullets, the sections that follow include several important aspects of this information collection:

• Annual site selection;

• Sampling of visitors within standard sites;

• Sampling of visitors within non-standard (small volume) sites;

• Estimation procedures deployed for payment card question

2.1 Statistical methodology for stratification and sample selection

2.1.1 Annual Site Selection

The sites for this collection are selected based on a randomized selection of park units, stratified by unit type and size in order to achieve a representative sample with results that are generalizable across the NPS System. A blend of natural/urban/historic sites, along with units of varying annual visitor volumes are included in the sample. The sites to be considered for sampling fit into eight mutually exclusive strata based on a matrix of unit type and area population class (four categories) and visitation volume (two categories). Each qualifying NPS unit within the sampling frame is assigned to a stratum using these classifications. To qualify for sampling, a park unit must have five years of available visitor use statistics, such that they may be appropriately assigned to the high or low visitation strata. As of January 2025, there are 433 units of the National Park System and 387 of these have the required five complete years of reported visitor use stats available. This means just over 10% of NPS units are not eligible. Using a random number generator, a proportional number of parks are randomly selected from each of the eight strata to create a stratified random sample of the parks to be sampled. Those parks are listed in Table 1.1 above and are proportional to the eight strata. 

2.1.1.a Park Unit Type & Class (four categories) 

Each unit is designated into one of three types (nature, historic, recreation) adapted from Haefele, Loomis, & Bilmes, 2016; historic parks are further subdivided into urban and non-urban based on NPS Visitor Use Statistics (VUS) population class, resulting in four possible designation categories.

• (1) Nature – National Park units that focus on the preservation of natural environments and features, shorelines, and bodies of water. 

• Historic – National Park units that focus on the preservation of American history and culture or the commemoration and remembrance of significant events and people. 

• (2) Historic Urban – historic units with a population class of Urban; Suburban; or Mixed, with most of the surrounding population class considered Urban or Suburban. 

• (3) Historic Non-urban – historic units with a population class of Rural; Outlying; Remote; Mixed, with most of the surrounding population class considered Rural or Outlying; or No Boundary Data. 

• (4) Recreation – National Park s that focus on nature-based recreation opportunities. 

2.1.1.b Park Unit Size (two categories) 

The list of park units within each of the four Unit Type and Population Class categories described above (nature, historic urban, historic non-urban, recreation) was sorted in descending order (highest to lowest) by average annual visitation across the five-year period from 2013 through 2017. Within each category, the top n parks that together account for 80% of the category’s total annual visitation are designated “large” parks, with the remaining parks designated “small” parks. 

2.1.2 Sampling of Visitors within Standard Park Sites  

At each site, visitors will be randomly contacted onsite to participate in a short, 6-minute, intercept survey about their current NPS trip. Verification of potential respondent eligibility will be first established, ensuring they are at least 18, have not already taken the survey at the specific park, and are not a park employee. Where groups are encountered, the adult with the most recent birthday will be the requested respondent. The same visitor will be asked to complete a follow-up survey (mail-back or online) upon completion of their trip. The follow-up survey will capture details about the respondents’ trip and time at the NPS site.  The only variations that will occur will be based on park logistics for intercepting visitors and specific park attributes and needs within questions. For example, smaller parks with a single entrance may only require sampling at one entry/exit point, whereas other parks will have multiple intercept locations. Similarly, individual questions on the surveys will be made applicable to park circumstances. For example, questions about mode of transportation for entering the park will only reflect those options available at a park (e.g., land-based parks won’t have an option for entering by watercraft, etc.).

The park type and size strata described above are exclusively used to select the park sites for inclusion in the study in each year; the sampling plan, instruments, and procedures used to contact visitors within each park will remain consistent across all park units.  

A random sampling of visitors will be intercepted while visiting one of the selected NPS sites during a consecutive 10-day sampling period. Intercept times of day will be varied by park and established via communication with park staff regarding typical visitation hours. Typically, 3-5 surveyors will be stationed at specific intercept locations within each NPS unit (e.g., visitor centers, attraction areas, trailheads, and near park entrances) based on insights from park staff, NPS visitor use statistics, prior research, and professional experience. Depending on park circumstances, these surveyors may be spread out across locations or stationed at a single location (e.g., single entry/exit point). An example 10-day visitor contact schedule is displayed below for Great Smoky Mountains National Park (Table 2.1).

Table 2.1: Example ten-day sample schedule

	Number of initial contacts: Great Smoky Mountains NP
	Sat	Sun	Mon	Tue	Wed	Thu	Fri	Sat	Sun	Mon	Site Total
Sugarlands VC	X	X	X	X	70	70	X	X	X	X	140
Gatlinburg Entrance	70	70	X	40	X	40	40	X	40	70	370
Townsend Entrance	X	X	70	X	X	X	X	40	X	X	110
Cherokee Orchard Turnaround	X	X	X	30	X	30	X	30	X	X	90
Greenbrier	X	X	X	X	X	X	30	X	30	X	60
Cherokee Entrance	70	X	70	X	70	X	70	70	X	70	420
Cataloochee	X	70	X	X	X	X	X	X	X	X	70
Deep Creek	X	X	X	70	X	X	X	X	70	X	140
Daily Total	140	140	140	140	140	140	140	140	140	140	1,400

Surveys that require intercepting visitors in vehicles will be conducted by safely flagging them into a designated (traffic-coned off) survey area. Site safety measures will be reviewed with appropriate park staff at each site to ensure both visitor and surveyor safety. Surveyors will be instructed to attempt to intercept every nth vehicle passing based on the anticipated volume and number of visitor contacts required at each NPS unit. Where surveying requires intercepting individuals on foot or otherwise outside of their vehicles, visitors traveling past the intercept locations or within the designated survey area will be randomly approached. Surveyors will be instructed to attempt to intercept every nth group passing based on the anticipated volume and number of visitor contacts required at each NPS unit.  Again, the adult with the most recent birthday will be requested.

2.2 Unusual problems requiring specialized sampling procedures

Sampling of visitors within non-standard (small volume) sites

As can be viewed in Table 1.1, several parks have rather small visitation levels, even in peak visitation months. As such, changes to the standard 10-day sampling window must be made. For example, Grant-Kohrs Ranch National Historic Site (#17 from Table 1.1) has a total average annual visitation of 21,760 and a peak month of 4,345 visitors. This yields only 1,402 total visitors contained in the standard estimate of the respondent universe. Given protocols call for surveying only one member per group, and limited to adults, this volume would not meet the targeted 1,400 contacts as previously outlined. To accommodate small visitation levels, the sampling period will be extended beyond the 10-day period, while attempting to stay within the peak month(s) or those within 10% of the peak month. Further, surveyors on site at any given time will be reduced and spread across the increased days. The total duration and number of surveyors will be site specific for these non-standards parks. Deviations from the standard calculation of the respondent universe will be reported in park-specific final reports.

A sampling schedule for a small park such as Grant-Kohrs is displayed below. It is preliminarily estimated that the onsite time for this park would be 20 days, thus increasing the estimated respondent universe to 2,804. As can be seen in Table 2.2, it is expected that roughly half the typical daily volume of visitor contacts will be made (and half the number of surveyors (i.e., 2) will be on-site at any one time), but the overall number of contacts will remain consistent with those in the standard 10-day sample (n=1,400).

Table 2.2: Example Schedule for low-volume park requiring extended surveying period.

	Number of initial contacts: Grant Kohrs Ranch NHS
	Sat	Sun	Mon	Tue	Wed	Thu	Fri	Sat	Sun	Mon	Site Total
Grant Kohrs VC/Parking	70	70	70	70	70	70	70	70	70	70	700
	Sat	Sun	Mon	Tue	Wed	Thu	Fri	Sat	Sun	Mon
Grant Kohrs VC/Parking	70	70	70	70	70	70	70	70	70	70	700
TOTAL											1,400

2.3 Estimation procedure, and degree of accuracy needed for the purpose described in the justification

For 27 of parks contained in the collection effort, the research team will target approximately 400 returned mail-back surveys to meet sample size needs (Sample Equation Below); these are considered Normal Effort Units.

The example sample size (n) using Great Smoky Mountains National Park was calculated using to the following formula¹:

Unlimited Population -

Finite Population Correction -

Where:

• Z score (based on 0.95 Confidence Level): z = 1.96

• Population Proportion: = 0.5

• Population Size: N = 467,320

• Margin of error: ɛ = 0.05

Given the equations and values above the finite population correction n’ = 384.

For the remaining three parks, we will target approximately 800 returned mail-back surveys per park; these are considered Increased Effort Units. A discussion Normal and Increase Efforts Units was included in question 1 above. In both cases, these sample sizes are large enough to achieve a 95% confidence level with sampling errors at +/- 5% for each park unit (or subsections within Increased Effort Units). For human dimensions of natural resource studies, this confidence level and sampling error is sufficient for creating reliable data that can be generalized to the larger population of interest (Vaske, 2019).

2.3.1 Willingness to Pay (WTP) Question

The exception to the above estimation and degree of accuracy information is the willingness to pay question. The question, its method of estimation, and associated justification is presented below:

Willingness to pay (WTP) questions are used to determine the overall valuation respondents place on their visit to an NPS unit. No other question contained in the survey can effectively achieve the level of economic information as the proposed WTP question. The information collected in this question will provide the NPS with a more complete understanding of the value park visitors place on their trips. Furthermore, it will provide NPS managers with vital information leading to a demonstration of return on investment in preserving and maintaining various park units. The question format used (dichotomous choice) is a familiar construct that imposes little in the way of cognitive burden on respondents. Two sets of bid amounts were developed for this question - one characterized by slightly higher dollar amounts ranging from $5 to $2000 to be asked at parks that have overnight use (version A), and another with slightly lower dollar amounts ranging from $5 to $750 to be asked at parks that have primarily day use (version B). The full list of bid amounts for each version (A and B) are also included in the grayed annotation box preceding Question 17 on the intercept survey.

Version A of the question (for parks that have overnight use) is as follows:

QUESTION: As you know, some of the costs of travel such as gasoline, hotels, rental cars, and airline tickets often increase. If your share of the total trip costs were $X more, would you still have taken this trip to [NPS site]? Please mark (●) one.

• Yes, I would still take this trip.

• No, I would not take this trip.

[The $X would be randomly filled in with one of ten bid amounts ranging from $5 to $2,000 - specifically,

$5, $20, $50, $100, $200, $350, $500, $750, $1,200, $2,000]

Version B of the question (for parks that have primarily day use) is as follows:

QUESTION: As you know, some of the costs of travel such as gasoline, hotels, rental cars, and airline tickets often increase. If your share of the total trip costs were $X more, would you still have taken this trip to [NPS site]? Please mark (●) one.

• Yes, I would still take this trip.

• No, I would not take this trip.

[The $X would be randomly filled in with one of ten bid amounts ranging from $5 to $750 - specifically, $5, $20, $35, $50, $75, $100, $200, $350, $500, $750].

The above question, found on the intercept survey instrument, immediately follows a question about the number of people splitting the trip expenses. As such, it is readily apparent to the respondent that the trip is the commodity to be evaluated and that the listed values are related to current trip expenditures. Key considerations for the appropriate deployment and scaling of the WTP question include:

• Consideration of the wide range of expected total trip expenses. This expectation is based on if respondent is local, length of trip, type of park unit visited, and group size, among other visitor attributes. As such, possible bid amounts presented must cover reasonable ranges for a variety of trip types and initial trip costs.

• Consideration of response formats that are incentive compatible.

• Consideration of how many bid amounts to include and the nature in which the values increase.

The proposed WTP question addresses the above considerations. Following best practices, a dichotomous choice response format is used to address issues of incentive compatibility (Boyle, 2017; Johnston et al., 2017). One of ten bid amounts is randomly selected and presented to each respondent, ranging from $5 to $2,000 in version A and $5 to $750 in version B. A response of ‘yes’ indicates that the true WTP of the respondent is some amount greater than or equal to the bid amount presented. This relationship extends to all indicated responses; for respondents that answer ‘yes’ to the WTP question, their true WTP is greater than or equal to the present bid amount. For respondents that answer ‘no’ to the WTP question, their true WTP is less than the presented bid amount. More formally, following Haab and McConnell (2002), a linear WTP function can be defined as:

WTP (z_i, η_i) = γz_i + η_i (1)

where γ and z_i are a matrix of parameters and covariates associated with respondent i and η_i is symmetric, independent and identically distributed with mean zero. The respondent answers ‘yes’ to an offered bid A_j if WTP_i > A_j. When η is N(0, σ²) it can be converted to a standard probit. The probability of a yes response is:

Pr(yes_i) = Pr(WTP_i > A_j) = Pr((γz_i – A_j) > η_i) = Pr(γz_i – A_j) / σ > θ_i (2)

where θ_i is N(0,1). In this dichotomous choice model, (z, A) are covariates with coefficients γ/σ, -1/σ. An estimate of γ can be obtained using the estimated coefficients from a probit model. A consistent estimate of expected WTP and a consistent estimate of the median can then be calculated as:

E_η(WTP│z_i, γ) = MD_η (WTP│z_i, γ) = ( (3)

In addition to being appropriate for estimation by parametric modeling, as shown above, the data collected though the dichotomous choice question, and the associated average trip willingness to pay those responses imply, can also be estimated non-parametrically using the Turnbull approach (Haab and McConnell, 2002). This second, more robust, estimation method yields a complementary lower bound WTP estimate. It also addresses issues common to parametric modeling, such as fat tails (i.e., a large percentage of respondents agreeing to pay the highest bid amount).

The range of possible bid amounts are chosen based on results of a previous pilot study (described below), review of other trip expense literature, and consultation with subject matter experts. The bid amounts max out at $2,000 in version A and $750 in version B, a value within the realm of realistic change for park visitors, yet likely above the WTP for the large majority of respondents. The maximum value is also set so as not to significantly upward bias the responses.

Piloting of the WTP Question

During the Phase I pilot, a similar WTP question was evaluated in three select parks (Glacier National Park (GLAC), Klondike Gold Rush National Historical Park (KLGO), and World War II Valor in the Pacific National Monument (VALR)).² Each of the three pilots provided respondents with similar phrasing as described above. However, a payment card response format was used, which includes a menu of bid amounts that respondents can select from. The responses to that question helped inform the bid design for the WTP question proposed here. GLAC provided the full range of payment selection options up to $2,000. VALR and KLGO each rose to $750. In each of these pilots, we found no indication of an upward bias or artificial truncation of the responses. In each case, we identified fewer than 10% of respondents selecting the highest respective bid amounts (5% in VALR and GLAC and 8% in KLGO). As noted by Parsons and Myers (2016), problematic fat tails are typically characterized by a ‘yes’ response of 20% or greater. Thus, we believe the upper end of the bid range was sufficient.

Results from VALR have been peer reviewed and published (Sinclair, Huber, & Richardson, 2020). Several key highlights from this study include:

• Payment card question and additional variables hypothesized to influence an individual’s consumer surplus were explored/tested using a full, unrestricted regression model. These variables include: age, income, sex, education, residency, and whether the primary reason for visiting the National Memorial was to pay respects or learn more about American history and culture. These were the top two reported motivations for visiting the site.

• 86% of respondents answered the willingness to pay question, with 13% of respondents indicating their additional willingness to pay as $0, while 5% indicated the maximum value of $750.

• The income variable was identified as being significant and positive as economic theory would suggest.

• Estimated mean WTP for visiting VALR is $157.34 per person (in 2016 dollars), 95% CI [$133.09, $183.42]. Based on an average trip length of 1.06 days, this yields a value of $148.43 per person per day. Aggregating the individual consumer surplus estimate across all visitation in 2016 results in a total annual value of $270 million. This analysis indicates that preservation of this National Memorial is highly valued by the public.

2.4 Any use of periodic (less frequent than annual) data collection cycles to reduce burden

This is not applicable to this effort. As previously described (2.1.1), the sites for this collection are selected based on a randomized selection of park units, stratified by unit type and size. Each site is sampled once in a calendar year and is not sampled again until the entire bin of sites within the strata have been exhausted.

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

For visitors agreeing to participate, the intercept survey will be verbally administered by the surveyor and the responses will be recorded via an Android Tablet. If the visitor does not agree, surveyors will thank them for their time, attempt to ask the four non-response bias questions, and then sample the next nth visitor. This process will be standardized across all park units using the protocols established for surveyors.

To maintain intercept response rates as high as practical while collecting the necessary information, the intercept duration is kept to a maximum of six minutes. The instrument used in the Phase II Pilot for intercepts achieved this goal. Through previous direct experiences by the research team, intercepts longer than six minutes generate increased dropout rates and refusal rates. To further increase response rates for the intercept, surveyors are strategically positioned, when feasible, in locations to not only randomly intercept visitors, but also to find them in locations where they are less likely to feel rushed or pushed for time. These locations are identified via communications with park staff.

Following a brief introduction of the purpose of the survey, the potential respondent (adult group member with the most recent birthdate) will be asked if they would be willing to take part in the 6-minute survey. The intercept survey will include the questions used as the non-response bias check, as well as basic trip characteristics questions that apply to their current visit. Four potential outcomes are expected following the request to participate: (1) Complete refusal; (2) Partial refusal, answering non-response questions but nothing further; (3) Complete Intercept, but refusal to take mail-back; and (4) Complete Intercept and take mail-back.

As part of the intercept protocol, surveyors will add a unique identifier to each survey that will be linked to the mail-back survey, the postage-paid envelope, and on the cover letter inside the packet with the URL to the online survey. This unique identifier will also serve as the password to access the online survey.

The final question on the Intercept survey will provide the respondent an opportunity to provide their mail or email address that will be used for the follow-up protocol of the “Tailored Design Method” (reminder protocols for mail-back surveys). Respondents will first be asked to provide their mailing address, followed by their email, if home address is refused. Respondents may refuse both physical mailing address and email address and still be provided the mail back survey. Based on the Pilot Phase II, it is expected that response rates are highest for those who have provided mailing addresses followed by those providing email addresses. Finally, those who provided neither form of follow-up contact generated the lowest response rates. Typically, 65-75% of respondents provided some form of follow up contact (mailing/email address). The combination of these three groups (those who provide mailing address, those who provide email address, and those who accept the paper survey without providing contact details) yields our estimated 40% response rate.

To maximize response rates of the mail-back surveys, Dillman’s Tailored design method will be used to provide postcard reminders and two replacement surveys to those who provided mailing addresses. Similarly timed reminders will be delivered via email to those who provided electronic contact information. This email will contain a thank you note and link to the online version of the survey and reminder of the unique ID provided at the time of the intercept.

Addressing Non-Response

Example Introductory Script for Intercept Survey:

“Hello, I am working with [NPS Site] conducting a 6-minute survey to improve visitor experiences in the park. May I ask you several questions about your [NPS Site] experience?

• If NO – The surveyor will thank the visitor and ask them to answer the four questions that will serve as a non-response bias check (see below)

• If YES – The surveyor will begin the intercept visitor survey with the recruited individual after reading the Paperwork Reduction and Privacy Act below. The surveyor will verbally administer the survey and record responses on an Android Tablet.

Two potential sources of non-response bias exist in this study: 1) those who choose to participate in the intercept study compared to those who refuse and 2) those who participated in the intercept survey and complete the mail-back survey compared to those who participated in the intercept survey but refuse/do not send the mail-back survey back. For each park’s sample, the research team will monitor response rates for both scenarios to gauge whether non-response bias needs to be tested. If the response rate of either the intercept survey or the mail-back survey falls below 80%, a non-response bias test will be conducted on the appropriate group. For instance, if less than 80% of visitors accept the intercept survey (ex. choose to participate), we will test the responses between those who participated in the intercept survey and those who answered the four non-response bias questions below.

1. “Are you a permanent or seasonal/second home resident of the local area around [NPS Unit]?”

2. “Do you currently live in the United States?”

3. “On this trip away from home, have you [and your personal group] stayed, or will you stay overnight away from your permanent residence either in [NPS Unit] and/or within the local area? (Please refer to map of local area)”

4. Was visiting [NPS Site] the primary purpose for your overall trip away from home?Because the intercept survey will be linked to the online survey via a unique identifier which is also used as a password for the online survey, respondents who do not complete the follow-up survey (either by mail-back or online) will be compared to those who did participate. Thus, non-response bias checks will be conducted on both intercept and mail-back survey respondents. Chi-square tests (using a p-value of .05 as the indicator of significance) will be conducted between the respondents and non-respondents to explore and identify any issues of underrepresentation due to non-response bias. All reports will show the outcomes of the non-response bias checks and indicate any existing conditions in which non-response bias may be present. The multi-response mode approach to the mail-back/online survey will allow for more widespread participation among respondents, limiting non-response issues.

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of test may be submitted for approval separately or in combination with the main collection of information.

While this is a new information collection request, the NPS administered a pilot SEM program in 14-NPS units from 2015-2016 (Phase I) to identify and better understand the need for more advanced socioeconomic monitoring and to determine the viability and cost of such a program. In total, over 6,000 surveys were completed, providing the NPS with useful insights about in-park visitor characteristics.

The Phase I pilot study produced an overarching study design that will allow the NPS to fully implement this Socioeconomic Monitoring Study. The majority of survey questions were tested through the Phase 1 Pilot Study. The survey was tested in 14 parks across the System, refined based on findings, and reviewed with multiple subject matter experts. Additionally, the instrument in the Pilot Phase I consisted entirely of a mail-back survey. Upon review of the pilot findings, it was determined that an on-site intercept instrument followed by a mail-back survey could enhance the quality and quantity of responses collected.

Building on the findings, recommendations, and identified opportunities for improvement, a Phase II pilot was conducted in 2022 in 24 NPS units that utilized the refined two-phase collection methodology and questionnaires. These parks used nearly identical instruments as proposed here, except for the payment card question. Lessons learned from the composite of these two pilot efforts have been applied to this information collection submission.

The two pilot phases have allowed the NPS to (1) validate the survey questions³, (2) investigate various sampling methods, (3) estimate the respondent burden and response rates, and (4) determine the usability of the survey design across diverse park types and contexts.

Additionally, the final proposed instrument (both intercept and mail-back) was tested on nine members of the public of various ages (18 or older) and backgrounds. Recruitment was conducted using students, staff, and faculty at the University of Montana. Test respondents were asked to imagine they were taking the survey in a national park setting. All nine individuals were read the intercept survey and asked to respond out loud, similar to how the intercept survey is administered in the field. Research team observers noted where the respondents appeared confused and asked for feedback following completion.

Respondents took an average time of 5 minutes and 36 seconds to complete the intercept survey, with the shortest time being 4 minutes 22 seconds and the longest being 6 minutes 28 seconds. Some questions, particularly those related to group size, needed some clarification from the survey administrator. This observation of the necessity to provide occasional clarification on the intercept survey reinforces the need to periodically assess question performance and the value of verbally conducting this survey. Such an approach allows questions that may cause confusion to be clearly explained by the surveyor on site.

The mail-back/online survey took an average of 13 minutes to complete, with the shortest time being 8 minutes 20 seconds and the longest being 21 minutes 32 seconds. For the mail-back survey, the respondents read and completed surveys on their own (to reflect the conditions of how the survey is fielded).

5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Statistical Consultant

Chris Neher – Bioeconomics, LLC.

Dave Patterson – University of Montana

Collection and analysis agency:

Jake Jorgenson – RRC Associates

Jeremy Sage – RRC Associates

Bill Valliere – Otak

National Park Service Socioeconomic and Resource Recovery Division

Literature Cited

Boyle, K.J., 2017. Contingent valuation in practice. A primer on nonmarket valuation, pp. 83-131.

Haab, T.C., McConnell, K.E. 2002. Valuing environmental and natural resources: the econometrics of non-market valuation. Edward Elgar Publishing: Northampton, MA.

Haefele, M., Loomis, J.B., Bilmes, L. 2016. Total Economic Valuation of the National Park Service Lands and Programs: Results of a Survey of the American Public. 2016. HKS Working Paper No. 16-024, Available at http://dx.doi.org/10.2139/ssrn.2821124

Johnston, R.J., Boyle, K.J., Adamowicz, W., Bennett, J., Brouwer, R., Cameron, T.A., Hanemann, W.M., Hanley, N., Ryan, M., Scarpa, R. and Tourangeau, R. 2017. Contemporary guidance for stated preference studies. Journal of the Association of Environmental and Resource Economists, 4(2), pp.319-405.

Parsons, George, and Kelley Myers. 2016. Fat Tails and Truncated Bids in Contingent Valuation: An Application to an Endangered Shorebird Species. Ecological Economics 129: 210–19.

Sinclair, W., Huber, C., Richardson, L. 2020. Valuing Tourism to a Historic World War II National Memorial. Journal of Cultural Heritage, 45, 334-338.

Vaske, J. J. 2019. Survey research and analysis (2^nd ed.). Sagamore-Venture.

1 Sample Size Calculator | Good Calculators

2 OMB Control Number 1024-0224, Expiration Date: 5-30-2019, Programmatic Clearance for NPS-Sponsored Public Surveys

3 https://www.nps.gov/subjects/socialscience/upload/SEM_Pilot_Final_508accessible.pdf

11