1024-NEW Climate Change SSB 4-5.23

Supporting Statement B

National Park Service Visitor Perceptions of Climate Change Study

OMB Control Number 1024-NEW

Collections of Information Employing Statistical Methods

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

This is a proposal for an on-site, intercept visitor survey at 20 National Parks Service (NPS) and 12 US National Wildlife Refuge System (NWR) sites across ten strata (ten different geographic regions). This data collection effort is a replication of a 2011 visitor survey (Davis, Karg & Thompson, 2012) funded by the National Science Foundation (NSF). Data collection will be conducted for nine consecutive days at each site during its peak (highest) visitation month based on the monthly average from the most recent year recorded in the NPS Visitor Use Statistics Data Portal.

Park and refuge sample sites represent ten different geographic regions. The sample sites were selected to capture the greatest diversity in two key criteria: (1) types of climate change impacts and observations and (2) types of visitor use. Intended sampling sites are outlined below (Table 1.1). All surveys will be conducted on lands managed by the National Park Service and U.S. Fish and Wildlife Service. We will use a random, intercept sampling procedure with a goal of a total completed sample size of 12,013 on-site survey respondents (Table 2.1 and Table 2.2), enough for a 95% confidence interval with a +5% margin of error at each park or refuge. We expect an average response rate of 70% based upon previous studies using a similar sampling approach (Davis, Karg & Thompson, 2012).

The total respondent universe is estimated to include 45,869,132 adults (18 years or older): 38,452,551 NPS visitors (https://irma.nps.gov/Stats/Reports/Park), and 7,416,581 NWR visitors (N. Sexton, personal communication, November 15, 2022). Table 1.1 lists the 20 NPS survey sites, organized by the ten geographic areas, with each site’s total annual visitation in 2021. The month with the highest number of park visitors is labeled the “peak” visitation month. We will survey during the peak visitation month in order to increase our odds of engaging a broad and diverse sample within nine days. Table 1.1 also includes the average daily visits during the peak month; thus, the respondent universe for each site is estimated to be the average daily visits during the peak month multiplied by nine collection days.

Table 1.1. Annual NPS Visitation and Sampling Frame per Site, Organized by Region.

National Park Unit	Annual Visitation in 2021	Peak Month Visitation in 2021	Average Daily Visits in Peak Month	Respondent Universe (9 days)
Florida		March
1. Everglades	942,130	123,528	3,985	35,865
2. Big Cypress	2,563,810	326,285	10,525	94,725
Midwest		July
3. Pictured Rocks	1,313,179	363,157	11,715	105,435
4. Sleeping Bear Dunes	1,722,955	481,137	15,521	139,689
Central California		July
5. Sequoia	1,059,548	182,423	5,885	52,965
6. Muir Woods	657,722	98,481	3,177	28,593
7. Yosemite	3,287,595	509,874	16,448	148,032
Northwest		August
8. Olympic	2,718,925	795,063	25,647	230,823
9. Mount Rainier	1,670,063	449,446	14,498	130,482
10. Fort Vancouver	809,525	122,148	3,940	35,460
Northeast		July
11. Cape Cod	4,017,239	638,125	20,585	185,265
Southern Atlantic		August
12. Cape Hatteras	3,206,056	491,061	15,841	142,569
Alaska		July
13. Kenai Fjords	411,782	121,131	3,907	35,163
Pacific Islands		February
14. Pu’uhonua o Honaunau	250,919	29,125	940	8,460
15. Hawaii Volcanoes	1,262,747	130,475	4,209	37,881
Rocky Mountain		July
16. Glacier	3,081,656	796,868	25,705	231,345
17. Rocky Mountain	4,434,848	904,767	29,186	262,674
Desert Southwest		March
18. Joshua Tree	3,064,400	383,735	12,791	115,119
19. Santa Monica Mountains	830,451	88,727	2,958	26,622
20. Death Valley	1,146,551	171,617	5,536	49,824
Total	38,452,551	7,207,173	232,999	2,096,991

The respondent universe for each NWR site is also based on nine days of surveying. However, we only had access to annual visitation rates for 2021, so we calculated our estimate based on daily visitation rates (# of annual visitors ¸ 365 days). We chose a proposed sampling month that matched the peak month of visitation in the same geographic area as detailed in Table 1.1. Table 1.2 lists the NWR sites organized by the geographic strata and respondent universe for each location.

Table 1.2. Annual NWR Visitation and Sampling Frame per Site, Organized by Geographic Area.

National Wildlife Refuge	Annual Visitation in 2021	Proposed Sampling Month	Estimated Average Daily Visits	Respondent Universe (9 days)
Florida		March
1. J.N. Ding Darling NWR	791,403		2,168	19,514
2. Ten Thousand Islands NWR	230,000		630	5,571
Midwest		July
3. Minnesota Valley NWR	78,000		213	1,917
4. Ottawa NWR	352,110		964	8,676
Central California		July
5. Don Edwards San Francisco Bay NWR	875,000		2,397	21,575
Northwest		August
6. Billy Frank Jr. Nisqually NWR	188,941		517	4,653
Northeast		July
7. Sachuest Point NWR	339,115		929	8,361
Southern Atlantic		August
8. Pea Island NWR	2,130,000		5,835	52,515
Alaska		July
9. Kenai NWR	1,200,000		3,287	29,583
Pacific Islands		February
10. Kilauea Point NWR	160,012		438	3,945
Rocky Mountain		July
11. Lee Metcalf NWR	222,000		608	5,472
12. Rocky Mountain Arsenal NWR	850,000		2,328	20,952
Total	7,416,581		20,314	182,734

2. Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

This is a one-time collection that is intended to update our understanding of park/refuge visitors' perceptions of and attitudes about climate change. The procedure for data collection is a face-to-face intercept survey methodology at 32 sites in 10 geographic areas (i.e., strata). The research team will consist of five to six researchers at each site. Teams of two to three researchers will administer surveys at various stages of the visitor experience, including visitor centers near the entrance, popular viewpoints, trailheads, and on-site campgrounds. A random, intercept sampling method over the period of nine consecutive days will be used to collect information at each location. Having nine days will allow for two weekends of surveying, which typically see higher numbers of visitation. The research team will ask every nth adult visitor (18 years and older) encountered to take a survey, where n will vary based on the anticipated visitation and number of initial contacts at each site. At trailheads, this will include asking every nth visitor who is exiting the trail so as not to interrupt their recreational pursuits. Campground protocols will include asking every nth camper in inhabited campsites. Researchers will survey 6-8 hours per day or until the team reaches the quota for the day.

Our response rate is estimated to be 70% based on previous research using a similar sampling approach (Davis, Karg & Thompson, 2012). Given previous response rates for the parks and refuges, we will strive to recruit a sample size large enough to achieve a 95% confidence level with sampling errors at +5% for each park and refuges visited during this collection. For human dimensions of natural resources studies such as this, a 95% confidence interval with a +5% margin of error is considered to be suitable for creating reliable data that can be generalized to the larger population of annual visitors within these parks and refuges (Vaske, 2019). We recognize that park and refuge sites vary in annual visitation rates as shown through previous research (Davis, Karg & Thompson, 2012); however, using a sample size calculator, we can set recruitment goals for each site. For example, to calculate the sample size (n) for Everglades National Park, we used the respondent universe (N) for the nine-day collection in the following formula¹:  

Everglades National Park: n = [z² * p * (1 - p) / e²] / [1 + (z² * p * (1 - p) / (e² * N))]

Where as:

z = 1.96, p = 0.5, N = 35,865, e = 0.05

n = [1.96² * 0.5 * (1 - 0.5) / 0.05²] / [1 + (1.96² * 0.5 * (1 - 0.5) / (0.05² * 35865)

n = 384.16 / 1.0107 = 380.089

n ≈ 381; The sample size (with finite population correction) is equal to 381.

Assumptions:

1. A single simple random sample was used.

2. A normal distribution and the z-score were used because the sample size and degrees of freedom were unavailable.

3. The sample size (with finite population correction) is equal to 385
   
   

The sample sizes needed for each of the NPS and NWR sites in this collection is reflected in the Expected Number of Responses in Table 2.1 (NPS sites) and Table 2.2 (NWR sites). Expected non-respondents were calculated using the non-response rate (the inverse of the response rate for each site). To calculate the Number of Initial Contacts needed, the Expected Non-respondents were added to the Expected Number of Responses for NPS and NWR sites.

Table 2.1: Expected Respondents and Non-Respondents for NPS Sites

	Respondent Universe (9 sampling days)	Number of Initial Contacts	Anticipated response rate	Anticipated number of completed responses	Expected Non-responses
Florida
1. Everglades	35,865	496	70%	381	115
2. Big Cypress	94,725	498	70%	383	115
Midwest
3. Pictured Rocks	105,435	498	70%	383	115
4. Sleeping Bear Dunes	139,689	500	70%	384	116
Central California
5. Sequoia	52,965	497	70%	382	115
6. Muir Woods	28,593	494	70%	380	114
7. Yosemite	148,032	500	70%	384	116
Northwest
8. Olympic	230,823	500	70%	384	116
9. Mount Rainier	130,482	500	70%	384	116
10. Fort Vancouver	35,460	496	70%	381	115
Northeast
11. Cape Cod	185,265	500	70%	384	116
Southern Atlantic
12. Cape Hatteras	142,569	500	70%	384	116
Alaska
13. Kenai Fjords	35,163	496	70%	381	115
Pacific Islands
14. Pu’uhonua o Honaunau	8,460	479	70%	368	111
15. Hawaii Volcanoes	37,881	496	70%	381	115
Rocky Mountain
16. Glacier	231,345	500	70%	384	116
17. Rocky Mountain	262,674	500	70%	384	116
Desert Southwest
18. Joshua Tree	115,119	498	70%	383	115
19. Santa Monica Mountains	26,622	493	70%	379	114
20. Death Valley	49,824	497	70%	382	115
Total	2,096,991	9,938	70%	7,636	2,302

Table 2.2: Expected Respondents and Non-Respondents for NWR Sites

	Respondent Universe (9 sampling days)	Number of Initial Contacts	Anticipated response rate	Anticipated number of completed responses	Expected Non-responses
Florida
1. J.N. Ding Darling NWR	19,514	491	70%	377	114
2. Ten Thousand Islands NWR	5,571	468	70%	360	108
Midwest
3. Minnesota Valley NWR	1,917	418	70%	321	97
4. Ottawa NWR	8,676	479	70%	368	111
Central California
5. Don Edwards San Francisco Bay NWR	21,575	492	70%	378	114
Northwest
6. Billy Frank Jr. Nisqually NWR	4,653	462	70%	355	107
Northeast
7. Sachuest Point NWR	8,361	479	70%	368	111
Southern Atlantic
8. Pea Island NWR	52,515	497	70%	382	115
Alaska
9. Kenai NWR	29,583	494	70%	380	114
Pacific Islands
10. Kilauea Point NWR	3,945	457	70%	351	106
Rocky Mountain
11. Lee Metcalf NWR	5,472	467	70%	359	108
12. Rocky Mountain Arsenal NWR	20,952	492	70%	378	114
Desert Southwest
Total	182,734	5,696	70%	4,377	1,319

In sum, 15,634 total initial contacts are needed to obtain 12,013 total responses.

We do not anticipate any unusual problems requiring specialized sampling procedures, nor will it be necessary to use periodic (less frequent than annual) data collection cycles to reduce the burden. We anticipate being able to recruit a statistically acceptable number of respondents over the nine-day collection. Table 2.3 is an example of the sampling schedule for small (<10,000 visitors over 9 days), medium (10,000 – 50,000 visitors over 9 days), and large (> 50,000 visitors over 9 days) sites. For illustration we used a representative NPS and NWR site from the same geographic region for each category. We anticipate having greater visitation to all sites over the weekends as opposed to during the weekdays. Additionally, Friday was intentionally left off of the schedule as a flexible day off for the researchers. This also serves as an additional survey day should one of our previous days experience low numbers due to inclement weather or other unforeseen circumstances.

Table 2.3 Example Nine-day Sample Schedule

Number of Initial Contacts
	Sat	Sun	Mon	Tues	Wed	Thurs	Fri	Sat	Sun	Total
Small Sites (<10,000 visitors)
Pu’uhonua o Honaunau NPS	80	80	39	40	40	40	-	80	80	479
Kilauea Point NWR	65	65	49	49	49	50	-	65	65	457
Medium Sites (10,000 – 50,000 visitors)
Kenai Fjords NPS	70	70	54	54	54	54	-	70	70	496
Kenai NWR	70	70	53	53	54	54	-	70	70	494
Large Sites (>50,000 visitors)
Cape Hatteras NPS	70	70	55	55	55	55	-	70	70	500
Pea Island NWR	70	70	55	54	54	54	-	70	70	497

3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Our goal is to minimize the respondent burden. To increase efficiency in the data collection, the survey team will use tablets to administer the surveys. Each surveyor will have two tablets and thus be able to have two visitors engaged with a survey at one time. The following sample script will be used to recruit individuals to take the survey:

Hello, I am a student from Slippery Rock/Northern Michigan University working with the National Park Service and Fish Wildlife Service conducting visitor surveys at [this Park/Refuge]. This survey is about visitors’ perceptions of climate change at this [Park/Refuge] and takes about 7 minutes to complete. Your participation is completely voluntary and anonymous, and you can stop taking the survey at any time. Are you at least 18 years of age or older? [If yes, continue…] Would you be willing to help the National Park Service/Fish and Wildlife Service better understand visitors’ opinions by taking this survey?

Following the introduction script, participants will be handed a tablet to complete the survey. The survey will display the following information on the first screen before the respondent answers any questions:

This survey is part of a research effort by the National Park Service, Northern Michigan University, and Slippery Rock University. Your participation in taking this survey is completely voluntary. In the survey you will find questions about your present knowledge of climate change, your beliefs regarding the causes, impacts, and solutions to climate change, and some more general questions. All the answers you give will be treated anonymously. Every question of the survey provides clear instructions. Please read them carefully before answering the questions. If, for any reason, you do not want to continue filling out the survey, you can always stop at any time. However, a completed survey would greatly help us in our understanding of visitors’ opinions. Filling out the survey will take approximately seven minutes. Thank you for your participation.

Paperwork Reduction Act Statements:

The National Park Service is authorized by 54 U.S.C. 100702 and the U.S. Fish and Wildlife Service is authorized by the National Wildlife Refuge System Administration Act of 1966 (16 U.S.C. 668dd–668ee) to collect this information. This collection involves a visitor survey to understand visitor perceptions and concerns about climate change to guide communication and engagement efforts. Your responses to this collection are completely voluntary and will remain anonymous. You can end the process at any time and will not be penalized in any way for choosing to do so. Data collected will only be reported in aggregates and no individually identifiable responses will be reported. A Federal agency may not conduct or sponsor, and you are not required to respond to, a collection of information unless it displays a currently valid OMB Control Number. Your response is not required to obtain or retain a benefit. OMB has approved this collection of information and assigned control number XXXX-XXXX. The expiration date is XXXXXXX

Estimated Burden Statements:

Public reporting for this collection of information is estimated to be about 7 minutes per survey. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to the Information Collection Clearance Officer, National Park Service, 12201 Sunrise Valley Dr. (MS-242), Reston, VA 20192 or Service Information Clearance Officer, Fish and Wildlife Service, U.S. Department of the Interior, 5275 Leesburg Pike, MS: PRB (JAO/3W), Falls Church, VA 22041-3803, or via email at [email protected].

If the visitor declines to take the survey, researchers will record the date, time, approximate age, and gender for any non-respondents. Refusals to take the survey will fall into one of two categories: “hard” refusals or “soft” refusals. A “hard” refusal is defined as any outright declaration that the visitor does not want to take the survey or any case in which the visitor displays any anger. A “soft” refusal is defined as any refusal in which the visitor offers an excuse for not taking the survey such as not having time or being too busy.

For any ‘soft’ refusals, the researcher will ask the non-respondent to answer 3 brief questions using the following script:

Thank you, instead could you answer three quick questions today?

If the participant agrees, the researcher will hand them a tablet with the following 3 non-response questions:

1. How important is the issue of climate change to you personally?

- Extremely Important

- Very Important

- Somewhat Important

- Not too Important

- Not at all Important

2. How worried are you about climate change?

- Very Worried

- Somewhat Worried

- Not Very Worried

- Not at all Worried

3. How much do you think climate change will harm the following?

	A Great Deal	A Moderate Amount	Only a Little	None at all	Don’t know
Future generations of people	O	O	O	O	O
You personally	O	O	O	O	O

After the visitor has completed the three non-response questions the researcher will thank the visitor for their time. Having this information from the non-responders will allow for the researchers to check for non-response bias by testing whether non-respondents significantly differ in their segmentation from the completed sample responses. If a non-response bias is found, the data will be weighted to reduce the effect of non-response bias and any implications will be discussed with NPS and FWS staff. The researchers estimate the time of taking the non-response survey to be less than two minutes. The researchers estimate that 10% of all refusals will be “soft” and converted into taking the 3-question non-response survey based on previous research which has demonstrated a 7 – 14% refusal conversion rate (Retzer, Schipani, & Cho, 2004).

4. Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The design of the current survey instrument is based off of the previous survey used in the 2011 study (Schweizer, Davis, & Thompson, 2013). A thorough literature review was conducted prior to the creation of the updated survey to prioritize and update existing questions, purge any dated questions, and to add any additional questions using current conceptions related to the purpose of the study. Six peer reviewers, which included members of the National Park Service Climate Change Response Program, the National Fish and Wildlife Service, and authors of the previously conducted Climate Change Education Partnership (CCEP) study, provided detailed feedback on question clarity, design, and format. The reviewers provided information on survey language, question order, number or questions, burden estimate and concepts to be studied over five successive rounds of development.

Additionally, a pretest was conducted with nine university students recruited through a convenience sampling procedure in order to pretest the initial draft of the survey. All students were asked to complete the survey using Qualtrics software and to provide feedback on any confusing questions, formatting issues, or question structure. Qualtrics automatically records the time each individual spends taking the survey. Students (n = 9) indicated some repetitiveness of certain climate change questions as well as some comments about the question format, positioning of questions, and demographic questions. Based on this pretest, we estimate the average time to complete the survey to be seven (7) minutes.

5. Provide the names and telephone numbers of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Individuals consulted on statistical aspects of the design:

Jes Thompson, PhD

Northern Michigan University

Professor

College of Business

Sustainability Advisory Council, Co-Chair

[email protected]

906-227-1057

Shawn Davis, PhD

Slippery Rock University

Associate Professor

Biology Department

[email protected]

724-738-2599

Contractors collecting and analyzing the information for the agency:

Jes Thompson, PhD

Northern Michigan University

Professor

College of Business

Sustainability Advisory Council, Co-Chair

[email protected]

906-227-1057

Shawn Davis, PhD

Slippery Rock University

Associate Professor

Biology Department

[email protected]

724-738-2599

References

Chryst, B., Marlon, J., van der Linden, S., Leiserowitz, A., Maibach, E., & Roser-Renouf, C. (2018). Global warming’s “Six Americas Short Survey”: Audience segmentation of climate change views using a four question instrument. Environmental Communication, 12(8), 1109-1122.

Davis, S., Karg, S. & Thompson, J.L. (2012). Climate change education partnership visitor survey summary report. Available at: https://www.nps.gov/subjects/climatechange/upload/NSF_CCEP_Survey_Summary_Report-508.pdf

Retzer, K., Schipani, D., and Cho, Y.I. 2004. “Refusal Conversion: Monitoring the Trends.” Proceedings of the American Statistical Association, Section on Survey Research Methods, pp. 426-431. Washington, D.C.: American Statistical Association.

Schweizer, S., Davis, S., & Thompson, J.L. (2013). Changing the Conversation about Climate Change: A Theoretical Framework for Place-Based Climate Change Engagement, Environmental Communication: A Journal of Nature and Culture, 7:1, 42-62.

Vaske, J. J. (2019). Survey research and analysis (2^nd ed.). Sagamore-Venture.

1 2015-2022 goodcalculators.com (retrieved November 9, 2022)

23