Expedited Form

National Park Service
U.S. Department of the Interior
Social Science Program
Expedited Approval for NPS-Sponsored Public Surveys
1.

Project Title ⎢
Submission
Date:

2.

Abstract:

3.

Title:
Affiliation:
Street Address:
City:

Robert

Last Name:

Powell

Assistant Professor
Clemson University, Department of PRTM
263 Lehotsky Hall, Clemson University
Clemson

Phone:

(864) 656 - 0787

Email:

[email protected]

State:
Fax:

SC

Zip code:

29634

(864) 656 - 2226

Park or Program Liaison Contact Information
First Name:

Gary

Title:

Chief

Last Name:

Oye

Park:
Park
Office/Division:

Wilderness Stewardship and Recreation
Management Division

Street Address:

1201 Eye (I) Street NW 10th Floor, Rm004

City:

7 / 27 / 08

The purpose of this research is to explore how visitors to a diverse set of public land areas
respond to the “Tread Lightly” off-highway vehicle (OHV) skills and ethics education
program. The three areas selected for study include two NPS units and one BLM unit..
Attitudes toward recommended “Tread Lightly” principles will be investigated in Big
Cypress National Preserve (BICY) and Canyonlands National Park (CANY) and one Bureau
of Land Management site, Imperial Sand Dunes Recreation Area (ISDRA). Mail-back
surveys will be used, following a modified Dillman approach (2000).
(not to exceed 150 words)

Principal Investigator Contact Information
First Name:

4.

Preliminary study of the Tread Lightly! off highway vehicle visitor
education efforts on public lands (Big Cypress, Canyonlands, Imperial
Sand Dunes)

Washington, D.C.

Phone:

(202) 513 -7090

Email:

[email protected]

State:
Fax:

DC

Zip code:

(202) 371 - 2401

20005

Project Information
5.

Park(s) For Which Research
is to be Conducted:

6.

Survey Dates:

7.

Type of Information Collection Instrument (Check ALL that Apply)
X Mail-Back
questionnaire
‰

8.

Big Cypress National Preserve, Canyonlands National Park,
Imperial Sand Dunes Recreation Area

8/01/2009

‰

On-Site
Questionnaire

(mm/dd/yyyy)

‰

to

Face-to-Face
Interview

12/15/09

‰

(mm/dd/yyyy)

Telephone
Survey

‰

Focus
Groups

Other (explain)

Survey
Justification:
(Use as much space
as needed; if
necessary include
additional
explanation on a
separate page.)

Social science research in support of park planning and management is mandated in the
NPS Management Policies 2006 (Section 8.11.1, “Social Science Studies”). The NPS
pursues a policy that facilitates social science studies in support of the NPS mission to
protect resources and enhance the enjoyment of present and future generations (National
Park Service Act of 1916, 38 Stat 535, 16 USC 1, et seq.). NPS policy mandates that
social science research will be used to provide an understanding of park visitors, the nonvisiting public, gateway communities and regions, and human interactions with park
resources. Such studies are needed to provide a scientific basis for park planning,
development, operations, management, education, and interpretive activities.
Land managers need the ability to predict and influence recreational behavior in order to
manage wildlands effectively. This includes further understanding of off-highway vehicle
(OHV) drivers’ practices and ethics, particularly compliance with recommended
minimum-impact practices. Currently, there is currently a lack of compliance research
and literature regarding minimum-impact OHV visitor education generally and Tread
Lightly! (TL!) programs specifically. TL! is the most widely used minimum-impact OHV
education program. Research describing how visitors perceive and respond to TL! and
other minimum-impact educational programs has been described as insufficient, and the
current research base and state of knowledge remains inadequate (Chavez & Knap, 2006;
Vancini, 1989).
This study is designed to identify salient variables that account for OHV operators’
compliance with TL! recommended practices in three different ecosystems with different
types of educational efforts. The research utilizes Fishbein and Ajzen’s Theory of
Reasoned Action (Ajzen & Fishbein, 1980). The study also investigates from which
source(s) OHV visitors learn about TL!, and whether the type of source influences their
behaviors (Rogers, 1995).
The majority of items included in this questionnaire are based on earlier work by Powell,
Wright, and Vagias (2008) on “Leave No Trace” practices among backcountry hikers.
Items used in the earlier research were pre-tested extensively before being implemented
(OMB #1024-0224, NPS 07-038).
The results will be utilized to inform management decisions regarding the future direction
of the Tread Lightly! program and to improve existing educational tools to reach a
broader segment of the OHV public to enhance both enjoyment and resource protection.

9.

Survey
Methodology: (Use
as much space as
needed; if necessary

(a) Respondent universe:
Adult visitors, age 18 or older, who are 2008 annual permit holders for OHV travel in Big
Cypress (BICY); 2008 backcountry OHV permit holders in Canyonlands (CANY); and
members of the American Sands Association OHV club that supports travel to the

include additional
explanation on a
separate page.)

Imperial Sand Dunes Recreation Area (ISDRA).

(b) Sampling plan/procedures:
A systematic random sampling technique will be used to sample OHV visitors to the
three sites. The areas where data are to be collected represent a range of ecosystems and
TL! education efforts. The variation in educational programming between sites means
investigators will treat each unit as a case study to gain further understanding of the
impact of site-specific educational approaches. No attempt will be made to generalize the
findings to other contexts in which minimum-impact OHV programs are being employed.
The OHV permit holder address lists at BICY and CANY and the 2008 American Sands
Association membership list will be utilized as sampling frames. The American Sands
Association works closely with BLM to provide TL! educational information for ISDRA.
All drivers of OHVs in BICY and CANY are required to obtain permits. A random
numbers table will be utilized to identify the first individual to be selected from the
mailing lists and then every kth individual from the lists will be selected. The sampling
interval will be determined by dividing the number of 2008 permit holders/members by
the necessary sample size (700 initial contacts from each site).
(c) Instrument administration:
Selected individuals will first be sent a postcard indicating their selection for the study.
Cooperation from these individuals will be solicited using a modified Dillman approach
(Dillman, 2000). Individuals in the sample will be mailed an introductory letter and the
survey instrument. Ten days later, those who have not yet responded will be mailed a
postcard, reminding them about the survey. Approximately two weeks following the
postcard mailing, respondents who have not responded will be mailed a follow-up letter
and a replacement questionnaire.
(d) Expected response rate/confidence levels:
An overall response rate of 60% is expected for the mail-back questionnaire. This is
expected to be consistent across all three areas studied. Although a 2007 survey of OHV
permit holders at BICY only achieved a 46% response rate, the PIs and the NPS at BICY
have undertaken extensive stakeholder outreach and allowed for the review of the
research instruments to promote greater participation and trust in the research results.
Because of this, and because of our experience using similar methods which have resulted
in response rates of 70% or greater (Powell, 2004; Powell, Wright, & Vagias, 2008), we
consider 60% response rate to be a reasonable estimate of the final response rate.
A sample of approximately 700 individuals from each site will be mailed the survey.
Approximately 420 individuals per research site are expected to complete and return the
survey. A confidence interval of +/-5.0% is expected for each research site.
(e)
Strategies for dealing with potential non-response bias:
A portion of the original sample that did not return the questionnaire will be contacted via
telephone in order to ascertain potential differences between those who returned a
completed questionnaire and those who chose not to participate. Telephone numbers are
available from OHV permits at CANY and BICY and from the American Sands
Association membership list at ISDRA. In addition to comparing respondent data with
available frame data, we will attempt to complete 30 phone interviews from nonrespondents. Non-response bias will be checked by sampling the age, sex, and experience
use history of non-respondents and statistically comparing these data with those of the
respondents.(script attached in appendix).
(f) Description of any pre-testing and peer review of the methods and/or instrument
(recommended):

A number of processes and procedures were undertaken to develop the Tread Lightly
questionnaire based on recognized social science research procedures (Babbie, 2001;
DeVellis, 2003; Foddy, 1993; Fowler, 1993). The TL! instrument is identical for all three
study locations, except where exact locations are referenced in the questionnaire. There
are, however, additional modules of questions specific to each research site included at
the request of agency personnel in the respective units. (Section G in the attached
questionnaire).
Developing the questionnaire involved the following processes:
First, the initial draft of the survey instrument was based on prior research, most notably
the recently completed “Leave No Trace” (LNT) study that also assessed a minimumimpact visitor education program and utilized a related theoretical framework and a
similar data collection method (Powell, Wright, & Vagias, 2008). The LNT questionnaire
was extensively pilot tested and refined prior to collecting data in three NPS units in
2007/08. After analysis of these data, including writing an NPS Technical Report
(Powell, Wright, & Vagias, 2008), a Ph.D. dissertation (Vagias, 2009), and three journal
articles (in final draft stages) (Vagias, Powell, & Moore, a+b, Vagias & Powell), the
researchers further refined the TL! questionnaire.
Second, a literature review of previous OHV research was undertaken and an item pool
developed. and refined based on this review (Cordell, Betz, Green, & Owens, 2005; Fly,
Stephens, Askins, & Hodges, 2002; Lewis & Paige, 2006; Lord, Elmendorf, & Strauss,
2004; Schoenecker, 2006; Smail, 2007; Yankoviak, 2005). Each of these studies was
reviewed and the primary authors contacted to obtain full questionnaires.
After the initial item pool was developed, the questionnaire was constructed based on the
results of the LNT study, the theoretical framework, and the research questions.
The TL! survey instrument was reviewed by NPS staff from BICY and CANY and by
BLM staff at ISDRA..
After review by agency personnel, ORV stakeholder groups at two of the study sites and
TL! staff members at the national Tread Lightly office completed an extensive review and
provided written and oral comments. During this stage, the researchers attended a public
BICY ORV Advisory Committee meeting in Everglades City, FL to present the purpose
of the study and how the results will be used. Questions were answered, and a
subcommittee of BICY ORV enthusiasts/representatives was formed to review and
comment on the questionnaire and research design. Similar outreach efforts were
undertaken in ISDRA with OHV stakeholders and BLM staff via conference calls.
Next, all items in the survey, especially questions 9 and 12, were extensively reviewed by
stakeholders and comments were received and incorporated. Reviewers were asked to
identify questions that might elicit socially desirable answers (King & Bruner, 2000) or
were confusing or poorly worded. This process had similar goals and results to cognitive
testing (Conrad, Blair, & Elena, 1999; Presser, et al., 2004).

10.

Total Number of
Initial Contacts |
Expected
Respondents:

Survey:
2100

1260

11.

Phone:
50

30

Estimated Time to
Complete
Instrument (mins.):

Survey:
1

15

Phone:
1

3

Total
Burden
Hours:

347

13.

Reporting Plan: (a) General: A full technical report of results will be written and submitted to each of the
three participating research sites in May 2009 (depending on approval date).
Additionally, a copy of the survey report will be submitted to the NPS Social Science
Program in order to be archived. Finally, subsequent peer-reviewed journal articles
will be submitted in 2010.
(b) Statistical Analyses: Data will be subjected to the following analyses upon
conclusion of collection phase:
•
Assessment of quality/completeness of data. This includes examination for
coding errors, univariate and multivariate outliers, and distribution of missing
data.
•
Descriptive statistics. This includes developing a description of the sample,
comparing respondents to nonrespondents, testing the reliability of the study’s
scales, and correlation analysis of the study’s variables.
•
Inferential statistics. This includes model building utilizing multiple regression
analyses, path analysis, or structural equation modeling techniques.
The study is designed to be a series of three case studies. The sites do not represent all
NPS parks where OHV use occurs. Consequently, the results cannot be generalized
beyond the three parks. Nor can they be combined across the 3 study sites.

REFERENCES:
Ajzen, I., & Fishbein, M. (1980). Understanding Attitudes and Predicting Social Behavior.
Englewood Cliffs, NJ: Prentice-Hall.
Babbie, E. (2001). The Practice of Social Research (9th ed.). Belmont, CA: Wadsworth.
Chavez, D. J., & Knap, N. E. (2006). Manager perceptions of issues and actions for off-highway
vehicle management on National Forests in California. Albany, CA: Pacific Southwest
Research Station, USDA Forest Service, U.S. Dept. of Agriculture.
Conrad, F., Blair, J., & Elena, T. (1999). Verbal reports are data! A theoretical approach to
cognitive interviews. Paper presented at the Proceedings of the Federal Committee on
Statistical Methodology Research Conference.
Cordell, H. K., Betz, C. J., Green, G., & Owens, M. (2005). Off-highway vehicle recreation in the
United States, regions and states: A national report from the national survey on
recreation and the environment (NSRE): USDA Forest Service, Southern Research
Station.
DeVellis, R. F. (2003). Scale development: Theory and applications (2nd ed.). Thousand Oaks,
CA: Sage Publishing.
Dillman, D. A. (2000). Mail and Internet Surveys: The tailored design method. NY: John Wiley
& Sons.
Fly, M. J., Stephens, B., Askins, L., & Hodges, L. (2002). Tennessee OHV user survey.
Knoxville, TN: Human Dimensions Research Lab, The University of Tennessee.
Foddy, W. (1993). Constructing Questions for Interviews and Questionnaires: Theory and
Practice in Social Research. Cambridge: Cambridge University Press.
Fowler, F. J. (1993). Survey Research Methods (Revised ed. Vol. 1). Newbury Park, CA: Sage
Publications.
King, M. F., & Bruner, G. C. (2000). Social desirability bias: A neglected aspect of validity
testing. Psychology and Marketing, 17(2), 79-103.
Lewis, M. S., & Paige, R. (2006). Selected results from a 2006 survey of registered off-highway
vehicle owners in Montana: Montana Fish, Wildlife, and Parks.
Lord, B. E., Elmendorf, W. F., & Strauss, W. (2004). Pennsylvania's ATV riders and their needs.,
The Pennsylvania State University, University Park.

Powell, R. B. (2004). 2003 Grand Canyon National Park Commercial River Visitor Study:
Effects of Participation on Knowledge, Attitudes toward Park Management,
Environmental Values, Environmental Behaviors, and Satisfaction. New Haven: Yale
University, School of Forestry and Environmental Studies.
Powell, R. B., Wright, B. A., & Vagias, W. M. (2008). Preliminary evaluation of recreational
skills and ethics training programs occurring on public lands: The Leave No Trace
visitor education program. Clemson, SC: Clemson University and U.S. National Park
Service.
Presser, S., Couper, M. P., Lessler, J., Martin, E., Martin, J., & J., R. (2004). Methods for testing
and evaluating survey questions. Public Opinion Quarterly, 68(1), 109-130.
Rogers, E. (1995). Diffusion of Innovations (4th ed.). NY, NY: The Free Press.
Schoenecker, A. H. (2006). Describing and differentiating recreational ATV rider preferences.
University of Minnesota.
Smail, R. (2007). Wisconsin all terrain vehicle owners: Recreational motivation and attitudes
towards regulation. University of Wisconsin at Stevens Point, Stevens Point.
Vagias, W. M. (2009). Preliminary evaluation of the Leave No Trace visitor education program
in two U.S. National Park Service Units. Unpublished Ph.D. Dissertation; Department of
Parks, Recreation and Tourism Management, Clemson University, Clemson, South
Carolina.
Vagias, W., Powell, R.B, & Moore, D. (In preparation) The Wildland Ethics Scale. Leisure
Sciences
Vagias, W., Powell, R.B., & Moore, D. (In preparation- b) Intentions to comply with
Leave No Trace practices: Assessing determinants planned behavior, perceived difficulty,
and perceived knowledge. Society and Natural Resources
Vagias, W. & Powell, R.B. (In preparation) Diffusion Strategies and Measures of Perceived
Effectiveness of the Leave No Trace Visitor Education Program in Two U.S. National
Parks. Journal of Leisure Research.
Vancini, F. W. (1989). Policy and management considerations for off road vehicles:
Environmental and social impacts. Cornell University, Ithaca, NY.
Yankoviak, B. M. (2005). Off-road vehicle policy on USDA National Forests: Evaluating user
conflicts and travel management. University of Montana.