Guidelines for Programmatic Review Process

National Park Service

U.S. Department of the Interior

Guidelines and Submission Form for Programmatic Review for NPS-Sponsored Public Surveys

June 2011

Contents

Introduction

Authorities and Guidance within the Scope of the Programmatic Review

Topic Areas

TOPIC AREA 1:- Respondents’ Characteristics and Knowledge

TOPIC AREA 2 - Trip Planning

TOPIC AREA 3 - Trip Behaviors

TOPIC AREA 4 - Preferences/Motives/ Attitudes

TOPIC AREA 5 - Crowding and Visitor Experiences

TOPIC AREA 6 - Evaluations/Opinions of Services, Facilities, and Management

TOPIC AREA 7 – Economic Impact and Benefit Analysis

Submission and Review Process

Frequently Asked Questions

Programmatic Review Instructions and Forms

Checklist for Submitting a Request for Programmatic Review

Introduction

The National Park Service (NPS) sponsors public surveys to provide park managers with information needed for park planning, management, operations and evaluation of performance related to protecting park resources and meeting the needs of the public. In consultation with the Office of Management and Budget (OMB) and the Department of the Interior (DOI), the NPS has developed a programmatic review process for NPS-sponsored public surveys. It streamlines the process required by the Paperwork Reduction Act of 1995. This booklet provides guidelines as of May 2006 for using the programmatic process and a copy of the necessary submission form.

What Kinds of Studies Are Covered?

The programmatic review applies to NPS-sponsored surveys designed to furnish useful applied knowledge to NPS managers and planners. Questions asked under the programmatic review must show a clear tie to NPS management and planning needs. The programmatic review may only be used for non-controversial surveys that are unlikely to attract or include topics of significant public interest in the review process. Investigators who are unsure if a study qualifies under the programmatic review are strongly encouraged to contact the Social Science Branch for clarification early in the planning process.

Who May Be Surveyed?

The programmatic review is limited to three specific segments of the public—park visitors, potential park visitors, and residents of communities near parks.

Park Visitors: Park visitors include persons visiting any unit of the National Park System for recreational purposes, to participate in educational activities, to use resources legally available in a given unit, and for non-recreational purposes, such as traveling through or making deliveries. “Visitors” include the general public, participants in organized tour groups, teachers and students participating in NPS programs or activities, commuters, concession or partner employees, and subsistence resource users.

Potential Park Visitors: Potential park visitors include those individuals, who might visit any unit of the National Park System for recreational purposes, to participate in educational activities, to use resources legally available in a given unit, and for non-recreational purposes, such as traveling through or making deliveries.

Residents near Parks: Residents near parks include people living in gateway communities near any unit of the National Park System, communities within the boundaries of park units and related areas (e.g., National Heritage Areas, National Scenic Trails), in-holders, concession or partner employees, and subsistence communities within park boundaries or who traditionally use park resources.

What Kinds of Information Collection Methods Are Covered?

The requirements described in this publication apply to any information collection in which the same questions are asked of ten or more people. This includes all forms of surveys (mail, on-site, telephone, Web-based), focus groups, semi-structured interviews, and field experiments. Studies in which information is collected about people solely through observation are exempt from the approval process.

Authorities and Guidance

The authorities and guidance that apply to the programmatic process are:

• Paperwork Reduction Act of 1995 (P.L. 104-13 May 22, 1995)

• 5 CFR Part 1320: Controlling Paperwork Burdens on the Public; Regulatory Changes Reflecting Recodification of the Paperwork Reduction Act

• United States Code, Title 44, Chapter 25: Coordination of Federal Information Policy

• OMB, The Paperwork Reduction Act of 1995: Implementing Guidance (draft, August 16, 1999)

• DOI Collection of Information from the Public: Interim Guidelines (March 20, 1997)

• NPS Director’s Order #78, Social Science, Section III (October 7, 2002)

• NPS, Social Science Surveys and Interviews in the National Parks and for the National Park

• Service: A Guide to NPS and OMB Approvals (July 2002)

• OMB, Guidance on Agency Survey and Statistical Information Collections: Questions and Answers When Designing Surveys for Information Collections (January 2006)

Topic Areas within the Scope of the

Programmatic Review

To qualify for the programmatic review, all questions in a survey must fit within one or more of the approved topic areas and must be approved by the NPS and OMB. Researchers have flexibility, within accepted standards of good survey design and OMB regulations, to develop specific questions within the topic areas. The seven topic areas are identified below. A description of the scope of each topic area follows.

TOPIC AREA 1: RESPONDENTS’ CHARACTERISTICS AND KNOWLEDGE

The questions in this section will be used characterize the population of respondents participating in each sample. Individual characteristics collected will be attributes of individual park visitors or visitor groups, potential visitors or groups, and residents of communities near parks. Individual characteristics collected will be relevant and limited to the mission, management, and/or operations of National Park System units. The scope of the information will be limited to those that are germane to the topic being studied and relevant to the park and its management. This information will be used to check the representativeness of each sample against the population. Variables such as age, education, and knowledge are often good predictors of demand and visitation behavior.

TOPIC AREA 2 - TRIP PLANNING

The section on Trip Planning includes aspects of travel which affect a trip or decisions which individuals make prior to, during, or following their trips to parks, related areas, and nearby communities. Trip characteristics will be relevant to the mission, management, and/or operations of National Park System units that are included in the scope of this topic area. The scope of the information collected will be limited to those that are germane to the topic being studied and relevant to the park and its management.

TOPIC AREA 3 - TRIP BEHAVIORS

Individuals participate in many activities during their visits to parks, related areas and nearby communities. The questions in this section will be used to identify individual activities, behaviors, or uses of natural and cultural resources which are relevant to the mission, management, and/or operations of National Park System units Understanding the current and future uses and purposes of park visitors will be helpful to managers when considering updating park management plans and educational efforts.

TOPIC AREA 4: PREFERENCES/MOTIVES/ATTITUDES

Questions in this topic area will be used to collect data concerning the public's awareness and observations of the natural and social environments in the parks. Preferences, motives and attitudes will be measured to determine how individual observations influence overall experiences. For purposes of the programmatic review process perception questions will be limited to topics the park or the NPS can control and manage goods and services currently or potentially

TOPIC AREA 5 - CROWDING AND VISITOR EXPERIENCES

Crowding and conflict are among the most intractable problems faced by recreation mangers. Concern over rising visitation in parks, and accompanying impacts on resources and on visitor experience, has led the National Park Service to focus increasing attention on the concept of crowding and carrying capacity. Crowding and conflict arise from the social encounters and interactions among recreationists ranging from basic competition for space (e.g., crowding) to conflicts between forms of activity and related expressions of acceptable or appropriate use. Research on crowding norms and conflict has been particularly helpful in establishing guidelines for Limits of Acceptable Change planning efforts by identifying key social impact indicators and the data required to monitor them. The questions in this topic area will help managers understand the factor associated with the acceptability of crowing and visitor carrying capacity.

TOPIC AREA 6 - EVALUATIONS/OPINIONS OF SERVICES, FACILITIES, AND MANAGEMENT

Public opinion of the services and facilities helps management teams understand the values people hold in relation to park resources and the visitor experience and is critical to creating a plans that can be successfully implemented. Underlying all fundamental planning decisions are competing values, which must be resolved by a decision as to which value is of greater importance in a particular situation. A planning decision is the compromise between competing values at a given point in time. Understanding public values enables the management teams to make informed planning decisions. The questions in this topic area will be used to help managers learn about public concerns, issues, expectations, and values. The collected data can be used to provide information on how people use the park, on existing and desired conditions and on the acceptability of proposed indicators.

TOPIC AREA 7 – ECONOMIC IMPACT AND BENEFIT ANALYSIS

Visitor expenditure and income information is needed to calculate the economic impact and benefit of park visitation. Economic impact measures how much the money people spend visiting parks and surrounding areas contributes to the local economy in terms of jobs and income. Economic benefit measures how much visitors value a park above what they spend, or the increase in social welfare a park provides. . Accurate impact assessment requires identification of those portions of expenditures that occur in the local region and inside the park. Questions will be used to develop average spending estimates for visitors to the park and local region. Income and income foregone questions provide information about the value, or economic benefit, of a visit to the park.

Surveys outside the scope of the programmatic review require clearance through the standard information collection review process outlined by the Paperwork Reduction Act.

Submission and Review Process

The Principal Investigator (PI) is responsible for initiating a request for programmatic review and providing a complete and accurate package of review materials. PIs may be park resource management specialists, NPS interpretive designers, agency scientists, researchers from universities, and individuals from organizations cooperating with the NPS, among others. The submission must include:

• (a) a completed programmatic review form,

• (b) a complete copy of the proposed data-collection instrument (e.g., survey or interview guide), and

• (c) other supporting materials (such as cover letters, introductory scripts, follow-up letters, and survey logs).

The programmatic review form is available in this booklet and online at the following Web site: http://www.nature.nps.gov/socialscience/docs/ex_guide.pdf

The request for programmatic review and submission of a complete and accurate package must be made at least 60 calendar days prior to the first day the PI wishes to minister the survey instrument to administer the instrument to the public. Submissions received in the busiest months of May and June will likely experience significant delays in review times.

The NPS will provide an administrative and technical review of all materials and will notify the PI of the results. If revisions are necessary, the PI should complete them as soon as possible so that the NPS can forward materials to OMB in a timely fashion. If no revisions are necessary, the NPS will promptly submit the review package to OMB for review. A description of the steps in the programmatic review process follows.

Step 1

The PI completes the Programmatic Review Form and prepares the proposed data-collection instrument. Instructions are provided for each item on the form (see page 6). In addition, questions in the proposed data-collection instrument must have each topic area clearly identified. Please list topic areas by number and title; for example "Topic Area 1-Respondent Characteristics." Topic areas must be included with all instruments, both quantitative and qualitative (including focus group questions). Submissions lacking this information will be returned to the PI, resulting in delays in the review process.

Step 2

The PI submits the completed form and a copy of the proposed survey instrument to the NPS Information Collection Review Coordinator for review. The submission package must include:

1. any introductory script used in contacting the public,

2. all cover letters, postcard reminders or follow-up letters to be sent to potential respondents,

3. all survey or interview questions, each question being clearly identified as to the topic area under which it is being submitted

4. (topic area number and title),

5. necessary Paperwork Reduction Act compliance language inserted into the survey instrument1,

6. if applicable, scripts for non-response bias analysis follow-ups, and

7. any other supporting materials (such as survey logs).

Be sure that for qualitative surveys (including focus groups) there is an introductory script, including the necessary PRA compliance language. For person-to-person information collections, such as on-site interviews, telephone interviews and focus groups, a short statement describing how the PI intends to communicate PRA compliance information to respondents is required in the description of the survey methodology. All submission packages must be formatted as MS Word documents (the recent version up to and including MS Word XP) and sent via e-mail to the NPS Social Science Branch at the following address: [email protected]

**Please do not send materials in PDF format, as these will be returned.

Step 3

The NPS Social Science Branch staff conducts an administrative and technical review of the submission. The staff recommends either: approval, revision, resubmission under the Standard Paperwork Reduction Act approval process, or rejection of the proposed survey. The NPS Information Collection Review Coordinator will make the decision based upon a complete review of the submitted package, and the PI will be promptly notified.

Should a submission be rejected, the PI may submit an appeal, in writing, to the Associate Director, Natural Resource Stewardship and Science, for a final decision.

Step 4

If approved by the NPS, the NPS Social Science Branch staff transmits the submission to OMB for final review.

Step 5

OMB reviews the submission and notifies the NPS of approval or necessary revisions. Should OMB have specific questions about the survey instrument or proposed methodology, the NPS

Information Collection Review Coordinator will immediately inform the PI and work with the PI to make necessary revisions. The NPS Information Collection Review Coordinator will submit the PI's revisions to

OMB and inform the PI of the results.

Step 6

If approved by OMB, the NPS Information Collection Review Coordinator will notify the PI immediately and assigns an OMB number and an expiration date not to exceed three years or the expiration date OMB has assigned to the overall Program. The standard expiration date applied to Programmatic approvals is six months after the survey dates listed on the approval form. If OMB requires any special conditions for the approval (e.g., furnishing actual response rates to surveys), the PI will be informed and the conditions must be met for approval.

Step 7

The PI prepares a final revised survey instrument, submitting both a final electronic copy and hardcopy to the NPS Social Science Branch. The final survey instrument must (without exception) include the following:

a) the OMB number

b) the expiration date, and

c) the Paperwork Reduction Act statement.

Additions or changes to a survey instrument after it has been approved, even within the specific topic areas, are not allowed by OMB. An exception is that questions may be deleted after approval by OMB, if necessary. In addition, the PI must provide the NPS Social Science Branch with an archive copy of the final report describing the results of the survey.

All archive copies of reports will be catalogued in the Social Science Studies Collection, physically housed in the NPS Washington Office. Reports housed in the Social Science Studies Collection will also be made available electronically to park managers and the public through the NPS Focus Digital Library and Research Station.

Other Approvals

In addition to OMB approval, PIs conducting surveys within units of the National Park System must meet requirements of the National Park Service Research Permit and Reporting System. Research permits under this system are issued by the specific park unit(s) in which the research takes place. Parks may have additional requirements as well. PIs should contact staff members of respective park sites to make this determination. Information on the NPS Research Permit and Reporting System is available online at the following Web site:

http://science.nature.nps.gov/research

Frequently Asked Questions

1. Is approval required for information collected in focus groups or field experiments?

Information collected in focus groups or field experiments must be approved if the participation includes ten or more persons and they are asked identical questions. If a series of focus groups is conducted on the same topic using substantially similar questions and the total participants in all groups combined is more than ten, then approval is required.

2. I am a graduate student conducting a study of national park visitors. Do I need approval for my study?

NPS and OMB approval is required if the study is conducted, sponsored, or funded by the NPS. If you are receiving financial or in-kind support from the NPS, approval will be required. The submission should list your major professor or faculty advisor as the PI. Approval by university Institutional Review Boards (Human Subjects Committees) does not substitute for NPS and OMB approval. However, research occurring in national park units that is funded by external sources (e.g., independent grants) and is not assisted or reviewed by the NPS in any way does not require NPS and OMB approval.

3. I intend to study visitor response to interpretive exhibits. I will observe visitors' behavior as they approach, read, and interact with the exhibits under different experimental conditions. Will I need approval?

Observations are exempt from the review and PRA approval process if no information is solicited from the public. Also exempt are questions asked of the person that are specific to that individual or result from observation. Approval is needed if standardized questions will be asked of ten or more observed people.

4. How long does the Programmatic review process take?

The request for Programmatic review, and submission of a complete and accurate review package, must be made at least 60 calendar days prior to the first day the PI wishes to administer the survey in the field.

5. How do I provide Paperwork Reduction Act compliance information to the respondents of my survey?

Respondents to NPS-sponsored surveys must be informed that the information collection is approved and in compliance with the Paperwork Reduction Act. Depending on the type of survey instrument used, compliance information is passed along to the respondents in different ways. The compliance information can be printed on an on-site or mail-back questionnaire. General compliance information can be presented verbally in face-to-face interviews, focus groups, or telephone surveys. Additional information will need to be made available to respondents upon request. Sample compliance information appropriate to different situations can be found on the NPS Social Science Branch Web site.

6. I will be surveying small groups of park visitors. Is approval required?

NPS and OMB approval is required if identical questions are asked of ten or more persons.

7. Do I need approval if I am pre-testing a survey for later submission?

Pre-testing of survey instruments and methodology is encouraged. If pre-testing involves collecting the same information from ten or more members of the public, clearance for the pre-test is required. The request for approval of the pre-test can be submitted separately or with the final survey package, whichever is appropriate. However, if done with 9 or less people, pre-testing instruments prior to submission is recommended, especially if you are new to the survey design process and/or the survey includes questions that have not already been well-tested.

8. I intend to offer respondents to my survey a small token of thanks from the park cooperating association. Is this acceptable?

Generally, OMB discourages use of incentives in federal surveys because of the possibility of biasing a sample. Under certain circumstances, non-monetary incentives can be used. You should contact the NPS Social Science Branch to discuss your proposed use of an incentive in your survey.

9. Are there any restrictions on the use of Web surveys?

Use of Web surveys as an option for respondents is acceptable. The Web should not be the only method of survey administration. There are restrictions on surveys of Web site users. If you intend to do a survey of this population, please contact the NPS Social Science Branch early in your planning process. Further, in terms of sample selection, the Web should not be the means by which a sample population is recruited, since this will bias the sample. An exception occurs when the population being surveyed consists of visitors to a Web site, such as www.nps.gov. Contact the Social Science Branch if you are planning such a survey.

10. How should I anticipate my expected response rate?

Expected response rates should be based on previous like studies in which the method of data collection and sample population were similar. In reporting your expected response rate, please be sure to justify it with specific reference to these similar studies. If you are having difficulty with this, please contact the Social Science Branch for assistance.

11. Can I assure my participants that their identity (if known) will remain confidential?

Information collections approved under this program are subject to Freedom of Information Act requests. The Department of the Interior has no statutory authority to exempt studies from such requests. Therefore, confidentiality cannot be pledged. However, any information on surveys that identifies respondents can be removed or stored separately from survey databases so that the two are not linked. These steps should be disclosed to respondents.

Programmatic Review

Instructions and Form

1. Insert a title for the proposed study and include park name, if appropriate (e.g., Yosemite National Park Visitor Study). Include the date of submission of the approval request to NPS.

2. Summarize the proposed study with an abstract not to exceed 150 words.

3. Fill in the PI contact information. The NPS Social Science Branch will communicate with the PI listed here throughout the entire process. For studies in which graduate students are taking an active role, please list the faculty advisor as the PI.

4. Fill in the park or program liaison contact information; list only one park liaison for the purposes of the review process, even if a PI is conducting a multi-park study.

5. List the park(s) in which the data collection will be conducted or the park(s) for which the data is being collected.

6. List the time period in which the survey will be conducted, including specific starting and ending dates. The starting date should be at least 60 days after the submission date.

7. Check the type(s) of information collection instrument(s) that will be used. If “other,” please explain.

8. Provide a brief justification for the study, its purpose, goals, need for specific information, and utility to managers. NPS Social Science Branch staff can provide assistance as needed.

9. Provide a description of the survey methodology. This description must be specific and include each of the following:

a) the respondent universe (e.g., all adult visitors over 16 years of age to Yosemite National Park from Memorial Day to Labor Day 2012);

b) the sampling plan and all sampling procedures, including how individual respondents will be selected and a justification for the planned sample size;

c) how the instrument will be administered in the field, including follow-up procedures to increase response rates;

d) expected response rate and confidence levels, including a justification of the anticipated response rate by citing specific studies similar to the proposed one and their response rates;

e) a plan for analyzing and reporting the implications of any non-response bias detected (include a copy of your survey log, if applicable);

f) A description of any pre-testing and peer review of the methods and/or instrument.

10. Fill in the total number of initial contacts and the total number of expected respondents.

11. Fill in the estimated time to complete the initial contact and the survey instrument (in minutes).

12. Fill in the total number of burden hours. Burden hours refer specifically to interaction with the sample, including initial contact, reviewing instructions, and filling out a survey. Burden on non-respondents (such as initial contact interviews with individuals declining to participate) should be included in this total.

13. Provide a brief description of the reporting plan for the data being collected.

This might include a final technical report to the park, a briefing for park managers, a Park Science article, a peer-reviewed journal article, etc. A copy of all survey reports must be archived with the NPS Social Science Branch for inclusion in the Social Science Studies Collection.

Checklist for Submitting a Request for

Programmatic Review

• Survey population includes only park visitors, potential park visitors, and/or residents of communities near parks.

• All questions in the survey instruments are within the scope of the topic areas covered by the Programmatic review, are non-controversial, and are designed to furnish useful applied knowledge to NPS managers and planners.

• The Programmatic review package is being submitted to the NPS Social Science Branch at least 60 days prior to the first day the PI wishes to administer the survey to the public.

The Programmatic review package includes:

• a completed Programmatic review form

• a copy of the survey instrument or telephone interview script (with each question clearly identified as to the topic number and area under which it is being submitted)

• other supporting materials, such as

• cover letters to accompany mail-back questionnaires

• scripts for initial contact of respondents and for any follow-up calls as part of a non-response bias check

• a survey log recording the disposition of contacts, including refusals

• necessary Paperwork Reduction Act compliance language

• follow-up letters/reminders sent to respondents

The survey methodology presented on the Programmatic review form includes a specific description of:

• the respondent universe

• the sampling plan and all sampling procedures, including how respondents will be selected and a

• justification for the planned sample size

• how the instrument will be administered, including follow-up procedures

• expected response rate (with justification) and confidence levels

• plan for non-response bias analysis

• a description of any pre-testing and peer review of the methods and/or the instrument

• The burden hours reported on the Programmatic review form include the number of burden hours associated with the initial contact of all individuals in the sample (i.e., including refusals), if applicable, and the burden associated with individuals expected to complete the survey instrument.

• The package is properly formatted and sent to the NPS Social Science Branch.

All submission packages must be formatted as MS Word documents

(any recent version up to and including MS Word XP) and sent via

e-mail to the NPS Social Science Branch at the following address:

Phadrea D. Ponds

Information Collections Review Coordinator

Environmental Quality Division

Social Science Branch

National Park Service

[email protected]

970-267-7213 (Office)

Form for Submitting a Request for

Programmatic Review

National Park Service

U.S. Department of the Interior

P rogrammatic Review for NPS-Sponsored Public Surveys

1.

Project Title  Submission Date:

2.

Abstract:

(not to exceed 150 words)

3.

Principal Investigator Contact Information

First Name:

Last Name:

Title:

Affiliation:

Street Address:

City:

State:

Zip code:

Phone:

Fax:

Email:

4.

Park or Program Liaison Contact Information

First Name:

Title:

Park Office/Division:

Street Address:

City:

State:

Zip code:

Phone:

Fax:

Email:

Project Information

5.

Park(s) For Which Research is to be Conducted:

6.

Survey Dates:

(mm/dd/yyyy)

to

(mm/dd/yyyy)

7.

Type of Information Collection Instrument (Check ALL that Apply)

Mail-Back Questionnaire

On-Site Questionnaire

Face-to-Face Interview

Telephone Survey

Focus Groups

Other (explain)

8.

Survey Justification: (Use as much space as needed; if necessary include additional explanation on a separate page.)

9.

Survey Methodology: (Use as much space as needed; if necessary include additional explanation on a separate page.)

10.

Total Number of Initial Contacts | Expected Respondents:

11.

Estimated Time to Complete Initial Contact | Instrument (mins.):

12.

Total Burden Hours:

13.

Reporting Plan:

Mission Statement

The objectives of the NPS Social Science Branch are to conduct and promote state-of-the-art social science related to the mission of the National Park Service and deliver usable knowledge to NPS managers and to the public.

For additional information, contact:

Social Science Branch

National Park Service

Natural Resource Stewardship and Science (NRSS)

1201 Oakridge Dr., Suite 200

Fort Collins, CO 80525

http://www.nature.nps.gov/socialscience

Programmatic Review for NPS-Sponsored Public Surveys, Focus Groups, and Field Experiments Page 29