SUPPORTING STATEMENT
FOR OMB CLEARANCE
PART A
NASA Earth to Sky: An Assessment of Global Climate Change in Visitors to Public Lands
October 22, 2010
Part A: Justification
A.1 Explanation of Circumstances That Make Collection of Data Necessary
The National Aeronautics and Space Administration (NASA) Office of Education, requests that the Office of Management and Budget (OMB) approve, under the Paperwork Reduction Act of 1995, a clearance for NASA to conduct data collection efforts related to the evaluation of NASA’s Earth to Sky Partnership.
This study (embedded
within a larger collaborative effort) will assess if visitors to
public lands are benefiting from an interagency partnership, known as
Earth to Sky (ETS), by (1) measuring awareness and understanding of
global climate change in visitors to national parks and wildlife
refuges, and (2) identifying factors that influence program
effectiveness. ETS is a NASA-funded partnership between NASA's
Space and Earth Science disciplines, the National Park Service (NPS)
and US Fish and Wildlife Service (FWS). The partnership actively
fosters collaborative work between the science and
interpretation/education communities of NPS, FWS, and NASA, with the
ultimate goal of enriching the experiences of millions of park and
wildlife refuge visitors, by bringing NASA science into the context
of the public lands experience. The project was created to facilitate
the use of NASA research and resources by NPS and FWS interpreters
and educators in their communication with citizens who visit, and are
concerned with, our nation’s natural, cultural, and historic
resources.
A main role of NASA’s education and outreach work is to support the agency’s scientific efforts. This proposal directly addresses NASA’s Strategic Sub-goal 3A: “Study planet Earth from space to advance scientific understanding and meet societal needs.” NASA conducts the bulk of the research funded by the US Climate Change Science Program. NASA research plays a critical role in the world’s understanding of the causes and consequences of climate change. Through a series of professional development efforts, interpreters and educators were exposed to NASA researchers and research. In this way, informal educators learned about NASA’s current data and initiatives related to Earth Systems Science, with an emphasis on global climate change. These participants develop and deliver interpretive and educational programs and products to visitors at parks and refuges nationwide. This research seeks to assess program effectiveness by determining if visitor understanding of climate change increases as a result of their program participation.
A.2 How the Information Will Be Collected, by Whom, and For What Purpose
How Information Will Be Collected and by Whom
Visitor surveys will be administered by members of the ETS Evaluation Team. The team consists of Dr. Theresa Coble, Associate Professor, Stephen F. Austin State University, Dr. David Smaldone, Associate Professor, West Virginia University, and other professional colleagues and or graduate students as required under the direction of the principal investigator, Dr. Coble.
Respondent
universe:
This study will administer an onsite survey to approximately 1200 visitors at selected NPS and FWS sites where ETS participants work. Possible survey locations have been identified based on the number and type of programs developed and the potential number of visitors to be reached by programs delivered at the site. In addition, researchers seek to include a diversity of sites and resources.
Potential NPS survey sites include: Kenai Fjords National Park, Yosemite National Park, Everglades National Park, Sequoia & Kings Canyon National Parks, Golden Gate National Recreation Area, Congaree National Park, Mt. Rainier National Park, Bryce Canyon National Park, Indiana Dunes National Lakeshore, Gulf Islands National Seashore, Rocky Mountain National Park, Apostle Islands National Lakeshore, Cuyahoga Valley National Park, Rock Creek Park, Death Valley National Park, and Great Smoky Mountains National Park.
Potential FWS sites include: DeSoto National Wildlife Refuge, Leavenworth Fish Hatchery, Chincoteague National Wildlife Refuge, Patuxent Research Refuge, JN “Ding” Darling National Wildlife Refuge, and several Alaska refuges.
In January 2011, potential research sites will be evaluated based upon the selection criteria and final survey locations will be chosen. Surveys will be administered onsite by evaluation team members at 5-8 of the above locations. Additional surveys may be administered with the assistance of ETS participants (and colleagues) at selected sites. Using this method, park and refuge staff will administer surveys to visitors who attend interpretive and educational programs onsite.
Research Methods:
A survey will be administered onsite to visitors who attend interpretive and educational programs and/or who are exposed to interpretive and educational products related to global climate change. A sample of visitors will be drawn from adults and young adults (18 and older) visiting participating parks and refuges during the Summer 2011 study period. Approximately 200 visitors will be surveyed at 5-8 study locations.
Sampling
plan/procedures:
At sites with interpretive and educational
programs focused on global climate change, researchers will approach
all attendees (census sampling) immediately before (control
group) or after (treatment group) selected programs, asking
adult visitors to complete the survey onsite. At select locations
park or refuge staff will administer the survey to willing
participants before or after exposure to interpretive
or educational programs. Additionally, at sites where non-personal
interpretive or educational products or services related to global
climate change are provided (e.g., site bulletins, brochures, videos,
interactive kiosks, waysides, exhibits, etc.), researchers and/or
staff members will approach a random sample of visitors before
and after exposure to onsite interpretive and educational
products, asking them to complete the survey onsite.
At sites where the effectiveness of non-personal interpretive and educational products related to climate change is assessed, the researcher or staff member will use an “every nth person” sampling approach to approximate the random selection of participants. For example, researchers will approach every 3rd or 5th visitor who reads (or appears to read) a climate change exhibit panel during the sampling time and ask them to fill out the survey. For other products, such as printed materials, randomly selected visitors will be asked to view or listen to a product and then to complete the survey. For films, a random sample of audience members will be asked to complete the survey either before or after viewing the film. Data will be collected throughout the summer months (June-August) of 2011 at the selected parks and/or wildlife refuges/hatcheries.
Instrument
administration:
Onsite surveys will be used to collect the data. Surveys will be self-administered by the visitors. As noted above, after approaching the visitors, they will be asked to participate in the study, and be will notified that participation is voluntary, and results will remain confidential and anonymous. The survey will include questions about: 1) demographics, 2) awareness and understanding of global climate change, 3) interpretive and learning outcomes, and 4) perception of trust in various climate change information sources.
Expected
response rate/confidence levels:
The expected response rate should be between 80-90% of approached visitors, due in part to anticipated response rates of greater than 90% among those who attend interpreter or educator-led programs. Previous studies at NPS sites using similar onsite survey methods have obtained high response rates (Coble et al., 2007).
Strategies
for dealing with potential non-response bias:
As response rates increase, non-response bias becomes less problematic. However, a contact log will be used to track all visitors who are invited to participate in the study. Simple non-response bias checks comparing participation rates by gender, group size and composition, time of day, etc., will be conducted based on contact log information.
Description
of any pre-testing and peer review of the methods and/or instrument
(recommended):
The survey instrument has been peer reviewed by several academic experts associated with recreation and park research at both Stephen F. Austin State University and West Virginia University. Survey modifications were made based on peer review comments. In addition, education specialists at NASA, FWS, and the NPS assisted in the survey peer review process. Finally, several items used to assess public attitudes and understandings related to climate change were obtained from survey instruments developed and tested by the Yale Project on Climate Change Communication at Yale University.
For What Purpose
The purpose of this project is to assess if visitors to NPS and FWS sites are benefiting from an interagency partnership, known as Earth to Sky, by (1) measuring the awareness and understanding of global climate change among visitors who have and have not been exposed to onsite interpretive and educational programs and products, and (2) identifying factors that influence program effectiveness. An onsite survey will be administered to park visitors to assess their awareness and understanding of global climate change; interpretive and educational outcomes; and their trust in various sources of climate change information. Data will be collected at 5-8 NPS and FWS sites from June to August, 2011. Results will help NASA and other managers of the Earth to Sky partnership assess the success of the partnership efforts by examining program effects upon the ultimate “end of the pipe” target audience—the public. Results will help project leaders refine project activities and procedures while encouraging continued inter-agency collaboration.
Figure 1: Research Questions, Data, and Methods |
||||
Research Questions |
Data |
Sample |
Measures/Domains |
Analytic Approach |
Climate Change Awareness & Understanding |
|
|
|
|
|
Visitor survey |
5-8 sites; approximately 1200 total visitors. |
Survey items # 1, 2, 3, 7, 8, 9,10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 22, 23, 24, 25, |
Descriptive and inferential statistics (e.g., t-tests, analysis of variance, chi-square tests) |
Interpretive & Learning Outcomes |
|
|
|
|
|
Visitor survey |
5-8 sites; approximately 1200 total visitors. |
Survey items # 4, 5, 6 |
Descriptive and inferential statistics (e.g., t-tests, analysis of variance, chi-square tests) |
Trust in Information Sources |
|
|
|
|
|
Visitor survey |
5-8
sites; approximately 1200 total visitors. |
Survey items # 21, |
Descriptive and inferential statistics (e.g., t-tests, analysis of variance, chi-square tests) |
Factors that Influence Observed Outcomes |
|
|
|
|
|
Visitor survey |
5-8
sites; approximately 1200 total visitors |
Survey items # 3, 9, 14, 15, 16, 21, 23, 26, 27, 28, 29, 30, 31, 32, 33, 34 |
Descriptive and inferential statistics (e.g., t-tests, analysis of variance, chi-square tests) |
The overall goal of
the ETS professional development efforts is to advance NASA’s
capability to improve public understanding and appreciation of its
science and technology, through professional development for some of
our nation’s most accomplished informal educators: National
Park Service and other public agency Interpreters and educators. The
first ETS workshops were held in 2004. The 2009 and 2010 ETS
workshops focused on the research and implications of global climate
change and its impacts on natural, cultural, and historic locations.
Accurate, accessible,
and personally meaningful information on climate change is critically
important to our nation, and is in short supply. Climate change is a
complex topic prone to many misperceptions. According to the National
Science Foundation (2008), “Although Americans seem to accept
climate change, or global warming, as a real phenomenon, most do not
seem to have a great deal of concern about it.”
A recent Gallup poll (2008) found that only 40% of Americans
believe that the effects of global warming will have a serious threat
to them in their lifetime. Additionally, the average high-school
student fails a quiz on the causes and consequences of climate
change.
However, a 2007 study
of survey trends addressing public opinion about global climate
change indicated that public awareness of the issue had risen
dramatically over the past 20 years. While only 39% of the public
reported having “heard or read anything about the greenhouse
effect” in 1986, by 2006 that number had reached 90%. Much of
this increase in awareness is attributed to the increase in media
coverage on the issue of global climate change. Although awareness of
the issue has risen, a 2006-2007 ABC News study found that only 11%
of Americans felt they knew “a lot” about global warming,
an increase from 5% in 1997 (Nisbet & Myers, 2007, p.445).
These factors indicate
that while awareness has risen, public understanding of the issue of
global climate change has not enjoyed the same increase. Clearly
there is a need for improved public understanding. This need for
accurate and timely information combined with NASA’s topical
expertise provided the impetus for creating the ETS climate change
workshops. This project aims to address the need for further public
understanding of the issue of global climate change by infusing NASA
science into the many informal and formal educational opportunities
afforded by National Parks and Refuges. These natural and historic
locations provide ideal settings for communicating the immediate and
obvious effects of climate change, from rapidly melting glaciers,
increased intensity and length of fire seasons, to flooding of
archeological and historical treasures.
Studies conducted by
The Center for Climate Change Communication at Yale University
suggest that the messenger or source of climate change information is
critical to public understanding of this complex issue. Certain
messengers invoke trust while others are treated more skeptically. A
2008 national survey conducted by the Center found that 82% of
Americans trusted scientists as a source for climate change
information while only 47% trusted mainstream media. This suggests
that park and wildlife staff, of which many are scientists as well as
educators and interpreters, might offer an ideal avenue by which to
funnel climate change information to an uncertain public
(Leiserowitz, et.al, 2008, p.42).
Other studies found that encouraging attitudinal change regarding “green” behaviors will not be effective unless people care about the issue personally and are motivated and able to take action about climate change (Ockwell, et. al, 2009, p. 307). Park and wildlife interpreters and educators are trained in program delivery methods that facilitate visitors in making both emotional and intellectual connections to the resources being interpreted. Their theme-based programs are developed with an understanding of their audience and extensive knowledge about the resource. NPS and FWS staff seeks to make their climate change programs relevant to visitors by examining the tangible effects of climate change within their park or refuge. They also engage the visitor through communication and interaction to examine personal behaviors that might impact climate change and offer alternatives to those behaviors. This can be significant because although Leiserowitz et al. (2008), found that most Americans consider themselves fairly well informed about the causes, consequences, and solutions to global warming, more than half said that they needed more information to form a firm opinion about global climate change and what actions might be taken to curb it.
A.3 Use of Improved Information Technology to Reduce Burden
The data collection plan reflects sensitivity to issues of efficiency, accuracy, and respondent burden. Therefore, the surveys were designed so that only questions not available elsewhere were asked. At this time it is not feasible to obtain high-quality data that captures onsite experiences and outcomes electronically. Thus, surveys will be conducted onsite, in person, using traditional recording procedures.
A.4 Efforts to Identify and Avoid Duplication
This effort will yield data to evaluate visitor outcomes related to their participation in climate change programming provided at national parks, wildlife refuges and fish hatcheries. The evaluation of visitor outcomes provides an important indicator of the effectiveness of the interagency Earth to Sky program. At present, there is no similar evaluation being conducted in the U.S., thus, there is no alternative source to obtain the visitor outcome information.
Also, a variety of research sites will be chosen to ensure maximum variation in the data collected, avoiding unnecessary duplication in research locations and site attributes. Similarly, a variety of interpretive and educational programs and products will be incorporated into the data collection and analysis to further reduce unnecessary duplication.
A.5 Efforts to Minimize Burden on Small Business or Other Entities
The primary entities for the study are visitors to public lands. Burden is minimized for all respondents by requesting only the minimum required to meet study objectives. All primary data collection will be coordinated by the principal investigator, reducing burden to agency staff. No small businesses will be involved as respondents.
A.6 Consequences of Less-Frequent Data Collection
If the proposed data were not collected, NASA would potentially not be able to fulfill some of its objectives regarding their Strategic Sub-goal 3A: “Study planet Earth from space to advance scientific understanding and meet societal needs.” Thus, federal resources might be allocated, and program decisions might be made, in the absence of valid assessments of the effectiveness of current programs to increase public awareness and understanding of NASA-derived global climate change science and information.
A.7 Special Circumstances Requiring Collection of Information in a Manner Inconsistent with Section 1320.5(d)(2) of the Code of Federal Regulations
There are no special circumstances associated with this data collection.
A.8 Federal Register Comments and Persons Consulted Outside the Agency
In accordance with the Paperwork Reduction Act of 1995, NASA published a notice in the Federal Register announcing the agency’s intention to request an OMB review of data collection activities. The notice was published on May 19, 2010, in volume 75, number 96, page 28071. A 30-day period for public comments was provided. To date, no comments were received.
The survey instruments were developed by the principal investigator and the evaluation team with input from Earth to Sky project leaders including Ms. Anita Davis from NASA’s Office of Education.
A.9 Payments to Respondents
There will be no payments to respondents.
A.10 Assurance of Confidentiality
Every effort will be made to maintain the privacy of respondents, using several procedural and control measures to protect the data from unauthorized use. The evaluation team will follow procedures for ensuring and maintaining confidentiality, consistent with the Family Educational Rights and Privacy Act of 1974 (20 USC 1232g), the Privacy Act of 1974 (P.L. 93-579, 5USC 552a), and the Federal common rule or Department final regulations on protection of human research subjects. Data to be collected will not be released with individual visitor or educator/interpreter information, and data presented will occur only in aggregated form. A statement to this effect will be included in a letter accompanying each survey and will be read to visitors before administering the survey. Respondents will be assured that all information identifying them will be kept confidential, in compliance with the legislation (P.L. 103-382).
The procedures to protect data during information collection, data processing, and analysis activities are as follows:
All respondents included in the study sample will be informed that the information they provide will be used only for the purpose of this research. Individuals will not be cited as sources of information in prepared reports.
To ensure data security, all individuals hired by the contractor are required to adhere to strict standards and sign an oath of confidentiality as a condition of employment.
Hard-copy data collection forms will be delivered to a locked area at the contractor’s office for receipt and processing. The contractor will maintain restricted access to all data preparation areas (i.e., receipt, coding, and data entry). All data files on multi-user systems will be under the control of the principal investigators, with access limited to project staff on a “need-to-know” basis only.
Individual identifying information will be maintained separately from completed data collection forms and from computerized data files used for analysis. No respondent identifiers will be contained in public use files made available from the study, and no data will be released in a form that identifies individuals.
The principal investigator will also have all data collection protocols and surveys reviewed by the Institutional Review Board (IRB) at Stephen F. Austin State University (SFA). Prior to their use, SFA’s IRB will approve all data collection instruments, including the visitor consent forms and visitor assent forms. The IRB will assure that the data collection protocols and procedures, including consent forms, abide by strict confidentiality procedures. See Appendix F and G for draft versions of the consent and assent forms.
A.11 Questions of a Sensitive Nature
The questions included on the data collection instruments for this study do not involve sensitive topics. That is, although the subject of global climate change is sometimes at the center of controversial and politically charged public debates, asking park and refuge visitors to share their personal views and understandings, their reactions to professionally prepared and delivered interpretive and educational programs and products, and their possible subsequent pro-environmental or citizenship behaviors provides an opportunity for these respondents “to be heard”—whatever their point of view may be. Research conducted to elicit respondent attitudes, perceptions and behaviors about topics and behavior that is non-socially censured can be regarded as non-sensitive in nature.
A.12 Estimates of Respondent Burden
Figure 2 presents estimates of the annualized reporting burden for the surveys. The surveys will be administered to approximately 1200 visitors. We will assume an overall response rate of 89 percent, so that we may need to approach approximately 1350 in order to complete 1200 surveys. Estimates for the hour burden are based on time estimates provided by the survey developers of the originals surveys based upon similar surveys used in comparable evaluations.
Figure 2 Estimates of 2011 Annualized Burden Hours and Cost |
|||||||
Data Collection Sources |
Number of Respondents a |
Frequency of Response |
Total Minutes per Respondent |
Response Burden in Hours |
Estimated Cost Per Hour b |
Costs per Respondent |
Total Burden (Costs) |
Visitor Surveys |
1,200 |
1 |
15 |
322.5 |
$109 |
$29.44 |
$35,333 |
|
|
|
|
|
|
|
|
Notes: a Total number of initial contacts 1350. Expected response rate is 89%. b Approximately 1/3 of the funds that NASA and NASA-funded project partners allocated to researchers to evaluate the ETS project (i.e., 1/3 of $106,000 or $35,333) will be devoted to assessing visitor outcomes. |
|||||||
A.13 Estimates of the Cost Burden to Respondents
There is no annualized capital/startup or ongoing operation and maintenance costs associated with collecting the data. Other than their time to complete the surveys, which is estimated in Figure 2, there are no direct monetary costs to respondents.
A.14 Estimates of Annualized Government Costs
This information collection activity has been developed in the performance of the Prime Contract Number: NNH09CF00C and Subcontract Agreement: S09112-02. The subcontract agreement is a fixed price agreement; therefore, there are no separate costs associated with the development of survey data collection instruments.
A.15 Changes in Hour Burden
This is a new collection of information.
A.16 Time Schedule, Publication, and Analysis Plan
The schedule shown in Figure 3 displays the sequence of activities required to conduct the information collection activities and includes key dates for activities related to data collection, analysis, and reporting.
Figure 3 Time Schedule |
||
Activities and Deliverables |
Responsible Party |
Date |
Chose NPS & FWS sampling sites |
Evaluation Team |
Winter 2011 |
Survey Data Collection |
Evaluation Team |
June – August 2011 |
Data Analysis of Visitor Survey |
Evaluation Team |
Fall 2011 |
Draft Report |
Evaluation Team |
May 31, 2012 |
The principle investigators and their team will collect most or all of the data at NPS and FWS sites. Some agency personnel may assist in onsite data collection as well.
The surveys will be used to assess awareness and understanding of global climate change, interpretive and educational outcomes, and perceptions of trust in various sources of global climate change information among visitors. The analysis will incorporate descriptive and inferential statistics to examine outcomes and pinpoint factors that influence observed outcomes. An evaluation report will be prepared based on survey findings.
A.17 Display of Expiration Date for OMB Approval
NASA is not requesting a waiver for the display of the OMB approval number and expiration date on the data collection instruments.
A.18 Exceptions to Certification Statement
This submission does not require an exception to the Certificate for Paperwork Reduction Act (5 CFR 1320.9).
| File Type | application/msword |
| File Title | Abt Double-Sided Body Template |
| Author | Abt Associates Inc |
| Last Modified By | sfasu |
| File Modified | 2010-10-25 |
| File Created | 2010-10-25 |