0582 ss 062218 Part A revised

SUPPORTING STATEMENT

EVALUATION OF PUBLIC VISITORS’ EXPERIENCE OF EXHIBITS AT

MOKUPAPAPA DISCOVERY CENTER

OMB CONTROL NO. 0648-0582

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary.

This request is for extension of a currently approved information collection.

The enabling legislation for the National Marine Sanctuary system, the National Marine Sanctuaries Act (NMSA), denotes specific educational mandates. Section 309(c)(1) of the NMSA states that one of the purposes of the Office of National Marine Sanctuaries (ONMS) is:

“ . . .to enhance public awareness, understanding, appreciation and wise and sustainable use of the marine environment, and the natural historical, cultural and archeological resources of the national Marine Sanctuary System. Efforts supported, promoted, or coordinated under this subsection must emphasize the conservation goals and sustainable public uses of national marine sanctuaries and the System.”

In 2005, the planning committee of the National Marine Sanctuary Program (NMSP) developed a 10-year strategic plan of operations for the organization. Specific goals and strategies were established to guide the progress of the Education and Outreach program. The Education and Outreach goal is: “To enhance nation-wide public awareness, understanding and appreciation of marine and Great Lakes ecosystems and maritime heritage resources through outreach, education and interpretation efforts”.

The specific performance measure, in place since 2010, for evaluating this goal is:

“… all education programs implemented in national marine sanctuaries will be assessed for effectiveness against stated program goals and objectives and appropriate National and State education standards.”

The NMSP education team has embarked on an ambitious evaluation project that will allow the NMSP to assess education program outcomes and impacts across all sites and activities and to link outcome measures to program efforts. The purpose of this effort is to evaluate if current and future education efforts are meeting the goals and objectives of the education and outreach programs and the educational mandates of the National Marine Sanctuaries Act. The continued application of these findings will assist in assessing current installed content and creating new content and information delivery methods.

Program to be evaluated

Mokupāpapa Discovery Center (MDC), an ONMS interpretive facility, located on the island of Hawai`i, is an outreach education center designed to interpret the natural and cultural history of the Papahānaumokuākea Marine National Monument (PMNM). The Center was designed to interpret the natural sciences, culture, and history of the Northwestern Hawaiian Islands (NWHI) and surrounding marine environment. The MDC exhibits were created to inform the public and garner support for protection for the remote area. The abundant natural life of the NWHI comes alive within MDC with replicas of sharks swimming overhead, a 3,500 gallon aquarium, and a surround screen virtual tour of Papahānaumokuākea.

After eleven years of operation, MDC has a consistent annual 70,000 visitors per year. Based on our location, the MDC has achieved a good balance between local residents and visitors to the island. MDC is an integral part of downtown Hilo, with frequent repeat visitors. MDC collaborates with public, private and charter school educators and services approximately 7,500 students annually.

Being a Marine National Monument and UNESCO World Heritage Site* has had a major impact on the outreach activities we are trying to get across to the public. MDC is examining what concepts we are conveying in our exhibits and programmatic materials. As MDC develops new content, we are taking into account not only NOAA’s Papahānaumokuākea Marine National Monument (PMNM) messages, but also applicable messages from our co-trustees, the US Fish and Wildlife Service, Office of Hawaiian Affairs, and the State of Hawai`i, as well as the NMSP’s and NOAA’s goals.

MDC is requesting to evaluate patron acuity to determine successful concept attainment. By conducting thorough evaluations, it will aid in vital decisions regarding exhibit renovation, new exhibits, interpretational programs, and educational content. A survey very similar to the one proposed here, and which formed the basis for this survey, was completed in January 2010 (OMB Control No 0648-0582, approved in January 2009) and provided valuable data on visitor demographics, and exhibit effectiveness. Our current facility is four blocks down the street from our previous location but nearly all the exhibits and layout are the same. We have modified several of the exhibits in the facility, and added programming better tailored to the audiences described by the original survey. This survey will allow us to determine the effectiveness of these exhibits and program changes/improvements, and will also allow us to determine any changes in our audience. While we have been approved to re-conduct this survey at two points in the last few years (2012 and 2015) we were unable to due to changes in the management, staffing changes, and moving locations. Since conducting the last survey we have gained World Heritage status, were expanded 5 times our previous size, and we would also like to determine what change these changes may have on our audience.

The survey conducted in 2010 was created through a contract with a professional evaluation company (People, Places and Designs Research, http://ppdresearch.com/). From this survey we

* World Heritage is the designation for places on Earth that are of outstanding universal value to humanity and as such, have been inscribed on the World Heritage List to be protected for future generations to appreciate and enjoy.  Once a country signs the Convention, and has sites inscribed on the World Heritage List, the resulting prestige often helps raise awareness among citizens and governments for heritage preservation. Greater awareness leads to a general rise in the level of the protection and conservation given to heritage properties. A country may also receive financial assistance and expert advice from the World Heritage Committee to support activities for the preservation of its sites (from http://whc.unesco.org/en/faq/#q2).

were able to better determine our audience, as well as which exhibits were being used most

frequently. In response to the feedback from the first survey, we have expanded our live aquaria exhibits, have increased interactive exhibit content, and also tailored some of our program offerings to the needs of resident visitors, whom we were able to determine from the survey, composed half of our visitors. This request is to run the survey again, probably towards the end of this year, to see how visitor experience, and composition, is changing over time, and if our new/revised exhibits, as well as our new, larger location, are effective. Note: changes to exhibits did not cause changes to the questions.

2. Explain how, by whom, how frequently, and for what purpose the information will be used. If the information collected will be disseminated to the public or used to support information that will be disseminated to the public, then explain how the collection complies with all applicable Information Quality Guidelines.

The information from this survey will be used to align future exhibit and educational programs developed at Mokupāpapa Discovery Center. Additionally, information will be used to improve NMSP’s and Papahānaumokuākea Marine National Monument (PMNM) messages to the 70,000 + patrons. The survey will be conducted every five years.

• Questions 1, 2 & 14 provide us with demographics.

• Question 3, 3a are a succinct account of the patrons experience at MDC.

• Questions 4 & 5 gauge elementary concepts.

• Question 6 determines if people value the exhibits.

• Question 7 determines which exhibits people would like to see.

• Question 8 identifies comprehension of monument status.

• Question 8a identifies comprehension of NOAAʻs National Marine Sanctuary System

• Question 8b identifies that they are in a NOAA Office of National Marine Sanctuaries facility

• Question 8.c identifies comprehension of UNESCO World Heritage status.

• Question 9 determines archipelago and place comprehension.

• Question 10 identifies exhibit comprehension effectiveness.

• Question 11 gauges patron environmental comprehension.

• Question 12 identifies patron learning style.

• Question 12a identifies patron learning behavior.

• Question 13 identifies patron learning inclination.

For this survey instance we have made changes to questions 8 a,b,c to determine if visitors know they are in a NOAA facility, and their familiarity with the National Marine Sanctuary System in addition to our World Heritage Designation. Mokupāpapa Discovery Center is funded and operated by NOAA and ONMS and we would like to determine if that is obvious to visitors through our exhibits and signage. We have removed the question regarding whether respondents think it is important that Papahānaumokuākea is a World Heritage Site since it does not provide useful information to us, nor would it drive any changes to exhibits or signage. We are a World Heritage Site and that is not likely to change.

NOAA ONMS will retain control over the information and safeguard access, modification, and destruction, consistent with NOAA standards for confidentiality, privacy, and electronic information.  See response to Question10 of this Supporting Statement for more information on confidentiality and privacy. Information collected is designed to yield data that meet all applicable information quality guidelines. Information gathered is not expected to be disseminated to the public. The assessments results may be used in scientific, management, technical or general information publications. Should NOAA Office of National Marine Sanctuaries decide to disseminate the information, it will be subject to the quality control measures and pre-dissemination review pursuant to Section 515 of Public Law 106-554.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological techniques or other forms of information technology.

The MDC patron assessments will consist of intercept interviews and self-paced assessments. Interviews will be conducted on paper, using a clipboard, for the reasons below:

● Patron aversion to surveys can be tempered via the humanistic learning theory of instructional design (by calling to their values and judgments, interviewers build patron trust)

● Patrons who wish to have their comments recorded who are uncomfortable with a more formal interview assessment process will have the option to do a self-paced assessment.

4. Describe efforts to identify duplication.

This is the second instance that MDC is requesting a patron experience assessment. While we have been approved to re-conduct this survey at two prior times since the initial survey, we were unable to due to major changes in the facility operation, staffing changes, and even changes to the area we manage and interpret, such as expansion in 2016 so we have delayed conducting the survey until now. No other organization has conducted, or plans to conduct, such an assessment for this visitor center.

5. If the collection of information involves small businesses or other small entities, describe the methods used to minimize burden.

This project will not impact small entities, businesses, organizations, or government bodies. All respondents will be individuals or families.

6. Describe the consequences to the Federal program or policy activities if the collection is not conducted or is conducted less frequently.

If this evaluation were not conducted, MDC would not be able to assess whether it is fulfilling NOAA’s mandate of having an informed society that comprehends the importance of the oceans, coasts, and atmosphere in the global ecosystem to make the best social and economic decisions. In addition, MDC will not be able to modify our exhibits and education programs effectively to fulfill NOAA’s, NOS’, ONMS’ and Papahānaumokuākea Marine National Monument’s education and outreach goals.

The feedback we received from the first run of this survey instrument was very useful to us in determining our visitor composition and demographics, as well as which exhibits were the most effective, and what content visitors were most interested in. We have since used the results from this first survey instrument to modify and improve our exhibits, and to better tailor our program offerings to the visitor base we are getting. Finally, the Office of National Marine Sanctuaries is required to evaluate our formal and informal education and outreach programs, and this is one important component of our evaluations.

7. Explain any special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines.

There are no special circumstances that deviate from OMB guidelines as listed in Attachment 1 of the instructions.

8. Provide information on the PRA Federal Register Notice that solicited public comments on the information collection prior to this submission. Summarize the public comments received in response to that notice and describe the actions taken by the agency in response to those comments. Describe the efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

A Federal Register Notice published on April 25, 2018 (83 FR 08633) solicited comments from the public. No comments were received. Comments were also solicited from non-NOAA stakeholders. Two comments were received.

1. I am happy to review the proposal for another visitor survey for Mokupapapa.  It seems to me that the proposal is a valid one since the message has changed. (commenter is referring to the changes in the size and designation of the Monument, and also changes in the co-management structure that has caused us to change our messaging to the pubilc, i.e. “we are a World Heritage Site,” “we are now the second largest protected area on Earth”, etc) Many visitors arrive knowing little about the Papahanaumokuakea and even less about the joint management by the three co-trustees.  Assessing how successful Mokupapapa is in delivering its message is important in evaluating current strategies and planning how to strengthen them to achieve the facilityʻs goals.

A verbal questionnaire seems the most effective way to elicit thoughtful comments from visitors while keeping the time commitment from staff reasonable.  I support the proposal.

Raquel Dow

Friends of Midway Atoll volunteer 

2. Caren Loebel-Fried

P.O. Box 1354

Volcano, HI 96785

808-985-9561

Re:  National Oceanic and Atmospheric Administration Proposed Information Collection; Comment Request; Papahanaumokuakea Marine National Monument Mokupapapa Discovery Center Exhibit Evaluation AGENCY: National Oceanic and Atmospheric Administration (NOAA), Commerce. ACTION: Notice.

I am writing in support of a visitor survey for Mokupāpapa Marine Discovery Center. I support the need to solicit public comments with this survey, and the proposed method for data collection.

Thank you,

Caren Loebel-Fried

9. Explain any decisions to provide payments or gifts to respondents, other than remuneration of contractors or grantees.

No payments, gifts or incentives will be offered.

10. Describe any assurance of confidentiality provided to respondents and the basis for assurance in statute, regulation, or agency policy.

All interviews will be anonymous; no information will be collected that would identify the specific individual (e.g., name, address, phone number, social security number, driver’s license number); therefore, no assurance of confidentiality will be required or provided. Demographic information will be used only for statistical analysis and aggregate information about the sample (e.g., age, gender, area of residence, visitor group size and composition).

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private.

No questions of a sensitive nature are being asked in this survey.

12. Provide an estimate in hours of the burden of the collection of information.

Respondent sample: This study will seek one interview each from a sample of 250 visitor groups (pre-existing parties who arrived together, including single adults visiting alone, couples, families, etc.), randomly selected after they have seen exhibits at MDC and are about to exit the building. One adult (age 18+) per visitor group will be approached and invited to give his/her opinion; participation will be voluntary. Prior experience with this type of work, and the first implementation of this survey, shows that the response rate is approximately 85-90%. [From the social scientist researcher who developed the original study, we have information on that response rate and the rates of cooperation at similar facilities (aquariums, museums). In general, the cooperation rate averages about 90%; the rate from about 20 recent projects has ranged from 72% to 98%.]

Data sought from:	# of respondents	Responses per respondent	Total # Responses	Response Time	Total Burden	Labor Cost to Public *
Visitors to Mokupapapa Discovery Center	278-294 visitors approached to obtain a sample of 250	1 interview	250	7.5 min avg. per interview	31 hrs.	$508

Based on the US Census data from 2014, the average household income is $53,891 ($16.20 per hour for adults in household). The average estimated time per respondent is 7.5 minutes (12.5% of an hour). Therefore, the average labor cost per adult answering the questions would be $2.03, multiplied by the 250 responders, with a total burden of $508.

13. Provide an estimate of the total annual recordkeeping/reporting cost burden to the respondents resulting from the collection (excluding the value of the burden hours in Question 12 above).

a. Capital and start-up costs: none.

b. Operations and maintenance costs for the public: none (an interviewer will ask a series of questions, and the interviewer will write visitors’ answers on the interview form; no follow-up or mailing or other expense will be required of the visitors).

14. Provide estimates of annualized cost to the Federal government.

We estimate 120 hours of work for the Mokupāpapa Discovery Center Manager in this capacity as a normal part of his job, and 8 hours of work for three other Mokupāpapa Staff members, also part of their normal job hours. Collection of data will be conducted by a combination of both staff and volunteers, and overseen by the Manager. With the estimate of 68 hours of datacollection time (based upon actual data collection time, and intervals between survey participants), we anticipate only 24 hours will be of staff time, with the other 44 hours being conducted by volunteers. Processing of data will be handled by MDC Manager.

Personnel	Time	Additional cost
Manager Time	120 hours @ $25 per hour = $3,000	Normal job responsibilities
Staff Time	24 hours @ $20 per hour = $480	Normal job responsibilities
Volunteer Time	44 hours	No cost

Total cost of manager and staff time: $3,480.

15. Explain the reasons for any program changes or adjustments.

There are no changes.

16. For collections whose results will be published, outline the plans for tabulation and publication.

The purpose of this evaluation is to measure content achievement and design improvement of education and outreach goals. To facilitate qualified uses (e.g., among other marine sanctuaries), a short summary of the analysis will be made available on the PMNM web site (www.papahanaumokuakea.gov/education/) education homepage explaining how to request a full copy from MDC. 

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons why display would be inappropriate.

We are not requesting an exception to displaying OMB documentation.

18. Explain each exception to the certification statement.

No exceptions.

9