0582 ss 052015 Part A

SUPPORTING STATEMENT

EVALUATION OF PUBLIC VISITORS’ EXPERIENCE OF EXHIBITS AT

MOKUPAPAPA DISCOVERY CENTER

OMB CONTROL NO. 0648-0582

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary.

This request is for extension of a currently approved information collection.

The enabling legislation for the National Marine Sanctuary system, the National Marine Sanctuaries Act (NMSA), denotes specific educational mandates. Section 309(c)(1) of the NMSA states that one of the purposes of the Office of National Marine Sanctuaries (ONMS) is:

“ . . .to enhance public awareness, understanding, appreciation and wise and sustainable use of the marine environment, and the natural historical, cultural and archeological resources of the national Marine Sanctuary System. Efforts supported, promoted, or coordinated under this subsection must emphasize the conservation goals and sustainable public uses of national marine sanctuaries and the System.”

In 2005, the planning committee of the National Marine Sanctuary Program (NMSP) developed a 10-year strategic plan of operations for the organization. Specific goals and strategies were established to guide the progress of the Education and Outreach program. The Education and Outreach goal is: “To enhance nation-wide public awareness, understanding and appreciation of marine and Great Lakes ecosystems and maritime heritage resources through outreach, education and interpretation efforts”.

The specific performance measure, in place since 2010, for evaluating this goal is:

“… all education programs implemented in national marine sanctuaries will be assessed for effectiveness against stated program goals and objectives and appropriate National and State education standards.”

The NMSP education team has embarked on an ambitious evaluation project that will allow the NMSP to assess education program outcomes and impacts across all sites and activities and to link outcome measures to program efforts. The purpose of this effort is to evaluate if current and future education efforts are meeting the goals and objectives of the education and outreach programs and the educational mandates of the National Marine Sanctuaries Act. The continued application of these findings will assist in assessing current installed content and creating new content and information delivery methods.

Program to be evaluated

Mokupāpapa Discovery Center (MDC), an ONMS interpretive facility, located on the island of Hawai`i, is an outreach education center designed to interpret the natural and cultural history of the Papahānaumokuākea Marine National Monument (PMNM). The Center was designed to interpret the natural sciences, culture, and history of the Northwestern Hawaiian Islands (NWHI) and surrounding marine environment. The MDC exhibits were created to inform the public and garner support for protection for the remote area. The abundant natural life of the NWHI comes alive within MDC with replicas of sharks swimming overhead, a 3,500 gallon aquarium, and a mockup submersible.

After eleven years of operation, MDC has a consistent annual 60,000 visitors per year. Based on our location, the MDC has achieved a good balance between local residents and visitors to the island. MDC is an integral part of downtown Hilo, with frequent repeat visitors. MDC collaborates with public, private and charter school educators and services approximately 4,200 students annually.

Being a Marine National Monument and UNESCO World Heritage Site* has had a major impact on the outreach activities we are trying to get across to the public. MDC is examining what concepts we are conveying in our exhibits and programmatic materials. As MDC develops new content, we are taking into account not only NOAA’s Papahānaumokuākea Marine National Monument (PMNM) messages, but also applicable messages from our co-trustees, the US Fish and Wildlife Service and the State of Hawai`i, as well as the NMSP’s and NOAA’s goals.

MDC is requesting to evaluate patron acuity to determine successful concept attainment. By conducting thorough evaluations it will aid in vital decisions regarding exhibit renovation, new exhibits, interpretational programs, and educational content. A survey very similar to the one proposed here, and which formed the basis for this survey, was completed in January 2010 (OMB Control No 0648-0582, approved in January 2009) and provided valuable data on visitor demographics, and exhibit effectiveness. We had intended with the last (2012) extension request, to conduct the survey again in 2013 but due to relocation of the facility in 2013-2014, and re-opening in March of 2014, we were unable to do this. Our new facility is four blocks down the street from our previous location but nearly all the exhibits and layout are the same. We have modified several of the exhibits in the facility, and added programming better tailored to the audiences described by the original survey. This survey will allow us to determine the effectiveness of these exhibits and program changes/improvements, and will also allow us to determine any changes in our audience. Since conducting the last survey we have gained World Heritage status, and we would also like to determine what change this may have on our audience.

The survey conducted in 2010 was created through a contract with a professional evaluation company (People, Places and Designs Research, http://ppdresearch.com/). From this survey we

* World Heritage is the designation for places on Earth that are of outstanding universal value to humanity and as such, have been inscribed on the World Heritage List to be protected for future generations to appreciate and enjoy.  Once a country signs the Convention, and has sites inscribed on the World Heritage List, the resulting prestige often helps raise awareness among citizens and governments for heritage preservation. Greater awareness leads to a general rise in the level of the protection and conservation given to heritage properties. A country may also receive financial assistance and expert advice from the World Heritage Committee to support activities for the preservation of its sites (from http://whc.unesco.org/en/faq/#q2).

were able to better determine our audience, as well as which exhibits were being used most

frequently. A survey report was created, and we have submitted it with these responses. In

response to the feedback from the first survey we have expanded our live aquaria exhibits, have increased interactive exhibit content, and also tailored some of our program offerings to the needs of resident visitors, whom we were able to determine from the survey, composed half of our visitors. This request is to run the survey again, probably next year, to see how visitor experience, and composition, is changing over time, and if our new/revised exhibits, as well as our new, larger location, are effective. Note: changes to exhibits did not cause changes to the questions.

2. Explain how, by whom, how frequently, and for what purpose the information will be used. If the information collected will be disseminated to the public or used to support information that will be disseminated to the public, then explain how the collection complies with all applicable Information Quality Guidelines.

The information from this new survey will be used to align future exhibit and educational programs developed at Mokupāpapa Discovery Center. Additionally, information will be used to improve NMSP’s and Papahānaumokuākea Marine National Monument (PMNM) messages to the 60,000 + patrons. The survey will be conducted every five years.

• Questions 1, 2 & 14 provide us with demographics.

• Question 3, 3a are a succinct account of the patrons experience at MDC.

• Questions 4 & 5 gauge elementary concepts.

• Question 6 determines if people value the exhibits.

• Question 7 determines which exhibits people would like to see.

• Question 8 identifies comprehension of monument status.

• Question 8a & 8b identifies comprehension of UNESCO World Heritage status and value.

• Question 9 determines archipelago and place comprehension.

• Question 10 identifies exhibit comprehension effectiveness.

• Question 11 gauges patron environmental comprehension.

• Question 12 identifies patron learning style.

• Question 12a identifies patron learning behavior.

• Question 13 identifies patron learning inclination.

NOAA ONMS will retain control over the information and safeguard access, modification, and destruction, consistent with NOAA standards for confidentiality, privacy, and electronic information.  See response to Question10 of this Supporting Statement for more information on confidentiality and privacy. Information collected is designed to yield data that meet all applicable information quality guidelines. Information gathered is not expected to be disseminated to the public. The assessments results may be used in scientific, management, technical or general information publications. Should NOAA Office of National Marine Sanctuaries decide to disseminate the information, it will be subject to the quality control measures and pre-dissemination review pursuant to Section 515 of Public Law 106-554.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological techniques or other forms of information technology.

The MDC patron assessments will consist of intercept interviews and self paced assessments. Interviews will be conducted on paper, using a clipboard, for the reasons below:

● Patron aversion to surveys can be tempered via the humanistic learning theory of instructional design (by calling to their values and judgments, interviewers build patron trust)

● Patrons who wish to have their comments recorded who are uncomfortable with a more formal interview assessment process will have the option to do a self-paced assessment.

4. Describe efforts to identify duplication.

This is the second instance that MDC is requesting a patron experience assessment. No other organization has conducted, or plans to conduct, such an assessment for this visitor center.

5. If the collection of information involves small businesses or other small entities, describe the methods used to minimize burden.

This project will not impact small entities, businesses, organizations, or government bodies. All respondents will be individuals or families.

6. Describe the consequences to the Federal program or policy activities if the collection is not conducted or is conducted less frequently.

If this evaluation were not conducted, MDC would not be able to assess whether it is fulfilling NOAA’s mandate of having an informed society that comprehends the importance of the oceans, coasts, and atmosphere in the global ecosystem to make the best social and economic decisions. In addition, MDC will not be able to modify our exhibits and education programs effectively to fulfill NOAA’s, NOS’, ONMS’ and Papahānaumokuākea Marine National Monument’s education and outreach goals.

The feedback we received from the first run of this survey instrument was very useful to us in determining our visitor composition and demographics, as well as which exhibits were the most effective, and what content visitors were most interested in. We have since used the results from this first survey instrument to modify and improve our exhibits, and to better tailor our program offerings to the visitor base we are getting. Finally, the Office of National Marine Sanctuaries is required to evaluate our formal and informal education and outreach programs, and this is one important component of our evaluations.

7. Explain any special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines.

There are no special circumstances that deviate from OMB guidelines as listed in Attachment 1 of the instructions.

8. Provide information on the PRA Federal Register Notice that solicited public comments on the information collection prior to this submission. Summarize the public comments received in response to that notice and describe the actions taken by the agency in response to those comments. Describe the efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

A Federal Register Notice published on March 31, 2015 (80 FR 07290) solicited comments from the public. No comments were received. Comments were also solicited from non-NOAA stakeholders. Three comments were received.

1. I read the 2010 Visitor Perceptions of Mokupapapa Discovery Center report and reviewed the survey on which it is based. I find the results useful for assessing the facility and broader education facility planning for the monument. The survey is relatively short and collection of survey results does not appear to be overly burdensome to the public or the staff. It appears that the survey can easily be completed in less than 10 minutes.

Kem Lowry

Citizen Representative,

Northwestern Hawaiian Islands Coral Reef Ecosystem Reserve (NWHICRER) Advisory Council

2. I have reviewed the Mokupapapa Visitor Survey, the Supporting Statement and the Visitors' Perceptions report and feel that the results from the 2010 survey have been very useful for education planning for Papahānaumokuākea.

In addition, the report generated from the first survey was very comprehensive and informative, and the Supporting Statement clearly demonstrates that the estimate of burden upon survey participants of 7.5 minutes per survey is correct. I fully support the administration of a second exit survey.

Gail Grabowsky

NWHI Coral Reef Ecosystem Reserve Advisory Council Member

Education Representative

3. Thank you for sharing the Mokupapapa survey with me. It is impressively comprehensive.

I support the use of this survey and the information is valuable. I also feel that the estimate of burden on the public of 7.5 minutes per survey is accurate.

Laura Thompson

Conservation Member

Northwestern Hawaiian Islands Ecosystem Reserve Council.

9. Explain any decisions to provide payments or gifts to respondents, other than remuneration of contractors or grantees.

No payments, gifts or incentives will be offered.

10. Describe any assurance of confidentiality provided to respondents and the basis for assurance in statute, regulation, or agency policy.

All persons interviewed will be anonymous; no information will be collected that would identify the specific individual (e.g., name, address, phone number, social security number, driver’s license number); therefore, no assurance of confidentiality will be required or provided. Demographic information will be used only for statistical analysis and aggregate information about the sample (e.g., age, gender, area of residence, visitor group size and composition).

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private.

No questions of a sensitive nature are being asked in this survey.

12. Provide an estimate in hours of the burden of the collection of information.

Respondent sample: This study will seek one interview each from a sample of 250 visitor groups (pre-existing parties who arrived together, including single adults visiting alone, couples, families, etc.), randomly selected after they have seen exhibits at MDC and are about to exit the building. One adult (age 18+) per visitor group will be approached and invited to give his/her opinion; participation will be voluntary. Prior experience with this type of work, and the first implementation of this survey, shows that the response rate is approximately 85-90%. [From the social scientist researcher who developed the original study, we have information on that response rate and the rates of cooperation at similar facilities (aquariums, museums). In general, the cooperation rate averages about 90%; the rate from about 20 recent projects has ranged from 72% to 98%.]

Data sought from:	# of respondents	Responses per respondent	Total # Responses	Response Time	Total Burden	Labor Cost to Public *
Visitors to Mokupapapa Discovery Center	278-294 visitors approached to obtain a sample of 250	1 interview	250	7.5 min avg. per interview	31 hrs.	$508

Based on the US Census data from 2014, the average household income is $53,891 ($16.20 per hour for adults in household). The average estimated time per respondent is 7.5 minutes (12.5% of an hour). Therefore, the average labor cost per adult answering the questions would be $2.03, multiplied by the 250 responders, with a total burden of $508.

13. Provide an estimate of the total annual recordkeeping/reporting cost burden to the respondents resulting from the collection (excluding the value of the burden hours in Question 12 above).

a. Capital and start-up costs: none.

b. Operations and maintenance costs for the public: none (an interviewer will ask a series of questions, and the interviewer will write visitors’ answers on the interview form; no follow-up or mailing or other expense will be required of the visitors).

14. Provide estimates of annualized cost to the Federal government.

We estimate 120 hours of work for the Mokupāpapa Discovery Center Manager in this capacity as a normal part of his job, and 8 hours of work for three other Mokupāpapa Staff members, also part of their normal job hours. Collection of data will be conducted by a combination of both staff and volunteers, and overseen by the Manager. With the estimate of 68 hours of datacollection time (based upon actual data collection time, and intervals between survey participants), we anticipate only 24 hours will be of staff time, with the other 44 hours being conducted by volunteers. Processing of data will be handled by MDC Manager.

Personnel	Time	Additional cost
Manager Time	120 hours @ $25 per hour = $3,000	Normal job responsibilities
Staff Time	24 hours @ $20 per hour = $480	Normal job responsibilities
Volunteer Time	44 hours	No cost

Total cost of manager and staff time: $3,480.

15. Explain the reasons for any program changes or adjustments.

There are no changes.

16. For collections whose results will be published, outline the plans for tabulation and publication.

The purpose of this evaluation is to measure content achievement and design improvement of education and outreach goals. To facilitate qualified uses (e.g., among other marine sanctuaries), a short summary of the analysis will be made available on the PMNM web site (www.papahanaumokuakea.gov/education/) education homepage explaining how to request a full copy from MDC. 

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons why display would be inappropriate.

We are not requesting an exception to displaying OMB documentation.

18. Explain each exception to the certification statement.

No exceptions.

9