0582_SUPPORTING_STATEMENT_012512_Part_A_v2

SUPPORTING STATEMENT

EVALUATION OF PUBLIC VISITORS’ EXPERIENCE OF EXHIBITS AT

MOKUPAPAPA DISCOVERY CENTER

OMB CONTROL NO. 0648-0582

A. JUSTIFICATION

1. Explain the circumstances that make the collection of information necessary.

The enabling legislation for the National Marine Sanctuary system, the National Marine Sanctuaries Act (NMSA), denotes specific educational mandates. Section 309(c)(1) of the NMSA states that one of the purposes of the Office of National Marine Sanctuaries (ONMS) is:

“ . . .to enhance public awareness, understanding, appreciation and wise and sustainable use of the marine environment, and the natural historical, cultural and archeological resources of the national Marine Sanctuary System. Efforts supported, promoted, or coordinated under this subsection must emphasize the conservation goals and sustainable public uses of national marine sanctuaries and the System.”

In 2005, the planning committee of the National Marine Sanctuary Program (NMSP) developed a 10-year strategic plan of operations for the organization. Specific goals and strategies were established to guide the progress of the Education and Outreach program. The Education and Outreach goal is:

“To enhance nation-wide public awareness, understanding and appreciation of marine and Great Lakes ecosystems and maritime heritage resources through outreach, education and interpretation efforts”

The specific performance measure, in place since 2010, for evaluating this goal is:

“By 2010 all education programs implemented in national marine sanctuaries will be assessed for effectiveness against stated program goals and objectives and appropriate National and State education standards.”

The NMSP education team has embarked on an ambitious evaluation project that will allow the NMSP to assess education program outcomes and impacts across all sites and activities and to link outcome measures to program efforts. The purpose of this effort is to evaluate if current and future education efforts are meeting the goals and objectives of the education and outreach programs and the educational mandates of the National Marine Sanctuaries Act. The application of these findings will assist in assessing current installed content and creating new content and information delivery methods.

Program to be evaluated

Mokupāpapa Discovery Center (MDC), an ONMS interpretive facility, located on the island of Hawai`i, is an outreach education center designed to interpret the natural and cultural history of the Papahānaumokuākea Marine National Monument (PMNM). The Center was designed to interpret the natural sciences, culture, and history of the Northwestern Hawaiian Islands (NWHI) and surrounding marine environment. The MDC exhibits were created to inform the public and garner support for protection for the remote area. The abundant natural life of the NWHI comes alive within MDC with replicas of sharks swimming overhead, a 2,500 gallon aquarium, and a mockup submersible.

After eight years of operation, MDC has a consistent annual 60,000 visitors per year. Based on our location, the MDC has achieved a good balance between local residents and visitors to the island. MDC is an integral part of downtown Hilo, with frequent repeat visitors. MDC collaborates with public, private and charter school educators and services approximately 4,200 students annually.

Being a Marine National Monument and UNESCO World Heritage Site* has had a major impact on the outreach activities we are trying to get across to the public. MDC is examining what concepts we are conveying in our exhibits and programmatic materials. As MDC develops new content, we are taking into account not only NOAA’s Papahānaumokuākea Marine National Monument (PMNM) messages, but also applicable messages from our co-trustees, the US Fish and Wildlife Service and the State of Hawai`i, as well as the NMSP’s and NOAA’s goals.

MDC is requesting to evaluate patron acuity to determine successful concept attainment. By conducting thorough evaluations it will aid in vital decisions regarding exhibit renovation, new exhibits, interpretational programs, and educational content. A survey very similar to the one proposed here, and which formed the basis for this survey, was completed in January 2010 (OMB Control No 0648-0582, approved in January 2009) and provided valuable data on visitor demographics, and exhibit effectiveness. We have since modified several of the exhibits in the facility, and added programming better tailored to the audiences described by the original survey. This survey will allow us to determine the effectiveness of these exhibit and program changes/improvements, and will also allow us to determine any changes in our audience. Since conducting the last survey we have gained World Heritage status, and we would also like to determine what change this may have on our audience.

The survey conducted in 2010 was created through a contract with a professional evaluation company (People, Places and Designs Research, http://ppdresearch.com/). From this survey we

* World Heritage is the designation for places on Earth that are of outstanding universal value to humanity and as such, have been inscribed on the World Heritage List to be protected for future generations to appreciate and enjoy.  Once a country signs the Convention, and has sites inscribed on the World Heritage List, the resulting prestige often helps raise awareness among citizens and governments for heritage preservation. Greater awareness leads to a general rise in the level of the protection and conservation given to heritage properties. A country may also receive financial assistance and expert advice from the World Heritage Committee to support activities for the preservation of its sites (from http://whc.unesco.org/en/faq/#q2).

were able to better determine our audience, as well as which exhibits were being used most

frequently. A survey report was created, and we have submitted it with these responses. In

response to the feedback from the first survey we have expanded our live aquaria exhibits, have improved and updated the deep sea research area, increased interactive exhibit content, and also tailored some of our program offerings to the needs of resident visitors, whom we were able to determine from the survey, composed half of our visitors. This request is to run the survey again, probably next year, to see how visitor experience, and composition, is changing over time, and if our new/revised exhibits are effective. We modified the original survey instrument only slightly.

2. Explain how, by whom, how frequently, and for what purpose the information will be used. If the information collected will be disseminated to the public or used to support information that will be disseminated to the public, then explain how the collection complies with all applicable Information Quality Guidelines.

The information from this new survey will be used to align future exhibit and educational programs developed at Mokupāpapa Discovery Center. Additionally, information will be used to improve NMSP’s and Papahānaumokuākea Marine National Monument (PMNM) messages to the 60,000 + patrons. The survey will only be conducted once.

• Questions 1, 2 & 14 provide us with demographics.

• Question 3, 3a are a succinct account of the patrons experience at MDC.

• Questions 4 & 5 gauge elementary concepts.

• Question 6 determines if people value the exhibits.

• Question 7 determines which exhibits people would like to see.

• Question 8 identifies comprehension of monument status.

• Question 8a & 8b identifies comprehension of UNESCO World Heritage status and value.

• Question 9 determines archipelago and place comprehension.

• Question 10 identifies exhibit comprehension effectiveness.

• Question 11 gauges patron environmental comprehension.

• Question 12 identifies patron learning style.

• Question 12a identifies patron learning behavior.

• Question 13 identifies patron learning inclination.

NOAA ONMS will retain control over the information and safeguard access, modification, and destruction, consistent with NOAA standards for confidentiality, privacy, and electronic information.  See response to Question10 of this Supporting Statement for more information on confidentiality and privacy. Information collected is designed to yield data that meet all applicable information quality guidelines. Information gathered is not expected to be disseminated to the public. The assessments results may be used in scientific, management, technical or general information publications. Should NOAA Office of National Marine Sanctuaries decide to disseminate the information, it will be subject to the quality control measures and pre-dissemination review pursuant to Section 515 of Public Law 106-554.

3. Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological techniques or other forms of information technology.

The MDC patron assessments will consist of intercept interviews and self paced assessments. Interviews will be conducted on paper, using a clipboard, for the reasons below:

● Patron aversion to surveys can be tempered via the humanistic learning theory of instructional design (by calling to their values and judgments interviewers build patron trust)

● Patrons that wish to have their comments recorded who are uncomfortable with a more formal interview assessment process will have the option to do a self-paced assessment.

4. Describe efforts to identify duplication.

This is the second instance that MDC is requesting a patron experience assessment. No other organization has conducted, or plans to conduct, such an assessment for this visitor center.

5. If the collection of information involves small businesses or other small entities, describe the methods used to minimize burden.

This project will not impact small entities, businesses, organizations, or government bodies. All respondents will be individuals or families.

6. Describe the consequences to the Federal program or policy activities if the collection is not conducted or is conducted less frequently.

If this evaluation were not conducted, MDC would not be able to assess whether it is fulfilling NOAA’s mandate of having an informed society that comprehends the importance of the oceans, coasts, and atmosphere in the global ecosystem to make the best social and economic decisions. In addition, MDC will not be able to modify our exhibits and education programs effectively to fulfill NOAA’s, NOS’, ONMS’ and Papahānaumokuākea Marine National Monument’s education and outreach goals.

The feedback we received from the first run of this survey instrument was very useful to us in determining our visitor composition and demographics, as well as which exhibits were the most effective, and what content visitors were most interested in. We have since used the results from this first survey instrument to modify and improve our exhibits, and to better tailor our program offerings to the visitor base we are getting. Finally, the Office of National Marine Sanctuaries is required, as part of our PART Performance Measures to evaluate our formal and informal education and outreach programs, and this is one important component of our evaluations.

7. Explain any special circumstances that require the collection to be conducted in a manner inconsistent with OMB guidelines.

There are no special circumstances that deviate from OMB guidelines as listed in Attachment 1 of the instructions.

8. Provide information on the PRA Federal Register Notice that solicited public comments on the information collection prior to this submission. Summarize the public comments received in response to that notice and describe the actions taken by the agency in response to those comments. Describe the efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting format (if any), and on the data elements to be recorded, disclosed, or reported.

A Federal Register Notice published on November 8, 2011 (76 FR 69241) solicited comments from the public. No comments were received.

9. Explain any decisions to provide payments or gifts to respondents, other than remuneration of contractors or grantees.

No payments, gifts or incentives will be offered.

10. Describe any assurance of confidentiality provided to respondents and the basis for assurance in statute, regulation, or agency policy.

All persons interviewed will be anonymous; no information will be collected that would identify the specific individual (e.g., name, address, phone number, social security number, driver’s license number); therefore, no assurance of confidentiality will be required or provided. Demographic information will only be used for statistical analysis and aggregate information about the sample (e.g., age, gender, area of residence, visitor group size and composition).

11. Provide additional justification for any questions of a sensitive nature, such as sexual behavior and attitudes, religious beliefs, and other matters that are commonly considered private.

No questions of a sensitive nature are being asked in this survey.

12. Provide an estimate in hours of the burden of the collection of information.

Respondent sample: This one-time study will seek one interview each from a sample of 250 visitor groups (pre-existing parties who arrived together, including single adults visiting alone, couples, families, etc.), randomly selected after they have seen exhibits at MDC and are about to exit the building. One adult (age 18+) per visitor group will be approached and invited to give his/her opinion; participation will be voluntary. Prior experience with this type of work suggests that the response rate will be approximately 85-90%. [From the social scientist researcher who developed the original study, we have information on that response rate and the rates of cooperation at similar facilities (aquariums, museums). In general, the cooperation rate averages about 90%; the rate from about 20 recent projects has ranged from 72% to 98%.]

Data sought from:	# of respondents	Responses per respondent	Total # Responses	Response Time	Total Burden	Labor Cost to Public *
Visitors to Mokupapapa Discovery Center	278-294 visitors approached to obtain a sample of 250	1 interview	250	7.5 min avg. per interview	31 hrs.	$465

Based on the US Census data from 2010 the average household income is $49,445 ($14.86 per hour for adults in household). The average estimated time per respondent is 7.5 minutes (12.5% of an hour). Therefore, the average labor cost per adult answering the questions would be $1.86, multiplied by the 250 responders, with a total burden of $465.

13. Provide an estimate of the total annual recordkeeping/reporting cost burden to the respondents resulting from the collection (excluding the value of the burden hours in Question 12 above).

a. Capital and start-up costs: none.

b. Operations and maintenance costs for the public: none (an interviewer will ask a series of questions, and the interviewer will write visitors’ answers on the interview form; no follow-up or mailing or other expense will be required of the visitors).

14. Provide estimates of annualized cost to the Federal government.

We estimate 120 hours of work for the Mokupāpapa Discovery Center Manager in this capacity as a normal part of his job, and 8 hours of work for three other Mokupāpapa Staff members, also part of their normal job hours. Collection of data will be conducted by a combination of both staff and volunteers, and overseen by the Manager. With the estimate of 68 hours of data collection time (based upon actual data collection time, and intervals between survey participants), we anticipate only 24 hours will be of staff time, with the other 44 hours being conducted by volunteers. Processing of data will be handled by MDC Manager.

Personnel	Time	Additional cost
Manager Time	120 hours @ $25 per hour	Normal job responsibilities
Staff Time	24 hours @ $20 per hour	Normal job responsibilities
Volunteer Time	44 hours	No cost

15. Explain the reasons for any program changes or adjustments.

There are no changes.

16. For collections whose results will be published, outline the plans for tabulation and publication.

The purpose of this evaluation is to measure content achievement and design improvement of education and outreach goals. To facilitate qualified uses (e.g., among other marine sanctuaries), a short summary of the analysis will be made available on the PMNM web site (www.papahanaumokuakea.gov/education/) education homepage explaining how to request a full copy from MDC. 

17. If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons why display would be inappropriate.

We are not requesting an exception to displaying OMB documentation.

18. Explain each exception to the certification statement.

No exceptions.

7