Mokupapapa_SUPPORTING STATEMENT 082808 Part B

SUPPORTING STATEMENT
EVALUATION OF PUBLIC VISITORS’ EXPERIENCE OF EXHIBITS AT
MOKUPAPAPA DISCOVERY CENTER
OMB CONTROL NO. 0648-xxxx
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any
sampling or other respondent selection method to be used. Data on the number of entities
(e.g. establishments, State and local governmental units, households, or persons) in the
universe and the corresponding sample are to be provided in tabular form. The tabulation
must also include expected response rates for the collection as a whole. If the collection has
been conducted before, provide the actual response rate achieved.
Mokupapapa Discovery Center
(MDC)
Annual total visitor attendance (avg.)
at Mokupapapa Discovery Center
(MDC)
Annual attendance by GENERAL
PUBLIC visitors at MDC (excludes
school groups and professional
visitors)
Estimated number of adult visitors
(age 18+) in the MDC general public
visitor audience

60,000 persons

55,000 persons

40,000 adults

Desired sample size of general public
adult visitors in the MDC audience

278-294 visitors will be
approached to obtain a sample of
250 adults

Respondent selection method

One adult per randomly selected
visitor group, when exiting from
the exhibit area of MDC

Estimated rate of cooperation of
85% [x 294 or fewer visitors for a
randomly selected adult visitors
final sample of 250]
Note: In the experience of the social scientist researcher who will direct this
study, the actual rate of cooperation at similar facilities (aquariums, museums)
averages about 90%; the rate from about 20 projects in the last two years has
ranged from 72% to 98%.

2. Describe the procedures for the collection, including: the statistical methodology for
stratification and sample selection; the estimation procedure; the degree of accuracy
needed for the purpose described in the justification; any unusual problems requiring
specialized sampling procedures; and any use of periodic (less frequent than annual) data
collection cycles to reduce burden.
The characteristics of visitor populations at visitor centers and museums vary considerably and
randomly (e.g., a local family may be followed by a tourist couple who may be followed by a
single adult tourist, and so on). In places with relatively low volumes of visitors (such as the
Mokupapapa Discovery Center, compared to high volume places such as the Smithsonian) a
representative random sample of visitor groups can be obtained by using a “next available”
protocol, as follows:
The interviewer is positioned near the exit from the exhibit space (the ‘space’ is
essentially a storefront, with exhibits and an auditorium located behind the exhibit space
in the rear of the building). As any visitor group (usually 1-4 people) nears the exit, the
interviewer approaches and makes eye contact with the ‘first adult’ (in practice: the one
who is physically closest to the interviewer) and requests their participation in giving
feedback about the exhibits. The cooperation rate for this type of intercept interview
(using a brief introduction that explains the purpose in one sentence) typically averages
about 90%. If the adult visitor agrees, the interview is completed. Upon completion, the
interviewer will tend to step aside to complete their work on the interview form
(documenting the date and time of the interview, adding their own initials to it, reviewing
the form to check for completeness and readable handwriting, and also to put away that
completed interview form and have a new blank one ready); this process usually takes 25 minutes. When the interviewer is then prepared with a new blank interview form and
related materials (e.g., a photo board about the exhibits, used for some of the interview
questions), s/he looks up and selects the “next available” visitor group who are moving to
the exit.
The principle of this and other sampling methods is that the interviewer does not choose who to
interview by appearance, or by facial expression that might indicate enjoyment or not, or by
whether there are or are not children in the group; in essence, the visitor group selects themselves
(although they don’t know the sampling parameters) by deciding when to exit (e.g., there may be
another group being interviewed at the time when this group leaves, in which case they would
not be selected). Depending on the visitor flow, the next visitor group might be leaving right
then, or the interviewer might have to wait for 5-10 minutes for the next group to leave. This
characteristic of ‘low volume’ visitor facilities makes it impractical to use other methods such as
selecting every 4th visitor group, or using a random number chart (for example, from 1 to 5) to
decide which visitor group to select. Ultimately, however, it is not the efficiency of interviewer
time that is critical, it is the representativeness of the sample that matters. While additional
methods could be used to provide reliability assessments of the sampling method, 1 the budget is
1

Periodically over the course of conducting hundreds of studies like this (intercept interviews with visitors at museums,
aquariums, visitor centers, historic sites) this social scientist research contractor has used reliability studies to check on
the characteristics of the sampling. The most common way of assessing reliability has been to use a different method
such as ‘entrance interviews’ or admissions information collected by a cashier (i.e., a process which has 100% or near-

modest in this particular project, and we are choosing to put relatively more effort in the analysis
of open-ended questions than in conducting a rigorous reliability study, trusting that a wellconducted random sampling of “next available” visitor groups will result in a sufficiently
representative sample.
3. Describe the methods used to maximize response rates and to deal with nonresponse. The
accuracy and reliability of the information collected must be shown to be adequate for the
intended uses. For collections based on sampling, a special justification must be provided if
they will not yield "reliable" data that can be generalized to the universe studied.
Prior experience with intercept interviewing of visitors in museums and interpretive centers leads
our social science researcher to expect a response rate of 85%-90%. Therefore, non-response is
not likely to be a big factor in this study. Typically, experience has shown that inviting visitors
to give their opinions is a positive motivator, and that the way in which that invitation is
delivered can enhance or detract from visitors’ desire to cooperate (e.g., neat appearance of the
interviewer, a clear voice, pleasant demeanor, and in a small proportion of interactions: giving
assurances that the interview won’t take too long, or that it won’t be too hard).
When the survey instrument and procedures are approved for implementation, we will begin
monitoring the cooperation rate. If it is below 75% we will experiment with fine-tuning of the
logistics of the survey (where the interviewer stands, which sentence of the explanation comes
first) to seek improvements in the cooperation rate. In the very unlikely event that we encounter
an ongoing response rate below 75%, we will consider developing a ‘mail-back’ questionnaire
that could be handed to non-respondents, or a web-link survey, to be completed at their
convenience later (however, immediate cooperation for an on-site intercept interview is about
twice as likely as cooperation for follow-up questionnaires, so we would not use mail-back or
web-link as a primary method).
Due to routinely successful cooperation rates, we are consciously choosing to not give visitors
advance notification (when they enter the Center) that we will be seeking their opinions and
feedback, since this tends to cue people in ways that sometimes leads to changing their behavior
and use of exhibits (e.g., staying longer, feeling that they will be “tested” later), and we are
seeking to generalize to the normally occurring pattern of visitor experience. Of course, some
visitors will see that other visitors are being interviewed at the exit, but without knowing why
those people are being interviewed, this circumstance is unlikely to affect visitor behavior.

100% cooperation) to obtain a few pieces of information from a different sample of visitors, or perhaps from all visitors
(depending on the volume of attendance). Such information would typically include group composition [adults-only vs.
families-with-children], place of residence [local vs. out-of-area, or specific states], and prior experience [first-time-visitor
vs. repeat-visitor]). However, in almost all such examples, the ‘next available’ method of sampling visitor groups in exit
interviews has produced a sample that varies by only a few percentage points from the reliability study, a degree of
accuracy which would be sufficient for this research (the notable variations have occurred where a museum has multiple
exits, or a large lobby where it is difficult to identify exiting vs. entering visitors, etc.).

4. Describe any tests of procedures or methods to be undertaken. Tests are encouraged as
effective means to refine collections, but if ten or more test respondents are involved OMB
must give prior approval under the Paperwork Reduction Act.
A pilot survey of 8 visitors has been conducted, which has intuitively confirmed the essential
prerequisites for this survey, namely that visitors did not need an incentive to participate,
appeared to understand the questions, were able to give reasonable answers to those questions,
and each completed the interview (did not discontinue part way through). If, after the survey is
approved and the formal data collection begins, circumstances seem to change, ideas about
experiments and strategies for maintaining the scientific quality of the research will be
considered (e.g., as described in answer to the previous question: location of the interviewer, the
use of optional mail-back questionnaires).
5. Provide the name and telephone number of individuals consulted on the statistical
aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other
person(s) who will actually collect and/or analyze the information for the agency.
The social scientist researcher who developed the research design, and composed the survey
instrument, is:
Jeff Hayward, Ph.D., Director of People, Places & Design Research (www.ppdresearch.com);
413-586-9003.
He will also supervise the beginning of the implementation of the survey, including training
(about random selection, techniques for conducting intercept interviews and maintaining rapport
with visitors) for interviewers, and will coach and support the MDC staff coordinator Yumi
Yasutake (regarding the monitoring of the quality of interviewers’ work) who will organize and
manage the data collection process.
Dr. Hayward (with 30 years of work in the field of visitor studies) and his experienced staff (with
8 to 15 years of experience in this specialty) will analyze and interpret the data.
Linda Schubert, Manager of Mokupapapa Discovery Center, will be NOAA’s principal
representative in interpreting the data and articulating the possible implications for exhibits,
programs, and related ways of educating the public about the NWHI National Monument.