0582 ss 052015 Part B

SUPPORTING STATEMENT

EVALUATION OF PUBLIC VISITORS’ EXPERIENCE OF EXHIBITS AT

MOKUPAPAPA DISCOVERY CENTER

OMB CONTROL NO. 0648-0582

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g. establishments, State and local governmental units, households, or persons) in the universe and the corresponding sample are to be provided in tabular form. The tabulation must also include expected response rates for the collection as a whole. If the collection has been conducted before, provide the actual response rate achieved.

Conducting an exit survey, as was done in 2010, appears to us to be the most effective means of capturing responses: impressions are captured immediately after the Discovery Center experience, and the level of detail in our questions, as stated in Part A, enabled us to make several improvements after that first survey, as described in the report we have included in this response. Comments we have received through other venues, including our Facebook site, have been extremely general, such as “nice facility”.

	Mokupapapa Discovery Center (MDC)
Annual total visitor attendance (avg.) at Mokupāpapa Discovery Center (MDC)	60,000 persons
Annual attendance by GENERAL PUBLIC visitors at MDC (excludes school groups and professional visitors)	55,000 persons
Estimated number of adult visitors (age 18+) in the MDC general public visitor audience	40,000 adults
Desired sample size of general public adult visitors in the MDC audience	278-294 visitors will be approached to obtain a sample of 250 adults
Respondent selection method	One adult per randomly selected visitor group, when exiting from the exhibit area of MDC
Estimated rate of cooperation of randomly selected adult visitors	85% [x 294 or fewer visitors for a final sample of 250]

Note: Results of the social scientist researcher who conducted the first study, as well as the rates of cooperation at similar facilities (aquariums, museums) averaged 90%; the rate from about 20 projects in the last two years has ranged from 72% to 98%.

2. Describe the procedures for the collection, including: the statistical methodology for stratification and sample selection; the estimation procedure; the degree of accuracy needed for the purpose described in the justification; any unusual problems requiring specialized sampling procedures; and any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Characteristics of patron types at visitor centers and museums may vary considerably (e.g., a local family may be followed by a tourist couple who may be followed by a single adult tourist). In places with relatively low volumes of visitors (such as the Mokupāpapa Discovery Center, compared to high volume places such as the Smithsonian) a representative random sample of visitor groups can be obtained by using a “next available” protocol, as follows:

The interviewer is positioned near the exit from the exhibit space. As any visitor group (usually 1-4 people) nears the exit, the interviewer approaches and makes eye contact with the ‘first adult’ (in practice: the one who is physically closest to the interviewer) and requests their participation in giving feedback about the exhibits. The cooperation rate for this type of intercept interview (using a brief introduction that explains the purpose in one sentence) typically averages about 90%. If the adult visitor agrees, the interview is completed. Upon completion, the interviewer will tend to step aside to complete their work on the interview form (documenting the date and time of the interview, adding their own initials to it, reviewing the form to check for completeness and readable handwriting, and also to put away that completed interview form and have a new blank one ready); this process usually takes 3-5 minutes. When the interviewer is then prepared with a new blank interview form and related materials (e.g., a photo board about the exhibits, used for some of the interview questions), he/she looks up and selects the “next available” visitor group.

The principle of this and other sampling methods is that the interviewer does not choose who to interview by appearance, or by facial expression that might indicate enjoyment or not, or by whether there are or are not children in the group; in essence, the visitor group selects themselves (although they don’t know the sampling parameters) by deciding when to exit (e.g., there may be another group being interviewed at the time when this group leaves, in which case they would not be selected). Depending on the visitor flow, the next visitor group might be leaving right then, or the interviewer might have to wait for 5-10 minutes for the next group to leave. This characteristic of ‘low volume’ visitor facilities makes it impractical to use other methods such as selecting every 4^th visitor group, or using a random number chart (for example, from 1 to 5) to decide which visitor group to select. While additional methods could be used to provide reliability assessments of the sampling method, the budget is modest in this particular project, and we are choosing to put relatively more effort in the analysis of open-ended questions than in conducting a rigorous reliability study, trusting that a well-conducted random sampling of “next available” visitor groups will result in a sufficiently representative sample. We will make an effort to balance the sampling between weekday and weekend surveys to ensure capture of both local and visitor traffic.

3. Describe the methods used to maximize response rates and to deal with nonresponse. The accuracy and reliability of the information collected must be shown to be adequate for the intended uses. For collections based on sampling, a special justification must be provided if they will not yield "reliable" data that can be generalized to the universe studied.

Based on responses to MDC’s prior survey (completed in January of 2010, and noted in Part A, Question 1), and data from similar surveys conducted at aquariums and other interpretive facilities (noted in Part A, Question 12) there is an expected response rate of 85%-90%. Therefore, non-response should not be an issue in this study. Prior experience has shown that inviting visitors to contribute their opinions and feedback is a positive motivator.

When the renewed survey instrument and procedures are approved, MDC will begin monitoring the patron cooperation rate. If it is below 75%, MDC will modify the logistics of the survey (where the interviewer stands, which sentence of the explanation comes first) to seek improvements in the cooperation rate. Prior survey cooperation rates have yielded significant reliable data and were well above the 75% benchmark.

4. Describe any tests of procedures or methods to be undertaken.  Tests are encouraged as effective means to refine collections, but if ten or more test respondents are involved OMB must give prior approval under the Paperwork Reduction Act.

A prior pilot survey of 8 visitors had been conducted, which confirmed essential prerequisites for this survey. Visitors did not need an incentive to participate, comprehended questions, provided succinct responses, and completed the interview process.

The survey instrument was originally designed by a professional evaluation company called People, Places and Designs Research (http://ppdresearch.com/; http://ppdresearch.com/profile/) managed by Jeff Hayward. PPD Research works with many Science Centers and Museums and is highly respected in this field.

5. Provide the name and telephone number of individuals consulted on the statistical aspects of the design, and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The instructional designer and information scientist who adapted the research design from the original survey, and composed the survey instrument, is Kālewa Correa, MLISc, MEd,

Manager of Mokupāpapa Discovery Center, [email protected], (808) 933-8181.

Kālewa Correa will be NOAA’s informational designer and responsible for data compilation and synthesis. Representative data will be used for exhibits, programs, and related ways of educating the public about Papahānaumokuākea Marine National Monument.

2