Formative evaluation of exhibits for Governors Island National Monument/Fort Jay

Request for Approval under the “DOI Generic Clearance for the Collection of

Qualitative Feedback on Agency Service Delivery” OMB Control Number: 1090-0011

TITLE OF INFORMATION COLLECTION: Formative evaluation of exhibits for Governors Island National Monument/Fort Jay

PURPOSE:

Governors Island National Monument (GOIS), a unit of the National Park Service (NPS) seeks to understand how a general visitor will access and understand identified exhibit components and content in order to best deliver on the exhibition message and experience goal. This evaluation is designed to inform specific components of an exhibit to be installed November 2016 in and around Fort Jay, a star-shaped fort constructed in 1794. Formative evaluation is conducted during the design stage of the exhibition in order to detect problems and successes of interpretation before building the final product.

The NPS and its exhibit contractor will use formative evaluation to study how visitors experience three specifically identified exhibit components. The evaluation will test if the directives and content are understood by the public as they there intended by the curator and designer. The evaluation will also help the NPS and its exhibit contractor understand the effectiveness of the design choices – are the labels strategically placed, does the interactive work and why or why not, does a headline capture visitor attention and represent the label meaningfully for the public? The components and specific questions of this request were chosen to address areas identified by the NPS and its exhibit contractor as needing particular visitor feedback.

These components including specific images and exhibition labels will be mocked up in such a way that the respondents have an understanding of the materials in context. They will be asked to engage with the materials as though they were on site and looking at the exhibits.

Once they have looked at the components they will be asked to talk about their experience and answer questions about each component. The respondents will be representative of the general visitation at Fort Jay.

NPS and its exhibit contractor will use this information in the design development phase in order to provide timely feedback to the design process addressing any questions raised by the evaluation and ensuring the opportunity to make any appropriate adjustments before the exhibit components go into final production.

DESCRIPTION OF RESPONDENTS:

We will systematically recruit from individual stakeholder groups until we have reached the goal of 60 participants.

TYPE OF COLLECTION: (Check one)

[ ] Customer Comment Card/Complaint Form [ ] Customer Satisfaction Survey

[ ] Usability Testing (e.g., Website or Software) [X] Small Discussion Group

[ ] Focus Group [] Other:

CERTIFICATION:

I certify the following to be true:

1. The collection is voluntary.

2. The collection is low-burden for respondents and low-cost for the Federal Government.

3. The collection is non-controversial and does not raise issues of concern to other federal

agencies.

4. The results are not intended to be disseminated to the public.

5. Information gathered will not be used for the purpose of substantially informing influential policy decisions.

6. The collection is targeted to the solicitation of opinions from respondents who have experience with the program or may have experience with the program in the future.

Name: Susan Haines ___________________________

To assist review, please provide answers to the following questions:

Personally Identifiable Information:

1. Is personally identifiable information (PII) collected?

[ ] Yes

[ X] No

2. If Yes, is the information that will be collected included in records that are subject to the

Privacy Act of 1974?

[ ] Yes

[ ] No

3. If Applicable, has a System or Records Notice been published?

[ ] Yes

[] No

Gifts or Payments:

Is an incentive (e.g., money or reimbursement of expenses, token of appreciation) provided to participants?

[ ] Yes

[X ] No (If yes, please explain.)

Respondents will be given a unique behind-­‐the-­‐scenes tour of Fort Jay that has no monetary value.

_{BURDEN HOURS}

Category of Respondents	Number of Participants Respondents	Estimation of time (minutes) Time	Burden (hours)
Individuals or households			0
R/T Ferry Ride Exhibit Evaluation Tour	60 60 60	40 30 30	40 30 30
Totals	60		100

During the one hour spent on Governor’s Island one half of the visitors (n=30) will participate in the Exhibit Evaluation where they will look at the exhibit mock ups and answer a series of questions. The other half (n=30) will be given a guided tour of Fort Jay. At the end of 30 minutes the groups will rotate and switch places. At the end of the 60 minute sampling period all visitors will return to the ferry pick-up location.

FEDERAL COST: The estimated annual cost to the Federal government is a one-time cost of $17,485.

_{This includes} hourly salaries and benefits for NPS staff to facilitate this information collection. Currently there are three members of the staff ranging from GS-11 to GS-13 who will spend no more than two hours each participating in this collection of information by providing security for, escorting and hosting the participants. This also includes the NPS costs of ferry transportation ($2 per person) to bring survey participants from Lower Manhattan to Governors Island National Monument on Governors Island in New York Harbor and return.

_{Park Staff - $400}

_{Transportation - $120}

_{Consultant fee of $16,965}

_{Total Cost: $17,485}

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents

1. Do you have a customer list or something similar that defines the universe of potential

respondents and do you have a sampling plan for selecting from this universe?

[ X] Yes

[ ] No

If the answer is yes, please provide a description of both below (or attach the sampling plan). If the answer is no, please provide a description of how you plan to identify your potential group of respondents and how you will select them.

The respondent universe for this information collection will be the members of the following stakeholder organizations that have expressed interest or visited Governors Island in the past: 1) Governors Island Alliance, 2) Lower Manhattan Culture Council, and 3) New York Harbor School. We will recruit participants by sending an invitation requesting participation in the study. The invitation will include the purpose and of the study and the information about the behind the scenes tour of the GOIS. Our goal is to recruit at least 20 people per organization to participate in the study. It is expected that some will accept the invitation but fail to participate, leaving us with about 60 participants.

The invitation is as follows:

Dear…

You have been invited to visit Governors Island to review and offer comments on three proposed outdoor exhibits for Fort Jay. The exhibits are in the design stage now and we would like to have you look over some mock-up depictions of them. Then we would like to ask you several questions about their appearance, their content and design and an overall impression of them.

You will board the Governors Island Ferry at 10 South Street at the designated time [TBD] with about 30 others. On arrival, you will be divided into two groups, one to tour Fort Jay and the other to review the exhibits. About 30 minutes later, each group will switch places. It is estimated with your visit and ferry crossings, this opportunity will last about 2 to 2.5 hours. Unfortunately, this remainder of the island is closed to the public until the 2016 season begins Memorial Day weekend. All participants will remain in their groups and escorted at all times while on the island.

If you are interested in participating, confirm your reservation by sending an email to: [email protected] with Fort Jay exhibits in the subject line. In the email, please provide your name and phone number so you may be put on the ferry passenger list or contacted in the event of cancellation. Reservations are limited and offered on a first-come, first-served basis. A confirmation email will be sent to you with instructions.

Should you have questions or need more information, do not hesitate to contact us at: phone and email [TBD]

Administration of the Instrument

1. How will you collect the information? (Check all that apply)

[ ] Web-based or other forms of Social Media

[ ] Telephone

[X] In-person

[ ] Mail

[ ] Other, Explain

2. Will interviewers or facilitators be used? [X ] Yes [ ] No

Please make sure that all instruments, instructions, and scripts are submitted with the request.

Background Information

The participants will be divided into two groups of 30 individual and will be scheduled to participate in separate sessions. Both groups will be managed in an identical manner upon arrival. Each will arrive on Governors Island by ferry and will be greeted by the National Park Service Staff. The first group of 30 participants will be divided in half so that one group of 15 will go to the building for the exhibition evaluation and the other group of 15 will take a guided tour of Fort Jay. Then the groups will switch. The evaluation will take place during the off-season and the island will be closed to the general public. Once the evaluation and the tour are complete the groups will switch roles.

Instructions and script

Script for exhibit review

The facilitator will greet the group and welcome them to the meeting room.

Welcome to Governors Island National Monument. My name is Janet Rassweiler; I am an evaluator working with the Monument on the exhibit design project you are here for today. You will be here for a total of about 30 minutes. No voice recordings will be taken, only written notes.

Thank you for agreeing to help the National Park Service with their efforts to design exhibits that will work well with visitors to provide a more enjoyable experience for everyone.

Before we begin, I would like to get a sense of your experience with this Island. How many of you have been here before? What did you do while you were here? If you have not been here, what do you know about the place?

In a minute you will visit the room next door and look at the temporary exhibits. You will notice that they are not like the final components you would see in a museum or at this historic site. We are fortunate to have you here to help us test these out before we make the final product. There will be three main sections, so feel free to spread out so you can really look at each one. Please make sure you look at all three. Keep in mind that they will not be next to each other in their final placement; they are just this way for our testing purposes.

You will have about 15 minutes to look at all of the pieces in the room. Then during the remaining 15 minutes I will ask questions about your experiences with each of the components. I will be writing notes in order to give the exhibition designers insight from the discussion we have.

The group will gather in front of each component and the facilitator will ask the questions listed below. There may be clarifying questions used as follow up. Written notes will be taken during the process.

Exhibit #1: Welcome/introduction to Fort Jay

Introductory images and information about Fort Jay are presented in this component. Most prominent is an aerial view of the star-shaped fort. A bold headline and subtitle; brief introductory text, the view of the fort and an evocative historical image will set the stage. A particular focus for research will be on the element “experience the star” in which images and text convey what else there is to see within the fort.

Questions for Exhibit #1

1. Would you stop and read this panel?

   • If yes, why

   • If no, why not?

   • What could we change to make it more appealing to visitors?

2. From this panel, what do you think you might do or see inside Fort Jay?

3. From what you found out, do you think you would go into the Fort?

   • If yes, why?

   • If no, why not?

4. What could we change to make you more likely to go inside the fort?

Exhibit #1 - Map

Exhibit #1 will also include a sloped panel with tactile map. It will show the fort within the context of the entire island. The historic area is delineated and ferry landings and National Park Service sites are identified.

Questions about the map specifically:

1. What did you first notice?

2. After looking at this map, what do you remember about things to see on Governors Island?

3. What might you visit and why?

4. Is there anything else that you would want to know?

Exhibit #2 Eagle wayside

This exhibit will include a 3-D tactile of the sculptured eagle atop the entrance of Fort Jay. Positioned so that the sculpture is in view, here visitors see a close-up, full-color raised relief representation of the eagle, helping them to really “see it.” They can touch it to feel its shape; text identifies symbolism and meaning. There will be a sidebar that will provide information about the preservation efforts of the sculpture. The intention of this exhibit is to inform the visitor about the history of this sculpture, , and walk away understanding more about preservation efforts.

Questions for Exhibit #2 Eagle wayside

1. What did the title of this component bring to mind for you?

2. After reading this panel, what would you tell someone else about the eagle?

3. What was the most interesting aspect of the “sidebar”?

4. Why was it interesting to you?

In reference to the hands-­‐on 3d eagle element

1. What do you think this demonstrates?

2. How likely would you be to stop and touch this element?

3. What do you think we could change to make this more effective?

Exhibit #3 Model and interpretive reader rail

The exhibit provides an accurately scaled model that is designed to mimic the actual orientation of the fort so visitors can easily make connections between the model’s elements and the architectural elements they represent. A shallow reader rail of text and images will surround the model.

Questions for Exhibit #3­‐ Model

1. What do you think this model is about?

2. After reading these labels what specific details or general ideas about the fort do you remember and why?

3. What would you like to know more about and why?

4. Are there any words or phrases that may be confusing for visitors?

5. Did you find the information easy to understand?

6. What could we change to more interesting to you?

Is there anything else you would like us to know about your experiences with these exhibit pieces that would help us make them work well for other visitors? Thank you again for taking the time to meet with us.

INSTRUCTIONS

TITLE OF INFORMATION COLLECTION: Provide the name of the collection that is the subject of the request. (e.g., Comment card for soliciting feedback on xxxx)

PURPOSE: Provide a brief description of the purpose of this collection and how it will be used. If this is part of a larger study or effort, please include this in your explanation.

DESCRIPTION OF RESPONDENTS: Provide a brief description of the targeted group or groups for this collection of information. These groups must have experience with the program.

TYPE OF COLLECTION: Check one box. If you are requesting approval of other instruments under the generic, you must complete a form for each instrument.

CERTIFICATION: Please read the certification carefully. If you incorrectly certify, the collection will be returned as improperly submitted or it will be disapproved.

Personally Identifiable Information: Provide answers to the questions.

Gifts or Payments: If you answer yes to the question, please describe the incentive and provide a justification for the amount.

BURDEN HOURS:

Category of Respondents: Identify who you expect the respondents to be in terms of the

following categories: (1) Individuals or Households; (2) Private Sector; (3) State, local, or tribal governments; or (4) Federal Government. Only one type of respondent can be selected.

No. of Respondents: Provide an estimate of the Number of respondents.

Participation Time: Provide an estimate of the amount of time required for a respondent to

participate (e.g., fill out a survey or participate in a focus group)

Burden: Provide the Annual burden hours: Multiply the Number of responses and the

participation time in minutes and divide by 60.

FEDERAL COST: Provide an estimate of the annual cost to the Federal government.

If you are conducting a focus group, survey, or plan to employ statistical methods, please provide answers to the following questions:

The selection of your targeted respondents. Please provide a description of how you plan to identify your potential group of respondents and how you will select them. If the answer is yes, to the first question, you may provide the sampling plan in an attachment.

Administration of the Instrument: Identify how the information will be collected. More than one box may be checked. Indicate whether there will be interviewers (e.g., for surveys) or facilitators (e.g., for focus groups) used.

3