Formative evaluation of proposed exhibits at the Valley Forge National Historical Park Visitor Center

OMB Control Number 1090-0011

Visitor Script

Background Information

The evaluator will travel to the Valley Forge Historical Park Visitor Center to test the prototypes of six exhibit elements. The prototypes will be set up in an area separate from the Visitor Center; they will be in the order that they are expected to be placed in the proposed exhibition.

For the general public sessions, the evaluator will implement a random selection process to recruit up to ten visitors to view the prototypes and participate in a face-to-face interview. Visitors will be recruited as they enter the visitor center, near the information desk. The evaluator will conduct a separate discussion group with four fourth-grade teachers recruited by the Park. They will spend time viewing the prototypes set up in the classroom and then provide their perspectives on the proposed exhibition based on their experience as fourth-grade teachers.

Instructions and Script

Invitation to the on-site participants

Hi! Welcome to Valley Forge National Historical Park. My name is _______; I am an evaluator working with the park on the exhibit design project you are here for today. The National Park Service would like your help. We are developing a new exhibition for the visitor center about Valley Forge. Do you have a few minutes to look at six of the proposed exhibits that we have here? We are interested in understanding how the exhibit will be viewed by our visitors. This will take about 45 minutes of your time: 15 minutes to view and interact with the exhibits, and 30 minutes to answer a few questions about your impressions of the exhibits. We are hopeful that you will benefit from your experience as much as we will. Would you like to start?

 If a soft No the interviewer will say:- It is important to us that as many people as possible participate by helping us to collect data that will help us to understand our visitors perception of the exhibits, even those who do not participate in this study. Would you mind answering two quick questions?

1. Have you visited Valley Forge National Historic Park before?  Yes  No

 [If yes:] When was the last time you were here?

2. Can you tell me anything that you know about this place? [Probe for what time period, war, and/or significance it might have.]

 If a hard NO – the interviewer will thank them for their time, and begin random selection process again.

 If Yes – the interviewer will take them to area where the prototypes are set up; they may bring family/friends with them.

Before we begin, I would like to let you know that this survey has been approved by the Office of Management and Budget. I am also required to tell you that a Federal agency may not conduct or sponsor, and that you are not required to respond to, a collection of information unless it has a currently valid OMB control number. The control number for this collection is 1090-0011. Secondly, your participation is voluntary and your name will never be connected with your individual responses. Finally, if you have any questions about this survey, I have a name and contact number available if you would like to have it. Can we begin?

NOTE: The PRA statement below will be printed on 3x5 index cards and given to each interviewee at the time they agree to participate.

Paperwork Reduction Act Statement: The Paperwork Reduction Act requires us to tell you why we are collecting this information, how we will use it, and whether or not you have to respond. This information will be used by the National Park Service as authorized by 54 U.S.C. 100101. We will use this information to evaluate the new exhibits for Valley Forge Visitor Center. Your response is voluntary. No personally identifiable information is being collected. A Federal agency may not conduct or sponsor, and you are not required to respond to, a collection of information unless you are informed of the currently valid OMB Control Number which is 1090-0011.-We estimate that it will take about 45 minutes to complete this short on-site survey. You may send comments concerning the burden estimates or any aspect of this information collection to: Mary Grassick, at [email protected] (email).

OMB Control Number 1090-0011

After visitor responses are noted, the data collector will explain that the prototypes are set up in the order that they will be placed in the proposed exhibit. The visitor will be given 15 minutes to look at the exhibits. All behaviors will be observed and recorded at each section (e.g., to see if they read the text, call to one another to see/do something, point, etc.) and the data collector will note comments they make. Once they indicate they are finished, they will be asked a series of questions.

Introductory Script for Visitors:

Thank you for agreeing to help the National Park Service with their efforts to design exhibits that we are hoping will provide an enjoyable experience for everyone visiting Valley Forge. You will be here for a total of about 45 minutes. No voice recordings will be taken, only written notes.

Before we begin, I would like to get a sense of your experience with this park.

1. Have been here (Valley Forge National Historic Park) before?  Yes  No

[If yes:] When was the last time you were here?

2. Can you tell me anything that you know about this place? [Probe for what time period, war, and/or significance it might have.]

In the room next door you will have a chance to look at and interact with the temporary exhibits we have planned for this site. You will notice that they are not like the final components you would typically see at this historic site like this. We are fortunate to have you here to help us test these out before we make the final product. Please make sure you look at all the exhibits.

You will have about 15 minutes to look at all of the pieces in the room. After that, we will spend about 30 minutes talking about your experiences with each of the components. I will be writing notes in order to give the exhibition designers insight from the discussion we have.

Elements 1 and 2: Valley Forge Introduction & Context and Spinner Interactive: Introductory images and information in this display will provide context for the story of Valley Forge. The main panel will interpret the events that served as the impetus for the Revolutionary War. Visitors will learn that Valley Forge was not the first winter encampment of the war and that, even after the soldiers marched out in June 1778, the war continued for another five years. There will be an introduction wall to the right of the entrance that will feature the text to establish the mood and tone of the exhibit. Adjacent to this wall will be a large mural depicting soldiers from the Continental Army marching into Valley Forge. In front of the mural, there will be text discussing the events of the Revolutionary War. There will be a freestanding structure with cut-out flat figures of soldiers and camp followers marching into the camp. Labels on a series of spinning graphics will discuss the reasons why people came to Valley Forge. Visitors will learn the perspectives and motivations of soldiers and camp followers for fighting the war or following the army

Observations

Did the Visitor…	Notes:
 Glance around (no reading)
 Study labels/look closely at graphics
 Read text out loud
 Point to something
 Show/explain something to another visitor
 Talk about exhibit w/someone
 Turns spinnable cube(s)
# cubes turned: ___________

Interview questions

First, I want to ask you about the Elements 1 and 2 – the Introductory Area.

1. What was your first impression of this section?

Follow up probes

• What jumped out at you first?

• What do you think this section overall is trying to tell people?

• What makes you think that?

• Was there anything about it that might be confusing for other people?

Now let’s talk about the spinnable cubes

2. I noticed that you  did/ did not try this activity.

Follow up probes

• Was there something about the activity that stopped you from trying it? Please explain.

3. To what extent do the spinnable cubes help you understand what Valley Forge is about?

4. To what extent do the spinnable cubes help you understand the meaning of freedom…

…to people back then:

…to people now:

Element 3: Health and Medicine Dialogue Game

The purpose of this exhibit is to interpret the conditions at the encampment and exemplify how the army endured and persevered despite the odds. Visitors will learn about the threat of disease to the Continental Army. Visitors will sit opposite each other and take on the role of either patient or doctor. The patient will select one of four vectors of disease illustrated in a flip book - each vector is paired with a disease—smallpox, influenza, dysentery, and pneumonia—each with unique symptoms. The doctor sits opposite the patient and asks a series of questions to try to diagnose the patient. Once the doctor issues a diagnosis, the patient flips a panel to reveal the disease and see if the doctor was correct. At the end, visitors will learn the diseases that threatened the encampment and understand that soldiers would not be allowed to stay at the encampment, where diseases were easily spread.

Observations:

Did the Visitor…	Notes:
 Glance around (no reading)
 Study labels/look closely at graphics
 Read text out loud
 Point to something
 Show/explain something to another visitor
 Talk about exhibit w/someone
 Try the Doctor-Patient interactive
Diseases discussed: # questions asked:

Interview questions:

Now let’s talk about the doctor-patient interactive

5. I noticed that you  did / did not try this activity.

Follow up probe

• Was there something about the activity that stopped you from trying it? Please explain.

6. What is this exhibit trying to show people?

Follow up probe

• What makes you think that?

7. Did the instructions make sense?

Follow up probe

• Is there anything that might be confusing for others?]

8. What did you like about this activity?

9. What did you dislike about this activity?

Follow up probe

• What suggestions do you have to improve this activity?

10. After seeing this area, what would you expect to see or do next in the exhibit?

Element 4: Manual Exercise Interactive

The purpose of this exhibit will be to demonstrate the series of actions in which the soldiers in the Continental Army were taught to use a musket or rifle. Visitor to learn the steps involved in priming, loading, and firing a musket or rifle similar to that used by the soldiers. The intent is to provide an interactive opportunity in order to form an appreciation for the efficiency needed to fire and reload a weapon during the Revolutionary War.

A series of thirteen historic images and text will be used to explain the steps the soldiers used to fire a musket. A mocked-up block rifle will provided for visitors to use in the exercise. Visitors will be tasked with following instructions and firing as quickly as possible, while retaining accuracy and precision. This is used to demonstrate what it would take for a soldier to fire as many shots as possible during a battle.

Observations:

Did the Visitor	Notes:
 Glance around (no reading)
 Study labels/look closely at graphics
 Read text out loud
 Point to something
 Show/explain something to another visitor
 Talk about exhibit w/someone
 Handle block rifle  Use the timer
#times rifle activity attempted:

Interview questions

Now let’s go over to the Manual Exercise Interactive

11. I noticed that you  did/ did not try this activity.

Follow up probe

• Was there something about the activity that stopped you from trying it? Please explain.

12. What is this activity trying to show people?

Follow up probe

• What does this section tell you about why it was important for the soldiers to learn to load and fire quickly?

13. Did the instructions make sense?

Follow up probe

• Is there anything that might be confusing for others?]

14. What did you like about this activity?

15. What did you dislike about this activity?

Follow up probes

• What suggestions do you have to improve this activity?

• What can we do to make it more fun?

• What can we do to help people get a sense for the kind of training soldiers needed?

Elements 5 and 6: The Valley Forge Legacy

These elements will interpret the legacy of Valley Forge through interactive graphic panels that represent a shift forward in time showing that story of Valley Forge continues to evolve. The purpose of this exhibit is to encourage visitors to consider the evolution of Valley Forge over time and what it means today. Graphic panels and text will be used to explain major events, battles, and post-encampment transitions. The legacy will also include the progression from the original pastoral lands to the current National Park setting.

An interactive panel titled “What does Valley Forge Mean to You?” will invite visitors to participate in a post-it note activity. The visitor will receive instructions on how to use an iPad to post a message describing what Valley Forge means to them. Visitor responses will “fly” onto video monitors and join other responses to form a “word cloud”. As more visitors respond, the word cloud will grow larger.

Observations:

Did the Visitor	Notes:
 Glance around (no reading)
 Study labels/look closely at graphics
 Read text out loud
 Point to something
 Show/explain something to another visitor
 Talk about exhibit w/someone
 Uses iPad interactive (“What does VF mean to you?”)

Interview questions

Now let’s go over to the Perseverance/Legacy Area:

16. What was you first impression of this section? [Probe: What jumped out at you first?]

Follow up probes

• What do you think this section is trying to tell people?

• How does this section relate to what you just saw (Intro Area and rifle)?

• Was there anything about it that you thought might be confusing for other people?

• What information or elements do you need to understand that this section is about the importance of memorializing Valley Forge?

Now let’s talk about the “What does Valley Forge mean to you?” activity:

17. I noticed that you  did/ did not try this activity.

If visitor did not try, ask: Was there something about the activity that stopped you from trying it? Please explain.

18. What do you think this activity is trying to show people?

19. Did the instructions make sense?

Follow up probe

• Is there anything that might be confusing for others?

20. What did you like about this activity?

21. What did you dislike about this activity?

Follow up probe

• What suggestions do you have to improve this activity?

We are almost done. I just want to ask you some wrap-up questions.

22. Now that you have seen everything here, how would you describe what this exhibit is about?

Follow up probe

• What would you say is the main message?] Follow ups: What happened here at Valley Forge?

• What significance does Valley Forge have? Why is it important to preserve Valley Forge?

Thank visitors for their time and ask them to complete the demographic questions on the next page.

Visitor Demographic Questions

1. What is your gender:  Male  Female

2. To what age group do you belong?

	18-19 years		30-34 years		45-49 years		60-64 years		75-79 years
	20-24 years		35-39 years		50-54 years		65-69 years		80+ years
	25-29 years		40-44 years		55-59 years		70-74 years		Prefer not to answer

3. Who are you here with today? Check all that apply.

 Friends  Family  Alone Group:

Total number of adults: Total number of children under 18 years:

4. Are you Hispanic or Latino?

 YES

 NO

5. Which best describes your race? Check as many as apply.

 American Indian or Alaska Native

 Asian

 Black or African American

 Native Hawaiian or other Pacific Islander

 White

OMB Control Number 1090-0011

TEACHER DISCUSSION GROUP

[The park is responsible for recruiting 4 fourth grade teachers.]

Instructions and Script

Before we begin, I would like to let you know that this survey has been approved by the Office of Management and Budget. I am also required to tell you that a Federal agency may not conduct or sponsor, and that you are not required to respond to, a collection of information unless it has a currently valid OMB control number. The control number for this collection is 1090-0011. Secondly, your participation is voluntary and your name will never be connected with your individual responses. Finally, if you have any questions about this survey, I have a name and contact number available if you would like to have it. Can we begin?

Thank you for coming out today. As you know, the National Park Service would like your help in developing a new exhibition about Valley Forge that will replace the one currently in the Visitor Center. First, we want to invite you to look at the prototypes we have set up, and then afterwards we would like to talk with you about your reactions to the proposed exhibition. You will have about 45 minutes to look at all of the pieces in the room. Then during the remaining 45 minutes I will ask questions about your experiences with each of the components.

I will be writing notes in order to give the exhibition designers insight from the discussion we have. We are especially interested in your perspectives as fourth grade teachers who might bring your students to see the new exhibition.

1. What pre-visit activities do you typically do with your students before coming to Valley Forge?

• Probe for things they might do/discuss to get class ready.

2a. What is your overall impression of the Introductory Section as a fourth grade teacher?

Follow up probes

• What do you think the section overall will communicate to your students?

• To what extent do you think the spinnable cubes will help your students understand the meaning of freedom…

…to people back then:

…to people now:

2b. How do you think the doctor-patient interactive will help your students understand that the spread of disease could threaten the encampment?

2c. What do you think might be confusing for your students?

2d. What information or interpretive elements do you think your students will need to understand this Introductory Section?

3a. To what extent do you think that the rifle activity will be appealing to your students?

Follow up probe

• Would you say the instructions are easy to read/understand for a fourth grader?

• To what extent do you think this activity will help fourth graders understand the kind of training the soldiers needed?

3b. What, if anything, can we do to improve this activity?

4a. To what extent do you think that the word cloud activity will be appealing to your students?

Follow up probe

• Would you say the instructions are easy to read/understand for a fourth grader?

4b. To what extent do you think this activity will help fourth graders understand the kind of training the soldiers needed?

4c. What, if anything, can we do to improve this activity?

5a. Now, what is your overall impression of the Valley Forge Legacy Area as a fourth grade teacher?

Follow up probes

• What do you think the section overall will communicate to your students?

• How does this section relate to the other sections you just saw?

• What, if anything about it that you thought might be confusing for your fourth grade students?

• What information or interpretive elements do you think your fourth graders will need to understand that this section is about the memorialization of Valley Forge?

14

Valley Forge Visitor Center

Formative Evaluation Plan and Instruments