National Endowment for the Arts
Pilot Study: How the Arts Affect Audiences
Data Collection Guidelines:
Intercept Methodology for On-Site Survey Administration
In October 2010, the National Endowment for the Arts (NEA) released a strategic plan establishing a new set of goals, outcomes, and performance measures for the Agency. Under the plan, a critical NEA goal is to “engage the public with diverse and excellent art” so that three specific outcomes are achieved. The first outcome is that “Americans throughout the Nation Experience Art.”
To track its performance in achieving this outcome, the NEA will conduct a survey of audiences at NEA- funded arts events nationwide. The survey will allow the NEA to measure the percentage of audiences “affected” by attendance at NEA-funded projects.
The rationale for this measure is simple: individual encounters with art can prove transformative. As NEA Chairman Rocco Landesman has stated: “Art works on and within people to change and inspire them; it addresses the need people have to create, to imagine, to aspire to something more.” As a partial measure of its success, the Agency plans to gauge the quality, depth, and nature of audience responses to NEA- funded projects.
The purpose of this document is to communicate essential information to participating grantees, including instructions for creating a sampling plan, and guidelines for surveying audiences that will maximize the probability of successful data collection efforts. WolfBrown (www.wolfbrown.com) prepared these guidelines for the National Endowment for the Arts.
Rebecca Ratzkin, Consultant ([email protected])
Kyle Marinshaw, Impact Assessment Program Manager ([email protected])
WolfBrown
808A Oak Street
San Francisco, CA 94117
(415) 796-3060
This survey approach consists of two basic tasks: 1) intercepting patrons as they enter the specific exhibition/event of interest; and 2) administering the on-site survey to the intercepted-patrons as they exit the exhibition/event. In brief, as patrons enter the exhibition/event, survey workers will intercept and distribute red cards to every “Nth” visitor, meaning that that visitor has been pre-selected as a survey respondent - “Nth” stands for the number by which you count patrons in order to identify those who are intercepted (e.g. every 3rd person). Survey workers will then re-intercept visitors as they exit to check if they were handed a red tag, and ask them to complete the survey.
1) Intercepting patrons as they enter the exhibition/event
To intercept a random sample, one survey staff member will be the “interceptor” – the person who identifies every “Nth” adult who walks past the “counting area” (an area demarcated with landmarks, such as doors or an imaginary line near where staff is stationed outside the entrance to the exhibition/event). They will approach the visitor (see Intercept Script) and hand them a red card to be eligible to take a post-event survey.
Important notes about random selection:
The counting plan (counting every “Nth” visitor) of adults who walk past the “counting area” may need to be adjusted, meaning to increase or decrease “N”, during the fielding session depending on foot traffic. You may shift the count based on how crowded the museum is at a given time, but this shifting should be done at maximum once a day, and with sound justification (i.e., the counting plan should not be changed only to speed up survey collection). Any shift in the counting plan must be reported (see Reporting Requirements). Initial “N” will be determined in consultation with WolfBrown liaison, depending on the time of day and scale and scope of exhibition (e.g., traffic over the weekend will be higher so “N” will be higher).
Count all adults (aged 18+) whether they are in groups, couples, or alone
Always exclude – from both counting and intercepting – anyone under 18. If in doubt about whether the person is age 18+, either ask or err on the side of caution (e.g., assume they aren’t an adult and move onto the next person)
INCLUDE in counting, but EXCLUDE from intercepting:
Leaders of group tours
Museum employees or volunteers
School groups (younger than 18)
Only intercept one member of each visiting party to be given a red card. If the next “Nth” person is in the same party/group, begin counting again until you get someone not affiliated with that party/group
Do not allow anyone not intercepted and given a red card to take the survey.
If a selected visitor refuses but says “My wife will take it for me” (or something similar), explain that following a random sample is crucial to the validity of our study. Encourage the selected visitor again to take a red card, but if s/he still does not want to, say thank you and move on and begin counting again
If someone approaches you wanting to complete the survey when they have NOT been randomly intercepted, explain that we are taking a random sample of visitors for the survey
Visitors must have been inside the exhibition/event for an appropriate amount of time (at least five to 10 minutes, depending on the scale and scope of the exhibition) to complete the survey
Entrance Intercept Script to Distribute Red Cards
When intercepting visitors, we suggest the following language:
“Hello, we are conducting a very important survey for the National Endowment for the Arts on how the arts affect audiences. Would you be willing to spend about five to ten minutes taking a short survey about your reactions to the exhibition/event you are about to see on your way out?”
IF “NO” OR RESISTANT, politely press for participation using these points:
The survey is very important – it will help us to better understand how our work affects you
We need to get a representative sample of our visitors. Because we are only conducting surveys on certain days, you represent the many other visitors who will come on a day when we’re not here
It will only take about ten minutes
It is completely confidential and anonymous
Once visitor shows interest, hand them the red card and say:
“Please take this red card and hand it into the survey worker at the table (over there) on your way out of the exhibit.”
When interacting with audience members, remember to always be professional and polite (but not “chatty” or a “friend”)
2) Administering the on-site survey to the intercepted-patrons as they exit the exhibition/event
The second component of this data collection process consists of administering the on-site survey to the intercepted-patrons as they exit the exhibition/event. A second survey worker, an “exit interceptor,” will intercept visitors a second time as they exit the exhibit to check for whether they have a red card and direct them towards the survey table (note: this is a different survey worker and role from that defined in the section above about intercepting upon entrance to the exhibit). Some patrons with red cards may go directly to the survey table on their own. If not, please use the following script for intercepting:
“Were you handed a red tag? If so, please see my colleague at the table over there and he/she will instruct you about taking the survey. Thank you.”
Show them where to go (a nearby table with survey materials) and hand them off to a “survey administrator” (an optional fourth survey worker) who will provide the respondent with a clipboard and pen, and instruct them on how to take the survey. Chairs should be provided.
After completion of the survey:
“Thank you for taking the time to give us your feedback. Your feedback is meaningful to us.”
A team of at least three is needed. You will designate one as the “entrance interceptor,” another as the “exit interceptor” and a third as “survey administrator,” as described above. A potential fourth survey worker could be responsible for counting the number of men and women as they enter the exhibit. If your exhibit hall is large and you are expecting or experiencing heavy foot traffic, you may need additional survey crew to staff the survey table.
One member of the survey team should be responsible for:
Conferring with WolfBrown on the best location to intercept patrons
Gathering and preparing all needed materials (see Materials Checklist)
Coordinating with on-site staff (e.g., security, front of house) that you will be administering the survey and inform them of the plans (ex. how long, where, etc.)
Identifying your survey area with a large sign and wear “Survey Researcher” badge
Assigning roles to survey workers (see description above)
Ensuring that counts are taken of a) the number of refusals and b) the number of men and the number of women exiting the exhibition
Printing out an Administrative Sheet to write on during the survey administration (see Reporting Requirements)
Submitting online Reporting Requirements (see Reporting Requirements)
Mail all surveys (completed and unused) with the Administrative Sheet to WolfBrown (ATTN: NEA Affect Study, WolfBrown, 808A Oak Street, San Francisco, CA 94117)
Materials Checklist
The grantee is responsible for providing all materials listed below, except where specified:
1. Table and chairs for survey takers
2. Large sign identifying survey area (“NEA Survey Today – Tell Us What You Thought of the Exhibition!”)
3. Badges for survey workers (“Survey Researcher” and Name)
4. Clipboards and pens
5. Red cards (provided by WolfBrown)
6. Paper surveys (provided by WolfBrown)
7. Rubber bands and box or large envelope to ship surveys back to WolfBrown
8. Administrative Sheet (WolfBrown will provide digital copy for grantee to print)
Reporting Requirements
After the survey administration, please submit the following information using the online reporting system:
1. Name of exhibit [event]
2. Date of surveying
3. Start & End times of surveying for each date
4. Number of men and the number of women entering the exhibition
5. Total number of people intercepted (Number of red cards distributed + Number of refusals)
6. Number of red cards distributed (and specifically, number of males and females)
7. Number of refusals (and specifically, number of males and females)
8. Number of completed surveys collected
9. Notes about survey experience for each date (e.g. changes in counting approach, questions people had, anything notable or out of the ordinary that could affect that data)
1 From RFP
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Rebecca Ratzkin |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |