MFA Evaluation

Online Survey Questions

Museums for America Evaluation

March 18, 2010

PART A: MUSESUM BACKGROUND and OVERALL MFA APPLICATION PROCESS

COMPLETED BY ALL ELIGIBLE MFA APPLICANTS from 2004-2009

Introduction and Instructions

[Introduction to come: including discussion of survey purpose, and confidentiality]

IMLS will assist in wording of confidentiality

(INFO) = additional definition of terms in pop-up form

Museum Background

This section will be structured at the museum (institution\organizational unit) level (n~1800)

1. Pre-Populated Record <Fields> To Appear On The Online Survey

1.1 Are you knowledgeable about each of the following MFA applications?

 <Institution/Museum Name>  <OrgUnit>

All MFA applications submitted to IMLS by first submit date to last

Knowledgeable about?		Fiscal Year	Project Title	Grant Category	Current Status	Contact Name	Contact Title
No	Yes
No	Yes
No	Yes
No	Yes

2. Museum Background

2.1 Which of the following most closely describes your museum? (select one)

	Aquarium		Historic House/Site		Science/Technology Museum
	Arboretum/Botanical garden		History Museum		Specialized Museum (INFO)
	Art Museum		Natural History/Anthropology Museum		Zoo
	Children’s /Youth Museum		Nature center
	General Museum (INFO)		Planetarium		Other; please specify:_____________________

(INFO) A museum with collections representing two or more disciplines equally (e.g., art and history)

(INFO) A museum with collections limited to one narrowly defined discipline (e.g., textiles, stamps, maritime, ethnic group, tribal)

2. Which of the following most closely describes your museum’s governance? (select one)

	College, university or other academic entity		Federal		Local (county or municipal)
	Non-profit, non-governmental organization or foundation		State
	Native American Tribe/Native Hawaiian Organization

2.2.a. If your museum is governed by a college, university, or other academic entity, which of the following most closely describes your governance? (select one)

	Community college		Private four-year college or university
	Historically Black college or university		State four-year college or university
			Other; please specify: ___________________

  

2. What year was the museum first open to the public? ______

2. How many people have held the museum’s leadership role (e.g., director, curator, president) over the past seven years (since 2004)? ______

2. How many current staff members does the museum have?

___ full-time employees

___ part-time employees

___ unpaid staff (volunteers, docents, Board members)

2. Other Grant Programs

3.1 How many times has your museum applied for any IMLS grant in the last 12 months? ______ _ Don’t Know

3.2. Has your museum applied for any OTHER IMLS grants listed below?

3.2. If yes, were you awarded a grant?

3.3. If yes, was it before or after your MFA application?

Other IMLS Grant Programs	3.2. Did your institution apply?			3.3. If yes, were you awarded a grant?		3.4. If funded, was it before or after MFA application?
	No	Yes	Don’t Know	No	Yes	Before	After
a. 21st Century Museum Professionals	Skip to b		Skip to b	Skip to b
b. Connecting to Collections: Statewide Implementation Grants	Skip to c		Skip to c	Skip to c
c. Connecting to Collections: Statewide Planning Grants	Skip to d		Skip to d	Skip to d
d. Conservation Assessment Program	Skip to e		Skip to e	Skip to e
e. Conservation Project Support	Skip to f		Skip to f	Skip to f
f. Museum Assessment Program	Skip to g		Skip to g	Skip to g
g. National Leadership Grants	Skip to h		Skip to h	Skip to h
h. Museum Grants for African American History & Culture	Skip to i		Skip to i	Skip to i
i. Native American/Native Hawaiian Museum Services,	Skip to j		Skip to j	Skip to j
j. American Heritage Preservation Grants	Skip to 3.4		Skip to 3.4	Skip to 3.4

3.4 Has your museum applied for any federal agency grants other than to IMLS?

No

Yes

Don’t Know

III. Application Process

4.Application Process Experience 

In order to improve the MFA application process, IMLS is interested in your experience(s) with the process.  Please take into account all MFA applications your museum has submitted to IMLS.      

4.1. The IMLS Museums for America grant program requires museums to commit to a 1:1 cost sharing for the proposed project. Between 2004 and 2009, did your institution ever CONSIDER applying for a MFA grant but DID NOT APPLY due to the cost sharing requirement?

No

Yes

Don’t Know

4.2. For the MFA application(s) the museum did submit to IMLS, did the cost-sharing requirement have an impact on any of the following with respect to your proposed project(s):

	No	Yes	Don’t Know
Amount of time to raise the cost sharing funds
Type of MFA grant applied for
Duration of the proposed project
Scope of the proposed project
Staffing costs of the proposed project
Non-labor costs of the proposed project

4.3 Which of the following description(s) apply to the person(s) who wrote your MFA application(s)?

(select all that apply)

	Dedicated staffed grant writer (paid or unpaid)		Administrative Leadership (e.g., Executive Director, Head Curator, President)
	Hired grant writer(s) as consultant		Other staff member(s) (e.g., program coordinator)
	Other: please specify________________________		Don’t Know

4.4 Were you aware of the IMLS outreach activities listed below?

4.5 If yes, did you participate in any of the activities?

4.6 If yes, how helpful were the activities in completing your MFA application?

	4.4. Aware		4.5. If aware, did you participate		4.6. If participated, how helpful was the activity?
	No	Yes	No	Yes	Not at all helpful	Somewhat helpful	Very helpful
a. Information session(s) at conferences/meetings	Skip to b		Skip to b
b. Information from national, regional, or state associations regarding MFA application	Skip to c		Skip to c
c. Audio conference calls prior to application deadlines	Skip to d		Skip to d
Id. ndividual counseling through phone calls, emails, or in-person visits	Skip to 4.7		Skip to 4.7

4.7. Were you aware of the following MFA resources posted on the IMLS website?

4.8. If yes, did you utilize any of the resources?

4.9. If yes, how helpful were the resources in completing your application?

	4.7 Aware		4.8 If aware, did you utilize?		4.9 If yes, how helpful was the activity?
	No	Yes	No	Yes	Not at all helpful	Somewhat helpful	Very helpful
a. Sample MFA applications and tips on IMLS website	Skip to b		Skip to b
b. Outcome-based evaluation resources	Skip to 4.10		Skip to 4.10

10. To what extent do you disagree or agree with the following statements? 

Topic	Statement	Strongly Disagree	Disagree	Agree	Strongly Agree	Don’t Know or Not Applicable
Access to MFA grant information	Information about the MFA grant opportunities was easy to obtain.
Clarity of MFA application	The guidelines and requirements were clear.
	Instructions on budsget (e.g.;direct, indirect costs, cost-sharing) were clear.
	The evaluation requirements were clear.
	Instructions on submitting an application were clear.
	The MFA grant-review process was clear.
Grants.gov	Downloading the Grants.gov application from the website was easy.
	Submitting the online applications through the Grants.gov was easy.
	IMLS staff was helpful in answering questions about using the Grants.gov website.
	Submitting an application by hand and mail was easier than using grants.gov

4.11 Including yourself, has anyone on your staff served as a:

	No	Yes	Don’t Know
Field reviewer for the MFA grant program
Grant review panelist for the MFA grant program

5. Application Process Impacts

5.1. IMLS is interested in hearing if the application process itself, regardless of funding, had any effects on your museum. To what extent did the application process prompt your museum to : 

	Not at All Effected	Somewhat Effected	EffectedA Lot	Don’t Know
Create a mission/strategic plan for the first time
Revise your mission/strategic plan
Increase awareness of different programming
Increase awareness of your resources/strengths
Reframe your project in order to fit into the categories in IMLS guidelines
Adopt new ways of integrating outcomes-based planning in the project development and evaluation processes
Explore new external partnerships
Collaborate among departments/staff
Explore new technologies
Improve your institution's ability to apply for other (non IMLS) funding
Other; please specify:

5.2. Is the MFA application process, more difficult (more complex, requires more time), the same or less difficult (less complex, requires less time) than other types of grant applications?

	More difficult	Same	Less difficult	Don’t Know
Other Federal Grants
Other Government Grants (State, Local, County, etc)
Private Grants

1. Were you aware that IMLS DOES NOT track reapplications, therefore each application, whether a reapplication or not, is treated individually and on its own merits?

No

Yes

2. How likely is it that your museum will apply for another Museums for America grant in the future?

Not at all likely

Somewhat Likely

Very Likely

Don’t Know

5.4.a If not at all likely, why not? (Check all that apply)

	Already have an application or project in process
	Our Mission or strategic is no longer current
	Unavailable resources to plan and write an application
	Unavailable or inefficient resources to carry out a project if awarded
	Finding funds/resources for cost sharing is too difficult
	Funding is not needed at this time
	Don't see ourselves as competitive/ discouraged by previous failed applications
	The application process requires hardware and/or software not available to us
	Other; please specify: ____________________________________________

3. What recommendations do you have for improving the MFA application process?

SYSTEM NOTE : IF MUSEUM HAD ALL APPLICATIONS FUNDED Skip to Part C for individual funded project level.

SYSTEM NOTE: IF MUSEUM HAD ANY APPLICATIONS UNFUNDED Continue with Section B.

PART B: OVERALL UNFUNDED MFA APPLICATION REACTIONS

COMPLETED BY ALL INSTITUTIONS with ANY UNFUNDED MFA APPLICATIONS

IV. Unfunded MFA Project(s) (all institutions with any unfunded MFA applications)

• Structured at the institution\organizational unit level (n=1400)

• Unfunded project questions ask about overall experience (not app specific)

6. Application Experience

1. To what extent do you disagree or agree with the following statements:

	Strongly Disagree	Disagree	Agree	Strongly Agree	Don’t Know or Not Applicable
Our application(s) was/were given full and fair consideration.
The reviewers’ comments were useful for redesigning the project.
IMLS staff was helpful in providing feedback.

2. Did your museum do any of the following with any of the unfunded MFA applications:

Did you:	No	Yes	Don’t Know
redesign an unfunded project and resubmit the application to the IMLS-MFA program?
If YES, did you receive IMLS-MFA funding for the redesigned project?
resubmit a MFA unfunded project to a different IMLS grant program?
If YES, did you receive IMLS funding from a non-MFA grant program?
submit a MFA unfunded project to a non-IMLS funder?
If YES, did you receive funding from the non-IMLS funder?

6.2.a. If your museum did not receive any outside funding for unfunded MFA projects, were you able to execute any version of a MFA unfunded project?

No

Yes

Don’t Know

SYSTEM NOTE:IF NO FUNDED GRANTS - EXIT

Thank you for taking the time to reply to this survey.

Please hit the SUBMIT button if you are done.

SYSTEM NOTE:IF ANY FUNDED GRANTS continue with Part C at the individual funded project level.

PART C: FUNDED GRANT PROJECT ACTIVITIES and STATUS

COMPLETED BY ALL MFA GRANTEES (open and closed projects)

Structured at the museum\project level (n ~ 850)

Pre-populated fields for an individual funded project.

SYSTEM NOTE: Table below will be presented at the top of each new survey page for Section C, D, E

540 museums had 1 funded grant; 160 had 2;

42 had 3-5 THOSE WILL BE ASKED TO ANSWER QUESTIONS ON ONLY 2 awards – respondent will chose which ones based on knowledge

Knowledgeable about?		Grant#/Total Grants	Fiscal Year	Project Title	Grant Category	Current Status	Contact Name	Contact Title
No	Yes

V. Individual Grant Project Descriptions

7. Grant Project Background

7.1. Please identify the types of partner organizations (INFO involved in this MFA project.

(INFO) Partner organizations are defined as any outside organization which expended cash or in-kind resources on the project. In-kind resources include s any payment made in the form of goods and services, rather than cash.

	Other Museums; please identify museum type _(DROP DOWN MENU)_________________________
	Library

Government organizations
	State government
	Local or County government
	City Government
	Other; please describe ___________________________________

Non-Government Organizations/Community Organizations
	Community Health facility (hospital, mental health facility, health clinic, etc)
	Family Services Organization (day care, YMCA, family services center, etc)
	Youth Organization (Boys and Girls Club, 4H Club, Afterschool Program)
	Senior Services
	Arts and Culture Organization
	Legal Services Organization (family court, legal aid organization, etc)
	Local Media
	Civic Organization (Kiwanis, Chamber of Commerce, etc)
	Local Business
	Other Non-Profit Organization; please describe_________________________________
	Other; please describe

Education
	Elementary School
	Secondary/High School
	4 Year College or University
	Community College
	Other; please describe

Policy and Research Organization; please identify ______________

Online/ Technology Partner; please identify ______________________

	Other; please describe _____________________________________
	NO partner organizations were\are involved

7.2. Indicate which of the following types of activities (Programming, Exhibitions, Collections, Technology and Online Resources, or Organizational Development) your museum conducted or is in the process of doing as part of this MFA project.

Programming Activities– If this MFA project included programming activities, please indicate for each activity whether the museum developed a new activity, OR enhanced an existing activity (e.g., expanding for different age groups, upgrading materials) OR continued an existing activity.

	Select one - Radio button for selection
	Develop new programs or materials	Enhance existing programs or materials	Continue existing programs or materials
Concerts
Film festivals
Live Performances
Broadcasts
Demonstrations and Workshops
Lectures
Community discussion groups
Education programs at our institution
In-school programs
Afterschool programs
Curriculum guides
Interpretive programs or materials
Classes or institutes
Training sessions
Conferences
Internships, mentoring or apprenticeships opportunities
Other; please describe

No Programming Activities

B. Exhibitions (select all that apply)

	Upgrade/expanded current exhibit, including making current exhibit accessible (e.g., multi-languages, Braille)
	Develop concept for new exhibit
	Research new exhibit
	Plan new exhibit
	Fabricate a new exhibit
	Create traveling exhibition
	Other, please describe

No Exhibitions

C. Collections (check all that apply)

	Digitize collections
	Convert non-digital content to digital content
	Repurpose digital content (INFO)
	Create new digital content

(INFO) Utilized digital content in a different way (e.g., used for education programs)

If the MFA project involved digitizing the museum’s holdings, what portion of the collections did/would the project cover? (select one only)

			Less than 10% of collections
			Between 10-25% of collections
			Between 25%-50% of collections
			Between 50% and 75% of collections
			Over 75% but less than 100% of collections
			100% of our collections
			Not Sure
	Inventory collections
	Move collections
	Create public collections finding guides
	Create new collections management guidelines/procedures
	Implement new collections management system
	Other; please describe

No Collection Activities

D. Technology and Online Resources (select all that apply)

	Update or create new website
	Create online access to collections records or information
	Purchase technology equipment (e.g., computers, digital cameras, scanner, voice recorder)
	Consolidate multiple databases
	Develop searchable online database
	Upgrade, purchase or install new software (e.g., new collections management software)
	Create audio tour
	Create online exhibition
	Develop high-tech interactive exhibition
	Other; please describe

No Technology Activities

E. Organizational Development, including Staffing Resources (select all that apply)

	Provide staff, volunteer and/or docent training, including in use of new technology and online resources
	Hire full-time staff (e.g., program coordinator, director, curator, educator)
	Hire part-time staff (e.g., program coordinator, director, curator, educator)
	Hire a consultant (e.g., interpretive, planning, education, exhibition, web)
	Contract for services (fabrication, design, security, etc)
	Train interpreters, volunteers or docents
	Create or expand interpreter, docent or volunteer program
	Support a research and evaluation program, including conducting surveys
	Develop key management plans (e.g., comprehensive interpretive plan, emergency plans, collections management plans)
	Other; please specify

No Organizational Development

7.3. Which of these groups were\are directly served by this project? (select all that apply)

	Paid staff; please describe
	Unpaid staff (docents, interns, volunteers); please describe
	Adult learners
	Seniors
	Parents
	Youth
	Teachers
	Students
	Administrators
	Professionals
	Residents of particular neighborhood or community/geographic area within a community
	Policymakers
	Persons with disabilities
	Low income citizens
	Specific racial/ethnic communities
	Families
	Not group-specific
	Other; please specify

7.4 Age groups served by your grant project (select all that apply)

	Preschool
	Grades K-5
	Grades 6-8
	Grades 9-12
	Post high school/Young adults (18-25)
	Adults (26-64)
	Seniors (65 and older)
	Multi-age/Family
	Other, please specify

8. Status of Activities

Status of activities –the list of activities that grantees check off in the previous section will be populated to this section.

OPEN GRANTS ONLY

8.1 What is the completion status of the project’s activities?

Example:

	Completion Status
<populated with specified activities>	Will not do as planned	Still in progress	Completed	Don’t Know
Afterschool programs
Curriculum guides
Train interpreters or docents
Establish a docent program and recruit docents
Implement a research and evaluation program, including conducting surveys

8.2. Were any other activities conducted or planned?

Yes

No

If yes, please list activities and current status of each below

	Completion Status
Other activities:	Will not do as planned	Still in progress	Completed

CLOSED GRANTS ONLY

8.1 What is the completion status of the project’s activities?

Example:

<populated with specified activities>	Did not do as planned	Completed	Don’t Know
Afterschool programs
Curriculum guides
Train interpreters or docents
Establish a docent program and recruit docents
Implement a research and evaluation program, including conducting surveys

8.2. Were any other activities conducted or planned?

Yes

No

If yes, please list activities and current status of each below (e.g., Completed, Still In Process

Other activities:	Did not do as planned	Completed	Don’t Know

SYSTEM NOTE/NOT DISPLAYED:

IF this project is in progress OR recently completed AND the museum has no additional MFA granted projects EXIT

OTHER COMMENTS:

As a thank you for your time and interest in participating in this study, IMLS would like to send you a copy of the Museums for America Grant Evaluation Executive Summary.

If interested please fill out the following information:

Name:

Title:

Organization:

Address: City State Zip

Thank you for taking the time to reply to this survey.

Please hit the SUBMIT button if you are done.

SYSTEM NOTE/NOT DISPLAYED:

If this project is completed AND funded under the 3 earlier grant categories, THEN respondent will continue on to Section D: Impacts and Reflections.

If this project is in progress OR recently completed AND the museum has more MFA granted projects THEN the respondent returns to Section C and is asked questions about the next funded project.

PART D: IMPACTS and REFLECTIONS

COMPLETED BY MFA GRANTEES WITH COMPLETED GRANTS UNDER THE EARLIER 3 GRANT CATEGORIES 2004-2007

Structured at the museum\project level (n ~ 350 with ~ 22 with 2 completed projects)

VI. INDIVIDUAL Grant Project Impacts

9. Project Impacts

1. Please review the list of impacts this MFA project may have had on the organization or community.

Select each impact resulting from the MFA project.

ORGANIZATIONAL IMPACTS (select all that apply)

Programming
	Grant-funded programming continued beyond grant
	Sustained grant-funded programming
	Did not sustain grant-funded programming
	Sought additional funding to sustain grant-funded programming
	Developed related programming to expand value of grant-funded program
	Expanded our range of possibilities for programs or exhibitions
	Other; please describe:
	No changes in programming/not applicable to grant

Organizational Capacity
	Enhanced staff capacity in program development
	Enhanced staff capacity in creating new kinds of exhibitions
	Enhanced staff capacity in reaching new or larger audiences
	Enhanced staff capacity in forming external partnerships
	Enhanced staff capacity in working internally across departments
	Enhanced staff capacity in using outcomes based evaluation
	Improved staff cohesion and commitment to mission
	Greater alignment of staff responsibilities with mission
	Greater board involvement
	Helped institution fulfill its mission (e.g., meet certain benchmarks)
	Increased ability to attract outside funding
	Decreased ability to attract outside funding
	Other; please describe:
	No effect on our organization/not applicable to grant

COMMUNITY IMPACTS (select all that apply)

Audiences
	Reached new audiences (e.g., youth, families, minorities)
	Increased commitment by existing audiences
	Changes in regular audience participation
	Increased audience access (e.g., expanded hours, mobile programming)
	Other; please describe:
	No effect on audiences/not applicable to grant

Community Profile
	Changed institutional identity
	Strengthened museum’s public image
	Raised institution’s prestige in its community
	Increased local media coverage of institution’s activities
	Increased visibility of institution as a center of community learning
	Other; please describe:
	No change to our profile/not applicable to grant

Partnerships
	Improved skills in building partnerships
	Strengthened ongoing partnerships
	Developed new partnerships
	With schools or Institutes of Higher Education
	With youth organizations
	With other museums/historical societies
	With other community organizations
	Developed new types of partnerships
	Decreased ability to attract partners
	Increased ability to attract more partners
	Other; please describe:
	No effect on partnerships/not applicable to grant

IMPACTS DUE TO THE PARTNERSHIP(S) (select all that apply)

	Brought in new audiences
	Brought in new resources not normally available to our organization
	Brought in new staff, paid or unpaid
	Brought in new volunteers or memberships
	Raised community awareness of our organization
	Other; please describe:
	No effect from partnerships/not applicable to grant

9.2. Are there any other impacts not covered above? If yes, Please describe

__________________________________________________________________________

9.3. If the grant project enabled new programming, to what extent has your organization been successful in continuing the programs?

Not at all successful

Somewhat successful

Very successful

Not applicable

9.4. If the grant enabled you to bring in new audiences, to what extent has your organization been successful in sustaining these new audiences?

Not at all successful

Somewhat successful

Very successful

Not applicable

9.5. Was this MFA grant from IMLS larger, smaller, or about the same as other grants you may have received?

Larger

Smaller

About the same

Not applicable

9.6. Were the MFA grant funds sufficient to complete the planned project activities?

No

Somewhat

Yes

Don’t Know

SYSTEM NOTE/NOT DISPLAYED :

IF this project ended less than 3 years ago AND the museum has no additional MFA granted projects EXIT

OTHER COMMENTS:

As a thank you for your time and interest in participating in this study, IMLS would like to send you a copy of the Museums for America Grant Evaluation Executive Summary.

If interested please fill out the following information:

Name:

Title:

Organization:

Address: City State Zip

Thank you for taking the time to reply to this survey.

Please hit the SUBMIT button if you are done.

SYSTEM NOTE/NOT DISPLAYED:

If this project was completed 3 or more years ago THEN respondent will continue on to Section E: Long-term Impacts.

If this project is in progress OR recently completed AND the museum has more MFA granted projects THEN the respondent returns to Section C and is asked questions about the next funded project.

PART E: LONG-TERM IMPACTS

COMPLETED BY grantEEs WHERE PROJECT WAS completed 3 or more years ago

VII. Individual Grant Project Long-Term Impacts

10.1. Please describe any long-term impact(s) that the MFA project has had on your organization.

10.2. Please describe any long-term impact(s) that the MFA project has had on your community or community relationships.

10.3. Please describe any UNEXPECTED long-term impact(s) that the MFA project has had on your organization, community or community relationships.

SYSTEM NOTE/NOT DISPLAYED :

IF this project is the only/last one completed under the 3 earlier grant categories THEN GO TO EXIT

If there is another MFA project completed under the 3 earlier grant categories THEN the respondent returns to Section C and is asked questions about the next funded project.

EXIT

OTHER COMMENTS:

As a thank you for your time and interest in participating in this study, IMLS would like to send you a copy of the Museums for America Grant Evaluation Executive Summary.

If interested please fill out the following information:

Name:

Title:

Organization:

Address: City State Zip

Thank you for taking the time to reply to this survey.

Please hit the SUBMIT button if you are done.