Museums for America Webinar on Reporting and Evaluation

Webinar on Reporting and Evaluation
for Museums for America Grantees
January 6-8, 2009

Purpose of Webinar
• Inform you about the agency’s reporting
requirements
• Help you share the results of your grants
• Share with you the importance of regular
evaluation and provide resources
• Answer questions that you may have about
evaluation and reporting

Intended Outcomes
At the end of this session, you will know:
• What IMLS reporting requirements are
• How to complete Part 2 of the Final
Performance Report form
• What evaluation resources are available to
assist you

Format, Logistics

Part 1
• Reporting Requirements (20 minutes)
• Questions from grantees (15 minutes)
Part 2
• Evaluation and Resources (10 minutes)
• Questions from grantees (15 minutes)

Managing your IMLS grant
• Recipients of IMLS grants are required to adhere to grants
management terms and conditions. Resources for
managing your award are located at:
http://www.imls.gov/recipients/recipients.shtm
• Any change to a grant project’s activities, key personnel, or
budget must first be approved by IMLS. Requests for a
change to a grant must be signed by the authorizing official
and submitted to the appropriate IMLS program officer.
The request must be approved before the changes
can be made!

Grant Life Cycle / Role of Reporting

Apply
• Establish an
evaluation
plan

Implement
• Collect data,
conduct
evaluation

Report
• Report on
Outcomes to
agency,
funders
(narrative,
quantitative)

Disseminate
• Share
outcomes
with your
institution,
media, peers

The Value of Project Reports
• Your Organization
– Help demonstrate results in concrete terms
– Provide feedback for program planning and development
– Help you communicate results to staff, board,
community, media, conferences, newsletters

• IMLS
–
–
–
–
–

Government agencies
Media, project profiles
Annual reports to Congress
Reports to the field, profession
American public

IMLS’ Required Documents

• Narrative reports
– Interim - due every 6 months throughout grant.
– Final - due 90 days after the close of the grant.

• Quantitative report
– Final - due 90 days after the close of the grant.

• Grant products
• Financial report
These forms are available at
http://www.imls.gov/recipients/administration.shtm

IMLS Project Reporting Timeline

Award Date

6 Month Interval(s)
3 Months Later
Project End

Proposal
Submission
Interim Report(s) Due

Final Performance
Reports
(Parts 1 & 2)
Financial Report
Grant Products*

Glossary of Terms
http://www.imls.gov/pdf/Glossary.pdf
• Activity, activities. Actions through which the objectives
and goals of a grant are accomplished and deliverables are
created.
• Outcome. A gain or change in an individual’s knowledge,
skill, attitude, behavior, status, or life condition related to
the purpose of a project. An outcome is a type of result,
usually produced through some form of learning.
• Output. A measure of quantity (e.g. number or percent) or
of quality (e.g. produced to a specified standard) of
activities, products, or services.
• Program. A connected series of steps, services, products,
and/or experiences constructed to achieve a desired result.

Outputs/Outcomes
Output examples
• Developed 5 curricula
• Conducted 4 teacher training workshops, each attended by 25
teachers (total 100 teachers trained)
• Program was delivered to 850 students

Outcome examples
• Middle school teachers showed increased interest and improved
ability in teaching local history
• Middle school students showed increased knowledge of local history
after completing the program

Interim and Final Performance Report
Interim Report
• compare actual accomplishments with goals established
• describe activities or services
• describe any significant findings or accomplishments

Final Performance Report: Part 1 & 2
•
•
•
•
•

project activities and the audiences served
an analysis of the project’s achievements and their value
summary of lessons learned
the outcomes and the larger impact
future plans

Final Performance Report: Part 2

Three Main Sections to Part 2

A. Numeric Total of Project Activities
B. Numeric Total of Grant Products
C. Numeric of Project Participants/Users/Audience
[Note: No single form can encompass the range of activities
funded in this grant program. However this form is an
important starting point.]

Project Example: Brown County
Historical Society
Performance Report Narrative: Activities List
1) Develop exhibit (includes developing a Web site, an
audio tour, and a gallery guide)
2) Develop accompanying curriculum
3) Conduct teacher training institute

A. Activity Example: Develop Exhibit
Institution Name: Brown County Historical Society
Grant #: 12 - 34 - 67 - 899
A. SITE SPECIFIC PROJECT ACTIVITY: Develop and
mount local history exhibit, Web site, gallery guide
and audio tour
4. ___1______Total # of lectures, symposia, demonstrations,
exhibits, readings, performances, concerts, broadcasts,
Webcasts, workshops, multi-media packages, or other
learning opportunities provided for the public.

B. Product Example: Develop Exhibit
Institution Name: Brown County Historical Society
Grant #: 12 - 34 - 67 - 899
B. PORTABLE PRODUCTS (relating to the activity named
in section A.): Develop and mount local history
exhibit, Web site, gallery guide and audio tour
12. ___1______Total # of Web sites developed or improved
[include URLs/addresses: www.browncountyhistory.org].
13. _2___Total # of learning resources produced [includes__
oral histories, curriculum resources, curriculums, Webbased learning tools, or _1_ other (specify): gallery guide;
__1__ other (specify): audio tour].

C. User Example: Develop Exhibit
Institution Name: Brown County Historical Society
Grant #: 12 - 34 - 67 – 899
C. PARTICIPANTS/VISITORS/USERS/AUDIENCE (relating to
the activity named in section A.)
21.__2000___Total # of users of Web-based resources provided
by your grant (include all individuals the project served). Choose
the measure that best represents your use rate) ___ visits
(hits), __unique visitors, __ pages visited, ___ registered users,
___ other measure (specify).
22. _3500__Total # of individuals benefiting from your grant
(include all those from questions 18-21 plus others the project
served). Only include those who actually participated or used
your project services in some way.

First Question and Answer Period

Question guidelines:
– Please try to keep questions somewhat general
(i.e., relevant to other grantees)
– Detailed questions can be directed to your program
officer after the session or at a later date.

Evaluation and Resources

a) Overview of evaluation concepts
b) Links to evaluation resources

Let’s Take a Step Back

Apply
• Establish an
evaluation
plan

Implement
• Collect data,
conduct
evaluation

Report
• Report
outcomes to
agency,
funders
(narrative,
quantitative)

Disseminate
• Share
outcomes
with your
institution,
media, peers

What is evaluation?
A process that tries to determine the effectiveness
and results of specific program activities. (We do
what? For whom? For what outcomes?)
Evaluation helps you:
• measure the impact of your program
• show what a difference it made on the people
you served
• know that your program is on track

Where does evaluation fit in to my
project?
• front-end evaluation show what has audience appeal and
what makes for effective outreach.
• formative evaluation tells you which ways of
communicating information work best for your audiences.
• outcome evaluation helps you know (and show) that your
program creates intended results.
• remedial evaluation identifies what needs upkeep, repair,
or replacement.
• summary evaluation is done at the end of a project,
rather than during it

How do I know what evaluation
framework is right for my project?

– Web-based resources/curriculum.
– Evaluation guidebooks.
– IMLS program and research staff.

Web-based resource
www. shapingoutcomes.org

Logic model development

Evaluation Guidebooks

• Framework for Evaluating Impacts of Informal Science
Education Projects
– Edited volume covering a wide range of evaluation methods for
evaluating informal learning
– National Science Foundation

• Key Steps in Outcome Management Series
– 6 book series on different aspects of outcome evaluation
– The Urban Institute

• Introduction to Museum Evaluation
– American Association of Museums

IMLS Program and Research Staff
Program Staff:
Sandra Narva, Senior Program Officer
Phone: 202/653-4634 E-mail: [email protected]
Steven Shwartzman, Senior Program Officer
Phone: 202/653-4641 E-mail: [email protected]
Reagan Moore, Program Specialist
Phone: 202/653-4637 E-mail: [email protected]
Robert Trio, Program Specialist
Phone: 202/653-4689 E-mail: [email protected]

Research Staff:
Mary Downs, Research Officer
Phone: 202/653-4682 E-mail: [email protected]
Carlos Manjarrez, Associate Deputy Director for Research and Statistics
Phone: 202/653-4671 E-mail: [email protected]

Remember Evaluation……

…. should be designed to meet your project needs.
…. can be a do-it-yourself exercise.
…. may cover just one part of a broader project.
…. doesn’t require a great deal of resources.

Final Question and Answer Period