Form AAHC 1 AAHC 1 AAHC

Appendices
Contents
Appendices.................................................................................................................................................... 1
Appendix A – Respondent Contact Letters ............................................................................................... 2
Letters to grantees ................................................................................................................................ 2
Letters to other respondents (applicants, funders, and other key stakeholders) ................................ 3
Appendix B – Grantee Survey Instrument ................................................................................................ 4
Invitation Letter .................................................................................................................................... 4
Landing page/consent language ........................................................................................................... 4
Questionnaire ....................................................................................................................................... 6
Appendix C – Nonapplicant Survey Instrument ...................................................................................... 11
Invitation Letter .................................................................................................................................. 11
Landing page/consent language ......................................................................................................... 11
Questionnaire ..................................................................................................................................... 12
Appendix D – Semi-Structured Interview Guides ................................................................................... 14
Interview invitation emails ................................................................................................................. 14
Introductory and Consent Language................................................................................................... 15
Questions for applicants ..................................................................................................................... 17
Questions for grantees........................................................................................................................ 19
Questions for funders ......................................................................................................................... 26
Questions for other stakeholders ....................................................................................................... 28
Questions for IMLS staff...................................................................................................................... 30
Appendix E – AAHC Program Logic Model .............................................................................................. 32
Appendix F. Burden Estimates ............................................................................................................... 33
Appendix G. Cross Walk of Research Questions to Data Sources......................................................... 335

Appendix A – Respondent Contact Letters
This letter will be sent by IMLS on behalf of the evaluation to respondents, informing them of the purpose
of the evaluation, the Urban Institute’s role, and the forthcoming request for data collection (survey or
survey and interview) from the Urban Institute. This letter will be customized to the respondent type
(pre-2014 grantee, 2014-2019 grantee, and repeat grantee; applicants; funders; and other stakeholders).
This letter will be sent approximately 1 week before Urban plans to contact the stakeholder for data
collection (survey and interviews for Cohort 2 grantees and interviews only for other stakeholders). This
letter will not be sent to nonapplicants because they do not have a pre-existing connection to the AAHC
program – this group will instead just receive an introductory email from the Urban Institute (see
Appendix C).

Letters to grantees
To:

[Email address for identified contact]

From:

IMLS

Subject: Requesting your assistance in a review of the African American History and Culture Grant
Program

Dear {First Name},
As a grantee of the Institute for Museum and Library Services (IMLS)’s African American History and
Culture (AAHC) program, I’m inviting you to participate in an evaluation of the AAHC program’s
performance.
First, I want to assure you that this is not an evaluation of your project grant award, organization, or
institution. We are only interested in learning about your experience with the AAHC program.
The goals of this evaluation are to better understand your perspectives on the AAHC program, its
contributions to your organization’s outcomes, and its overall success in meeting its legislative objectives
to nurture museum professionals, build institutional capacity, and increase access to museum and archival
collections at African American museums and Historically Black Colleges and Universities (HBCUs).
IMLS has contracted with the Urban Institute, a national nonprofit research organization, to conduct this
evaluation. In the coming weeks, researchers from the Urban Institute will be reaching out to you to
collect your views. Your participation is entirely voluntary and will have no impact on your relationship
with the AAHC program, IMLS, or the Federal government. Any information you provide to the Urban
Institute will be reported to IMLS the evaluation in aggregate form only. No individual responses will be
reported.
We hope you are willing to participate in this effort as it will enable us to better understand the
performance of the AAHC program and help improve this program for future grantees.
If you have any questions or concerns about this evaluation, please contact Claire Boyd at the Urban
Institute ([email protected]) or Mark Isaksen of IMLS ([email protected]).

2

On behalf of the IMLS AAHC team, thank you in advance for taking the time to share your knowledge and
insights with us.
Best,

Letters to other respondents (applicants, funders, and other key stakeholders)
To:

[Email address for identified contact]

From:

IMLS

Subject: Requesting your assistance in a review of the African American History and Culture Grant
Program

Dear {First Name},
As leader in the field of preserving and celebrating African American history and culture, I would like to
invite you to participate in an evaluation of the African American History and Culture (AAHC) grant
program of the Institute for Museum and Library Services (IMLS).
The goals of this evaluation are to better understand your perspectives on the AAHC program, its
contributions to the field, and its overall success in meeting its legislative objectives to nurture museum
professionals, build institutional capacity, and increase access to museum and archival collections at
African American museums and Historically Black Colleges and Universities (HBCUs).
IMLS has contracted with the Urban Institute, a national nonprofit research organization, to conduct this
evaluation. In the coming weeks, researchers from the Urban Institute will be reaching out to you to
collect your views. Your participation is entirely voluntary and will have no impact on your relationship
with the AAHC program, IMLS, or the Federal government. Any information you provide to the Urban
Institute will be reported to IMLS the evaluation in aggregate form only. No individual responses will be
reported.
We hope you are willing to participate in this effort as it will enable us to better understand the
performance of the AAHC program and help improve this program for future grantees.
If you have any questions or concerns about this evaluation, please contact Claire Boyd at the Urban
Institute ([email protected]) or Mark Isaksen of IMLS ([email protected]).
On behalf of the IMLS AAHC team, thank you in advance for taking the time to share your knowledge and
insights with us.
Best,

3

Appendix B – Grantee Survey Instrument
Invitation Letter

Subject: We would like to hear from you: Your Perspective on the African American History and Culture
Grant Program
Dear {First Name},
As you may be aware, the Institute of Museum and Library Sciences (IMLS) has contracted the Urban
Institute, a nonprofit research organization, to examine the African American History and Culture grant
program (AAHC). This examination will help IMLS, policymakers, African American history and culture
museums, and other stakeholders identify ways to improve the program.

Because your organization has received one or more grants through the AAHC program, we’re
interested in your views on the program and are inviting you to share your views through a survey. Your
responses will help us understand 1) your perceptions on the AAHC program, 2) how the program
contributed to your organization’s goals, and 3) opportunities to strengthen the program.
This survey should take no more than 30 minutes to complete and you do not need to complete the
entire survey in one sitting. Your perspective is greatly appreciated and will help improve the AAHC
program so that it better supports organizations like yours. To access the survey, please click this link:
XXXX
If you have questions about the validity of this request, please contact Mark Isaksen at IMLS
([email protected]). If you encounter any problems filling out this survey, feel free to contact Claire Boyd
at the Urban Institute ([email protected]).
On behalf of the AAHC evaluation team, thank you in advance for taking the time to share your
knowledge and insights with us.
Best,

Landing page/consent language
Thank you for considering participating in this survey.
Your participation is entirely voluntary and you may choose to stop the survey at any time or skip
questions for any reason. Your responses will only be reported in group form to protect individual
identities. Your decision to participate or not will have no impact on the success of future applications to
AAHC or other IMLS programs. The information you provide will be kept private to the extent permitted
by law.
This survey is designed to take no more than 30 minutes to complete. You are not required to complete
the survey in one session and you can save and return to your progress.
We want to be sure that you freely consent to participate in this interview and are aware that you are
not obligated to answer any questions you do not wish to. Do you consent to participate in this survey?
 Yes
o

We greatly appreciate your participation in this survey!

4

o

If you are unable to complete the survey in one session, your answers will be saved
and you can return to where you left off by clicking the original link provided. You
may wish to keep a copy of your completed grant application or other grant
materials at hand as you complete this survey, however you will be given the option
to respond “Not sure” or “Don’t know” to any question.

o

Please complete the survey by TBD. If you have technical difficulties or questions
when completing this survey, please contact Claire Boyd at
[email protected] or 202-261-5713.

o

Thank you for your time. You may now exit this screen.

 No
 Not sure; I have questions or concerns.
o

If you have questions or concerns before deciding to participate, please contact
Claire Boyd at the Urban Institute ([email protected] or 202-261-5713)

5

Questionnaire
1. Are you knowledgeable with the IMLS AAHC grant(s) your organization received and the work
that it funded? (If your organization has received more than one, please focus on your most
recent grant award)
a. Yes [Proceeds to survey]
b. No
i. Is there someone else at your organization who is familiar with the grant and
the work that it funded?
1. Yes
a. Are they able to complete this survey?
i. Yes
1. Thank you for your time. Please exit this survey
and ask the person at your organization who is
familiar with the grant to complete the survey
using the link in the invitation email or please
kindly provide us with their contact information
and we’ll send them a link.
a. [Space to write-in]
ii. No
1. Thank you for your time. Please exit this survey
2. No
a. Thank you for your time. Please exit this survey
2. Please indicate how much you agree or disagree with the following statements based on your
experience with your AAHC grant:
Strongly
disagree

Disagree

Neither

Agree

Strongly
agree

a. The AAHC application process was clear
b. We had access to beneficial training and knowledge
development resources (e.g., webinars) through the
AAHC program
c. The AAHC program helped us create meaningful
connections with other grantees
d. The grant’s requirements (such as reporting
requirements and any applicable cost-share matches)
were clear

6

Not sure /
not
applicable

e. The grant’s requirements (such as reporting
requirements and any applicable cost-share matches)
did not represent a significant burden on our
organization
f. The AAHC program provided useful advice and
guidance on measuring and reporting our project’s
results
g. IMLS staff were responsive and helpful in
addressing our organization’s questions or concerns
h. IMLS seems to understand the challenges faced by
organizations such as ours

3. The goals of the AAHC program are to nurture museum professionals, build institutional
capacity, and increase access to museum and archival collections at African American museums
and HBCUs. Are there any activities or projects you want to do that are consistent with these
goals but ineligible for AAHC funding?
a. Yes
i. [please describe]
b. No
c. Not sure
4. Reflecting on the goals of your project, to what extent has the AAHC grant:
Not at all

Very
little

Somewhat

To a great
extent

Not
applicable
to the
goals of
our
project

a. Improved the professional development of your
personnel (staff, volunteers, interns)?
b. Enabled you to purchase or implement new
management tools (e.g., project management,
collections management, or accounting software)?
c. Made your collection more useful to researchers?
d. Made preservation of your collection more
secure for future use?
e. Improved or expanded your collection?
f. Increased the physical number of visitors to your
organization?
g. Increased the number of visitors to your
website?
h. Improved your website?

7

Too early
to tell

i. Improved your ability to sustain this work beyond
the grant period?
j. Deepened the level of engagement with your
audiences?

[4a. If respondent answered “Not at All” or “To a great extent” (the extreme response categories) to one
or more of the above items, they will be prompted to provide more detail as to why they felt that way.]
5. Have you attended an IMLS convening for AAHC grantees?
a. Yes, and found it very useful
b. Yes, and found it somewhat useful
c. Yes, and found it not useful or only slightly useful.
i. Please explain why.
d. No, but someone else from my organization did.
e. No, no one from my organization attended
i. Why not?
1. Did not have time to attend
2. Insufficient funding to attend
3. Other personal or professional commitments
4. Other
a. [Please describe]
f.

Not sure/don’t know.

6. Does your organization have a formal process for tracking your programs’ progress and results
(for example, procedures for collecting and storing data, calculating performance metrics,
and/or analyzing and reporting the data (e.g., Excel)?
a. Yes, one that provides reliable, timely, and accurate information
b. Yes, one that is pretty good but needs some improvements
c. Yes, but it provides information that is limited, unreliable, and/or delayed
d. No, but we’re building one now or have plans to do so
e. No and no immediate plans to build one
i. Why not? (select all that apply)
1. Insufficient funding
2. Insufficient personnel time or expertise
3. Limited interest
8

4. Unclear use or value
5. Other priorities are more important
f.

Not sure/don’t know

7. Did your organization measure and report on results from your project funded by the AAHC
grant?
a. Yes
i. How has your organization used data on these results?
1. [Write in]
b. No
c. Don’t know/not sure
8. In your opinion, how important to IMLS was measuring and reporting on project results? [Scale
response (1 = Not at all important; 7 = Extremely important)]
9. In your opinion, how could IMLS use project results more effectively?
a. [Write-in]
10. Were you able to use the AAHC grant to attract additional funding for your organization?
g. Yes
i. [Please describe]
h. No
i.

Not yet but we expect to

j.

Not sure

11. If AAHC grant funds had not been available, would you have been able to implement your
project?
a. Yes, exactly as planned
b. Yes, but on a different (delayed) timeline and/or with modifications to scope
c. No
d. Not sure
12. Are you aware of other sources that provide funding for similar purposes as the AAHC grant?
a. Yes
a. [Please describe organizations or programs you’re aware of that might provide
funding in a similar capacity]
b. No, I’m only aware of the AAHC grant
c. Not sure

9

13. What suggestions do you have for IMLS to improve the AAHC grant program so that it better
supports organizations such as yours?
g. [Space to describe]

Thank you!
Thank you for your time completing this survey.
14. Would you be willing to provide your contact information in case the evaluation team has follow
up questions?
a. No, thank you.
i.

Thank you for your time.

b. Yes
a. Thank you. Please enter your contact information:
i. [Name]
ii. [Phone number]
iii. [Email]
iv. [Organization]

10

Appendix C – Nonapplicant Survey Instrument
Invitation Letter

Subject: We would like to hear from you: Your Perspective on a Grant Program for African American
History and Culture
Dear {First Name},
On behalf of the Institute of Museum and Library Services (IMLS), an independent agency of the United
States Federal government, we would like to invite you to complete a short, five question survey about
the African American History and Culture grant program (AAHC).
The AAHC program supports projects that nurture museum professionals, build institutional capacity, and
increase access to museum and archival collections at African American museums and Historically Black
Colleges and Universities. For more information on AAHC, visit:
https://www.imls.gov/grants/available/museum-grants-african-american-history-and-culture
IMLS has partnered with the Urban Institute, an independent, nonprofit research organization in
Washington, D.C., to administer the survey. This short survey should take fewer than five minutes to
complete and will help improve the program so that it better supports organizations like yours. Any
information you provide will only be used in aggregate form.
Please click here to access the survey: TBD
The survey should take no longer than 5 minutes of your time, and we ask that you submit your responses
by TBD.
If you have any questions, feel free to contact Claire Boyd ([email protected]).
On behalf of the evaluation team, thank you in advance for taking the time to share your knowledge and
insights with us.
Best,

Landing page/consent language
Thank you for considering participating in this short survey on the African American History and Culture
grant program.
Your participation is entirely voluntary. Your responses will only be reported in aggregate (no individual
responses will be reported) and will be kept private to the extent permitted by law. Your decision to
participate or not will have no impact on the success of future applications to the African American
History and Culture grant or other Institute of Museum and Library Services programs.
We want to be sure that you freely consent to participate in this interview and that you are aware that
you are not obligated to answer any questions you do not wish to. Do you consent to participate in this
survey?
 Yes
11

o

We greatly appreciate your willingness to complete this short survey!

o

Thank you for your time. You may now exit this screen.

 No

Questionnaire
1. Before today, have you heard of the Institute for Museum and Library Services (IMLS)?
o Yes
o No
o Not sure
2. Before today, have you ever heard of the African American History and Culture (AAHC) grant
program of the IMLS?
o Yes
o No
o Not sure
3. To the best of our knowledge, has your current organization ever applied to the AAHC program
before?
o Yes, my organization has applied at least once before
 What year(s)?
• [Enter]
• Not sure
o No, I do not believe we have every applied
 Why not? (Select all that apply)
• I don’t know
• We were not aware of this program
• We didn’t have time or staff to apply
• We did not need this funding
• The cost-share requirement was a barrier
• The program’s other requirements were beyond our capacity to meet
• Our organization’s needs didn’t align with the program’s goals
• The available funding wasn’t large enough for the project(s) we wanted
to fund
• We heard that IMLS was difficult to work with
• We did not want to work with the Federal government
• We didn’t meet the program’s eligibility requirements
• Other
o [Please describe]
o I don’t know if we have ever applied
o
4. Does your organization regularly collect the following data? [For each option, respondents can select
"yes", "no", or “not sure”]
o the number of visitors to your museum
o the number of visits to your museum's website
o the condition of your collection

12

o

visitor feedback about the quality of their experience with their museum/collection

5. Which of the following types of funders have provided support that is vital to your organization’s
financial sustainability? (Yes/No, select all that apply)
o Local or state government
o Federal government
o Philanthropic foundation
o Individual donor
o Corporate donor
o Other (Please describe)
o
Thank you for taking the time to answer this survey and provide feedback. The information you provided
will be used to help improve a program serving organizations such as yours. For more information about
the African American History and Culture grant program, please navigate to:
https://www.imls.gov/grants/available/museum-grants-african-american-history-and-culture

13

Appendix D – Semi-Structured Interview Guides
This section provides (a) the invitation emails that will be sent by the Urban Institute, inviting
respondents to participate (see Appendix A for Respondent Contact Letters from IMLS which will
precede this outreach); (b) introductory and informed consent language that will be used by the Urban
Institute at the start of every interview; and (c) questionnaires for each respondent group.

Interview invitation emails

Subject: We would like to speak with you: An Evaluation of the African American History and Culture
Grant Program
Dear {First Name},
I am part of a team of researchers from the Urban Institute working with the Institute for Museum and
Library Services (IMLS) to learn more about the performance of their African American History and
Culture grant program (AAHC), including [the application process; its contributions to the outcomes of
grantees such as you; its contributions to the wider field]. This effort aims to understand how the AAHC

program is meeting its objectives to nurture museum professionals, build institutional capacity, and
increase access to museum and archival collections at African American museums and Historically Black
Colleges and Universities.
We would like to interview you…

•

to learn more about your views as an applicant who understands the application process and are
aware of the challenges facing organizations like your own. We believe your perspective will be
invaluable in informing how to improve the application process and better understand barriers for
prospective applicants.

•

to learn more about your views as a grantee about what has worked and what could be improved
relative to the AAHC program. Specifically, we are interested in learning more about your
experiences in the application process, the trajectory of the project/organization, any specific
capacity challenges you may have, and contributions the AAHC has made to your project or
organization.

•

to learn more about your views as a funder on the landscape of organizations in this space, the
challenges that they face, and how targeted grants may best support them.

•

to learn more about your views as a key stakeholder and thought leader in this field who is
knowledgeable about the landscape of organizations in this space, the challenges that they face,
and the potential for programs like the AAHC to help.

This interview is entirely voluntary and, should you choose to participate, will last no longer than one hour.
We will be conducting interviews between April 21, 2020 and May 29, 2020 at a time convenient to you. If
you are willing to participate, please indicate your preferred interview time by responding to this Doodle
poll (in which you should please be sure to also record your name, where prompted): XXX OR by responding
to this email with preferred dates and times. Once we confirm a date and time, we will send you a calendar
invitation to confirm the appointment, along with credentials for dialing in to the conference line that we
will use for the call.
Or, if there is someone else at your organization (or at another grantee organization) that you recommend
we interview instead of you (or in addition to you), we would be grateful for their name and contact
information

14

If you have any questions, please feel free to reply to this email or contact my colleague Claire Boyd at
[email protected]
Thank you again for considering this interview request.
Best,

Introductory and Consent Language
The following language will precede each interview.
The Institute for Museum and Library Services has asked the Urban Institute (a non-profit, nonpartisan
research organization in Washington, DC) to conduct an evaluation of the African American History and
Culture (AAHC) grant program. This evaluation is designed to collect information on the program’s reach
and its role in supporting grantee outcomes. This is an evaluation of the AAHC program itself, not an
evaluation or audit of grantee outcomes. The goal is to better understand how the AAHC program is
meeting its objectives to strengthen the capacity of grantees and to help identify ways to improve the
program, as necessary.
Your feedback is extremely important to the success of this evaluation because as:
•

a grantee, you/your organization have experience navigating the application process,
implementing an AAHC grant, and observing and reporting outcomes related to that grant.

•

an applicant, you/your organization have experience navigating the application process and are
aware of some of the challenges facing organizations like your own.

•

another funder in this space, you/your organization are knowledgeable about many of the
eligible organizations in this space, the challenges that they face, and how targeted grants may
best support them.

•

a key stakeholder, you/your organization are knowledgeable about many of the eligible
organizations in this space, the challenges that they face, and the potential for programs like the
AAHC to help

•

an IMLS staff member, you have experience navigating and shaping AAHC program goals and
operations that inform the selection and success of the grantees and their project goals.

This interview is completely voluntary. You may terminate it at any point for any reason and you may
decline to answer any question. Your responses will be used to inform our understanding of the AAHC
grant program and its role in supporting grantee outcomes, the landscape of African American museums
and collections at Historically Black Colleges and Universities, and opportunities for further attention by
IMLS and others.
We will be taking notes during this interview and would like to record the interview to ensure we are
accurately capturing the information you provide. The notes and the recording will only be accessed by
our research team at the Urban Institute – all of whom have signed confidentiality pledges. This data will
not be shared with IMLS except in aggregate/group form.
Your decision to participate or not will have no effect on your interactions with the Federal government,
including IMLS and the AAHC program.
15

We cannot guarantee confidentiality because our sample is small and IMLS knows the full list of those
we are interviewing. However, we will not quote anything you say unless we receive your expressed,
written consent.
We want to be sure that you freely consent to participate in this interview and that you are aware that
you are not obligated to answer any questions you do not wish to. Do you consent to participate in the
interview?
Do you consent to this interview being recorded [not a requirement to continue]?

16

Questions for applicants
[Note: these respondents will be selected from eligible organizations that have applied to the AAHC
program at least once, but which have never received funding through the program. Outreach to
organization will request we speak with someone involved with/familiar with their AAHC application(s).]
[These questions are semi-structured and tailored to the respondent and the flow of the interview. Not
all questions will be asked of all respondents.]
•

Please describe your name, title, and background including how long you’ve been at your
organization.

•

Your organization applied for AAHC funding in [2XXX] – were you involved in that application?
o

[If no, ask if there is someone at their organization who was involved and then move on
to questions on needs and other funding]

o

[If yes and can remember the process]:


How did you become aware of the AAHC program?



Why did you apply?



Did the timing of the application deadline affect the time and resources your
organization committed to completing this application? [Prompt: If the deadline
was at a different time, do you think that your organization would have
dedicated more/less time and resources to this application process? Did this
application deadline align with gaps in organizational funding?]



What are your recollections of the application process? [Use completed
application on hand to prompt specific reflections]
•



If you were to estimate, about how many hours did the application take
to complete?

Did you receive feedback on your application?
•

[If yes] was this information useful?

•

[If no] would you have liked to? What kind of information would have
been beneficial?

•

Did you apply again? Why or why not?
o

[If yes] If you applied multiple times, did you change anything
about your application to become more competitive?

o

Was there a certain threshold after not receiving a grant that
made applying to this program no longer worth the resources to
produce an application?

17

•

After being informed of the determination on your AAHC application, did you pursue and/or
receive funding for the same project through another source?

•

The IMLS’s AAHC grant program is designed to enhance grantee capacities – are there specific
capacities in your organization that you believe a grant like this could help enhance?
o

•

•

[Prompt as needed with examples of capacities]

What are primary sources of your funding?
o

What’s the relative weight of these sources? [Prompt: for example, what percentage
comes from federal vs. state or local government vs. philanthropic grants vs. donations,
etc.]

o

How do you use these different funding sources? [Prompt: is one designed for capacity
building? Another for development?)

To your knowledge, has your organization received funding through any other IMLS grant
programs in recent years (since 2014)? [Prompt: list other IMLS grants: Museums for America,
National Leadership Grants, Inspire Grants for Small Museums, Collections Assessment for
Preservation, Community Catalyst Initiative, Museum Assessment Program, Museums
Empowered]
o

[If yes], how would you describe your experience with those programs including their
application process (and how it differed from AAHC) and reporting requirements?


o

Was your experience with the AAHC program helpful to you in applying to these
other grant programs?

[If no] have you applied?


[if no] why have you not applied?

Concluding
•

Is there anything you would like to add that we didn’t cover already?
o

[Prompt: Any final thoughts or reflections?]

Thank you for taking the time to speak with us and offer your thoughts and perspectives on the African
American History and Culture grant program and how it relates to your work. As we explained earlier,
your, much appreciated, contribution will help inform our broader evaluation of this program’s
performance and support IMLS’s efforts to improve the program and effectively communicate the
program’s contributions to the field.
The evaluation findings will be published after a final draft is shared with IMLS in the fall. Please do not
hesitate to reach out if you have questions after this call, about the interview or the broader evaluation,
or to provide additional information that may be relevant.
Thank you again for your time!
18

Questions for grantees
[These questions are semi-structured and will be tailored based on grantee cohort, responses to the
grantee survey, and other available information (e.g., grant reports). Not all questions (only a relevant
set) will be asked of each respondent, with their background and the pace of the interview dictating
actual coverage.]
AAHC’s Role in Building Project-Specific Capacities Among Grantees
The first set of questions are about how the AAHC grant contributed to your efforts to build individual
staff members capacities as well as your organization as a whole.
Leadership & Staffing
1. Did the AAHC grant enable your organization to hire, train, or retain staff? If so, in what
roles or positions and/or what kinds of professional development did your staff receive or
participate in?
a. What impact did the creation of these new positions have on your operations?
2. Did you or others at your organization take advantage of trainings or webinars offered by
IMLS?
a. How effective were these trainings? What other support would your staff benefit
from?
Organizational Structures and Systems
3. Did the AAHC grant enable you to improve operations? [Prompt: For example, by purchasing
new equipment, improving existing or purchasing new technology, and/or improving
software?]
a. Did the AAHC grant enable you to address any other physical or technological
infrastructure needs?
4. Did your organization adopt any new policies or procedures over the course of the grant
term that helped to improve organizational performance and management?
5. During the grant term, did you develop or update a strategic plan as part of your grant
activities? [Prompt: While IMLS does not provide direct support or resources for this, a
grantee may choose to do this using resources and/or consultants from other service
providers]
Stewardship and Quality of Collection
6. Did your organization use AAHC program funds to invest in proper cataloguing frameworks,
the development/refinement of acquisitions policy or other processes that ensured the
stewardship and quality of the collection?
a. If so, what impact did the creation of these new systems have on your organization?
Accessibility of Collection
19

7. During and after receiving the grant, did you use new software or processes to manage
museum collections (e.g. cataloging a collection, digitization of a collection, providing access
to a collection using digital tools), track attendance, communicate internally with staff
members, communicate with external partners, etc.?
a. If so, what impact did this work have on your organization?
AAHC’s Role in Building General Capacities Among Grantees
Next, I’d like to ask you about steps you may have taken to develop new partnerships or join
professional associations or initiatives.
Social Capital Capacities
8. Over the course of the grant term, did you establish any new partnerships or strengthen
existing partnerships?
a. What was the nature of these partnerships?
b. Were these partnerships related to the grant or the grant-funded project?
c. Did you develop any partnerships with other AAHC grantees and did the program
help you create those?
9. In what ways has the AAHC grant enabled your institution to innovate or adapt best
practices from other museums?
10. Since you received the AAHC grant, has your organization joined or renewed membership in
professional service organization such as the Association of African American Museums
(AAAM)?
d. Or pursued or achieved accreditation with the American Alliance of Museum
(AAM)?
e. Or started working towards STEPS with the American Association of State and Local
History (AASLH)?
f.

Or engaged in any other museum professionalization efforts to help develop your
organization’s capacity?

11. To the best of your knowledge, have you or any other personnel (staff, volunteers, or
interns) joined any professional affiliation since your organization received the AAHC grant?
Leveraging New Capital Capacities
Next, we’d like to learn a little bit more about how the AAHC grant may have contributed to the financial
health and stability of grantees.
12. To what extent did the AAHC grant help your organization leverage additional funding (e.g.
new grant dollars, one-time contributions, responding to requests/applications to other
federal grant programs). Can you point to specific examples, programs, or initiatives?

20

a. Have/are your organization’s efforts to secure additional funding successful? Why or
why not?
b. Has your organization leveraged your designation as an AAHC grant recipient to
communicate with other funders?
AAHC’s Role in Supporting Exhibits and Public Programs & the Visibility and Recognition of Grantees
IMLS wants to make sure that its grantees have the support they need to advance exhibits and public
programs, and their visibility and recognition. These next questions speak to that.
13. Did your organization plan to or install new exhibits, undertake new outreach and/or public
programs, or add to or make accessible new collections? If so, please describe the exhibits
and/or programs?
14. Was your organization able to publish new scholarly publications, exhibition brochures or
catalogues, that may later be used to promote your organization or institution?
a. In what ways do you disseminate these materials?
15. Since you received the AAHC grant funding, has awareness of your organization or
institution increased among any specific groups (e.g. youth, seniors, students of all ages,
and/or other populations of interest)?
16. How has the museum attendance changed before and after receiving the AAHC grant?
a. What other changes in engagement has your organization noticed?
a. Have you tracked any of these? (e.g. outreach, community participation, webbased resource downloads, web traffic)
b. What would you need to improve or expand these efforts?
17. Since you received the AAHC grant funding, have you updated and/or enhanced your
organization’s website or increased your social media presence?
18. Has your organization or institution pursued or received accreditation? If so, did the
capacity building support from AAHC help you pursue this goal?
AAHC Application Process & Grantee Experience
19. How would you characterize the application process? Were the steps associated with the
process easy to understand and complete? Did you use any of the technical assistance
provided by the IMLS staff (e.g. webinars, application completion supports, etc.)?
a. If you were to estimate, about how many hours did the application take to
complete?
b. Did you have any difficulty meeting the cost share requirement? How did you go
about securing the cost share requirement?
20. What was/has it been like working with IMLS staff over the grant term? [Prompt: If you had
any questions, has the program team been responsive? Did you feel comfortable
21

approaching the team if you had any questions or needed more time to complete your
project?]
21. Has your organization encountered any challenges associated with IMLS’s administrative
practices? [Prompt: By administrative practices we mean how IMLS is managing the entire
grant process including the application, grant management and disbursement,
responsiveness to questions, trainings and advice for grantees, etc.]
a. What recommendations would you suggest to the IMLS to improve their management
of the AAHC program to improve grantee experience and/or the implementation of the
award?
b. Did the timing of your grant award (e.g. month of award notification, date of payments,
grant cycle, etc.) align with the needs of your organization and the project?
i. If not, how could IMLS improve this aspect of the administration of the
program?
AAHC’s Role in Outreach and Engagement with African American Museums and HBCUs and
Understanding the Universe of Prospective Applicants (Museums and HBCUs)
IMLS is interested in learning more about how effective their outreach has been and wants to gain more
insight into the broader universe of African American museums and HBCUs, in part, to understand some
of the perceptions about the AAHC program and potential barriers that might prevent new organizations
and institutions from applying to the AAHC program.
22. How did you learn about the AAHC program?
23. Based on your knowledge of the AAHC program, how could IMLS staff expand and/or
strengthen its existing outreach?
24. What strategies would you recommend to IMLS staff to tap a broader network of
community-based organizations, museums, and/or HBCUs to attract new and returning
applicants?
25. What barriers do you perceive current or prospective applicants face in applying to this kind
of funding opportunity?
26. How could IMLS restructure the AAHC grant process to be more equitable and improve
applicants’ experience? [Prompt: Is there anything in the application that seems repetitive or
could be shorter? Could you repurpose other grant application narrative for this
application?]
27. Do you know of any other funders that support these kinds of institutions? [Prompt: Funding
sources like NEA or NEH grants, for example?]
g. Have you ever applied to one of their programs?
h. What was your experience like?

22

Cohort Specific Questions
Cohort 1 Specific Module (Pre-2014 grantees)
IMLS wants to gain cohort-specific insight from AAHC grant recipients prior to 2014 to gain insight on the
first iteration of the program and what the potential residual impacts of the grant have been over time.
•

Some time has passed since your organization was awarded the AAHC grant. Since (year the
grant was awarded), what has happened to the project?
o

If the project focused on the professional development of any staff members, do they
still work in any capacity at your organization? If not, what career opportunities have
they gone on to pursue after working at your organization?

Cohort 2 Specific Module (2014-2019 grantees)
IMLS wants to gain cohort specific insight from recent AAHC grant recipients (2014-2019) to gain insight
on the universe of African American Museums and HBCUs, the extent to which IMLS practices have
adapted to recent philanthropic trends to support African American museums and HBCUs, the role of
AAHC in enhancing grantee capacity, and the alignment between organizational needs and IMLS’
administrative practices.
•

Why do museums and HBCUs pursue funding from IMLS and what other related training do they
receive?

•

How important is capacity building for museums and HBCUs?

•

How has capacity changed at the organization level since receiving funding from IMLS/AAHC?

•

What are the key benefits of capacity building for museum staff and leadership?

•

To what extent has your organization or institution sustained capacity building efforts since the
grant term ended?

Grant Recipients (2014-2018)
•

What training or capacity building opportunities has your organization been able to take
advantage of since receiving the grant funding from AAHC?

•

How do you intend to continue your capacity building work at the conclusion of the grant term?

•

What other funding opportunities has your organization or institution applied to over the last 12
months?

•

Would you recommend that IMLS integrate project evaluation as a key component of the
program as a requirement for grantees?
o

Do you have any concerns if IMLS does this?

New Grant Recipients (2019 grantees)
•

How did you learn about the program (if a first-time grantee or HBCU)?

23

•

Why did you apply to the AAHC program?

•

What are your initial impressions of the AAHC program?

•

Now that you are in the implementation phase of the grant program, what supports are you
receiving from IMLS staff or members of previous grant recipient cohorts? Are there any
supports that you think would best help the success of your project that are not already being
offered by IMLS staff?

•

What other funding opportunities has your organization or institution applied to over the last
calendar year? Have you received any additional funding?

Cohort 3 (Repeat Grant Recipients)
IMLS wants to gain cohort-specific insight from organizations and institutions that have received the
AAHC award more than once across key temporal stages. This group of grantees will be able to provide
insight on the universe of prospective applicants, the extent to which IMLS and the AAHC program have
adapted to trends and evolutions in eligible African American Museums and HBCUs, the extent to which
different parts of the AAHC portfolio have performed better than others, how best to integrate program
evaluation into the program model, and how well AAHC has performed in meeting its legislative goals
and in identifying areas for improvement.
•

Your organization has used the AAHC program to build capacity over a long period of time. What
capacities have been strengthened? What evidence do you have of this improvement? What has
been the effect of these grants over time?
o

Has the grantee experience changed over time? If yes, in what ways?

•

Can you describe the kind of staff turnover your organization has experienced since you
received the first grant from AAHC?

•

Where are the staff members now who were previously associated with this program? Have any
former staff members advanced in their museum careers whether at your institution or
elsewhere?

•

Based on your knowledge of the capacity-building supports your organization has received over
the multiple awards you have received from IMLS, can you describe evidence of systems
change?

Repeat Grant Recipients who have been awarded other IMLS grants
•

[Our records suggest/as you’ve indicated] your organization has been awarded grants from
another/other IMLS program(s). What motivated your organization’s decision to apply to the (X)
award? Did you receive any support from the IMLS staff in pursuing this additional source of
funding?

•

How would you compare the application and implementation stages of the AAHC program with
the (X) grant program?

24

•

How does your experience with the IMLS staff that manages the AAHC award compare to your
experience with the (X) program staff?

Concluding
•

Is there anything you would like to add that we didn’t cover already?
o

[Prompt: Any final thoughts or reflections?]

Thank you for taking the time to speak with us and offer your thoughts and perspectives on the African
American History and Culture grant program and how it relates to your work. As we explained earlier,
your, much appreciated, contribution will help inform our broader evaluation of this program’s
performance and support IMLS’s efforts to improve the program and effectively communicate the
program’s contributions to the field.
The evaluation findings will be published after a final draft is shared with IMLS in the fall. Please do not
hesitate to reach out if you have questions after this call, about the interview or the broader evaluation,
or to provide additional information that may be relevant.
Thank you again for your time!

25

Questions for funders
[These questions are semi-structured and tailored to the respondent and the flow of the interview. Not
all questions will be asked of all respondents.]
•

Please describe your name, title, and background including how long you’ve been at your
organization.

•

What are your organization’s overall funding objectives and approaches to funding?
o

•

[Prompt: Does your organization focus primarily on the same organizations who might
qualify for the AAHC (provide eligibility criteria)? Do you provide multi-year grants? Is it
through a competitive process? Is it for specific projects or general operational support?]

What are your priorities vis-à-vis African American history and culture?
o

What kinds of proposals or applications does your program or organization seek to fund
(e.g. specific focus area of project work or organization, type of art/collection, focus,
etc.)?

o

How much funding do you have available/have you provided for these entitites
annually? What is your typical grant size? [Prompt: Estimates and ranges are fine]

o

What are the main eligibility criteria that you rely on when narrowing your applicant
pool? [Prompt: Do you require a cost share? Do you require organizations to have a
budget over a certain amount or have a at least one paid staff member?]

o

What is your grant application like (e.g. required length, two-stage application, etc.)?
[Prompt: Are there any distinguishing factors about the type of procedure your
organization goes through in selecting projects to fund?]

o

How many organizations have you provided funding to?

o

How do you think about outcomes in this space?


•

•

Do you require funded organizations to identify and track outcomes? If so,
please describe.

What challenges do you see African American history and culture museums and archival
collections encounter?
o

[Prompt if does not emerge in response: Are there challenges related to staff capacity,
limited access to capital, and/or fiscal sustainability?]

o

How does your funding help address these challenges?

How would you characterize the availability of funding to support African American museums
and HBCUs? [Prompt for HBCUs: We’re interested in support for museums, historic collections,
archives and libraries, and training students for museum, archives, or library careers].
o

Other than IMLS, what other significant funding sources exist for these organizations?
26

o

Are there any trends that you are aware of for funding in this space? [Prompt: For
example, trends like supporting paid internship programs has become more popular
amongst funders in this space.]


o
•

Are there any blindspots or funding gaps that need attention?

What’s your awareness of IMLS’s funding for these organizations?
o

•

If so, what’s driving these trends?

How do you see your funding in relation to that provided by IMLS?

Are you aware of IMLS’s African American History and Culture grant program?
o

How would you describe the AAHC program, its role in the space, and the outcomes that
it supports?

o

Are you aware of any weaknesses or areas for growth for the AAHC program? [Prompt:
What are some lessons learned from your experience funding in this space that might be
applicable to the AAHC program staff?]

o

Have your grantees blended or strategically sequenced funding from your organization
and AAHC, to your knowledge?

Concluding
•

Is there anything you would like to add that we didn’t cover already?
o

[Prompt: Any final thoughts or reflections?]

Thank you for taking the time to speak with us and offer your thoughts and perspectives on the African
American History and Culture grant program and how it relates to your work. As we explained earlier,
your, much appreciated, contribution will help inform our broader evaluation of this program’s
performance and support IMLS’s efforts to improve the program and effectively communicate the
program’s contributions to the field.
The evaluation findings will be published after a final draft is shared with IMLS in the fall. Please do not
hesitate to reach out if you have questions after this call, about the interview or the broader evaluation,
or to provide additional information that may be relevant.
Thank you again for your time!

27

Questions for other stakeholders
[These questions are semi-structured. Stakeholders are identified as experts and leaders in this field who
are not being interviewed as grantees, funders, or applicants. These stakeholders may include professors,
AAHC grant applicant reviewers, NMAAHC staff, and others. The list will be identified jointly between
Urban and IMLS.]

Introduction/landscape
•

Please describe your title, organization, how long you’ve been there and your background prior?

•

From your perspective, what are the main challenges faced by African American museums and
collections at HBCUs?
o

What capacity constraints have you seen and observed?

o

What are the main funding sources?

o

What types of funding do they need?

•

What more can be done to strengthen and expand the influence of African American historical
and cultural institutions?

•

How would you segment these organizations? What are distinguishing characteristics in this
space? [Prompt: by size, geographic focus, age, accreditation status, etc.?]

•

Are there specific types of organizations that you feel have been overlooked by the AAHC
program or funders in this space more generally?

AAHC Program
•

Other than the AAHC program, do you have any past experience with IMLS?

•

Please describe your knowledge of or past experience with the AAHC program?
o

[Prompt: The AAHC program is designed to XXX. It does this by providing grants and
offering trainings and other supports like opportunities to network with other grantees]

•

What would you say are the program’s strengths?

•

How would you describe the impact of this award on African American museums and HBCUs
and the field broadly?

•

As you understand the program, what are the weaknesses or limits of the AAHC program?

•

How might the AAHC expand its applicant pool, including hard-to-reach applicants?

•

What are some barriers that might prevent new or returning applicants from applying to the
AAHC program?

Concluding
28

•

Is there anything you would like to add that we didn’t cover already?
o

[Prompt: Any final thoughts or reflections?]

Thank you for taking the time to speak with us and offer your thoughts and perspectives on the African
American History and Culture grant program and how it relates to your work. As we explained earlier,
your, much appreciated, contribution will help inform our broader evaluation of this program’s
performance and support IMLS’s efforts to improve the program and effectively communicate the
program’s contributions to the field.
The evaluation findings will be published after a final draft is shared with IMLS in the fall. Please do not
hesitate to reach out if you have questions after this call, about the interview or the broader evaluation,
or to provide additional information that may be relevant.
Thank you again for your time!

29

Questions for IMLS staff
[These questions are semi-structured and tailored to the respondent and the flow of the interview. Not
all questions will be asked of all respondents.]
Introduction/landscape
•

How long have you been at IMLS?
o

What is your background prior to joining IMLS?

•

How long have you been involved with the AAHC program and in what capacity?

•

How would you describe the landscape of African American museums and collections at HBCUs?
o

•

What are the main challenges that they face and how does this differ from other, nonAfrican American focused, organizations?

What other funders exist in this space?

Describing AAHC and Applicants
•

How would you describe the AAHC program to a lay person?

•

How does the AAHC program interact with other IMLS programs?

•

In your ideal vision, what is the AAHC’s role and impact?

•

What types of grantees have you seen apply and get funded?
o

[Prompt: beyond measures like size and geography, are there other, more subjective
factors and characteristics that you’ve observed?]

o

What types of projects get funded?

o

What types of capacities does the AAHC program fund, in your experience?


o

Are there projects or capacities that you would expect the AAHC could/should fund that
you are surprised you haven’t seen in applicants or surprised that do not get selected by
reviewers?


•

Are there types of grantee capacities that are important but are rarely or never
funded through AAHC? [Prompt: Maybe these are capacities that grantees don’t
realize they need to develop or don’t realize the AAHC program can fund or
maybe they are capacities that are poorly aligned with the AAHC program’s
design or restrictions]

[Prompt: for example, are you aware that there is a large need for professional
trainings/certifications of curatorial staff but have seen few applications for
this?]

Are the constraints on the program – including ones that may have been placed on it by the
legislation or by IMLS’s policies and procedures – that you wish could be revised?
30

•

What types of questions do prospective grantees tend to ask before applying?

•

What are the most common weaknesses in applications that you’ve seen?

•

Has the review process for applications changed over the course of the AAHC program’s
existence? If so, how do you think these review processes has changed the makeup of the
grantee cohorts?

•

If you were to estimate, about how many hours do you think the application takes to complete?

Outcomes
•

How prominent is measurement and evaluation within the AAHC program?
o

Is it clearly prioritized to grantees/applicants?

o

What efforts has IMLS made internally to document impact?

o

Has IMLS provided tools, resources, or advice to help grantees measure and document
outcomes of AAHC-funded programs?

•

What outcomes do grantees track and report on? Do these outcomes align with the legislative
goals outlined for the AAHC program overall?

•

What are your thoughts on the sustainability of improvements?

Concluding
•

Is there anything you would like to add that we didn’t cover already?
o

[Prompt: Any final thoughts or reflections?]

Thank you for taking the time to speak with us and offer your thoughts and perspectives on the African
American History and Culture grant program and how it relates to your work. As we explained earlier,
your, much appreciated, contribution will help inform our broader evaluation of this program’s
performance and support IMLS’s efforts to improve the program and effectively communicate the
program’s contributions to the field.
The evaluation findings will be published after a final draft is shared with IMLS in the fall. Please do not
hesitate to reach out if you have questions after this call, about the interview or the broader evaluation,
or to provide additional information that may be relevant.
Thank you again for your time!

31

Appendix E – AAHC Program Logic Model

This is an interpretation of the African American History and Culture grant program logic model by the Urban Institute.

32

Appendix F. Burden Estimates of Respondents
Respondent Time and Cost Estimates

Interview

Survey

Instrument Population1

Total
Population

Estimated
response
rate

Predicted
Target
responses responses

Burden
hour per
response

Total
burden
hours

Grantees (Cohorts
2 and 3)
74
75%
56
56
0.50
27.8
Nonapplicants
300
50%
150
150
0.08
12.5
Applicants (20142019)
26
60%
16
5
1.00
5.0
Grantees: pre-2014
(cohort 1)
27
50%
14
5
1.00
5.0
Grantees: 20142019 (cohort 2)
42
90%
38
20
1.00
20.0
Grantees: longterm (cohort 3)
32
90%
29
8
1.00
8.0
Funders
8
75%
6
4
1.00
4.0
2
IMLS staff
3
100%
3
3
1.00
3.0
Other key
stakeholders
20
75%
15
5
1.00
5.0
TOTAL
256
90.5
1
Total population for nonapplicants, funders, and other key stakeholders are estimates.
2
IMLS staff and hourly cost estimates are factored into the estimates of federal costs appearing in table
below.

Hourly cost
per
respondent

Total cost
for
population

$28
$28

$780
$350

$28

$141

$28

$141

$28

$562

$28
$60
N/A

$225
$238
N/A

$28

$141
$2,578

33

Federal Government Time and Cost Estimates
Description
Third-Party Independent
Evaluation1

Time
16
months
693
hours

IMLS Staff Salaries2
Total
1

Cost
$227,317
$48,447
$275,764

One-time, awarded contract to Urban Institute for conducting the independent evaluation

2 Estimate

based on time and salary aggregations of participating IMLS employees

34

Appendix G –Crosswalk of Research Questions to Data Sources
Research questions

Secondary data sources

Survey questions
(G=grantee survey; N=nonapplicant
survey)

Interview questions

1a. What is the universe of African
American museums and HBCUs?
What share have participated in the
AAHC program?

•
•
•
•
•

Association and program databases
Public Historian Journal
List of HBCUs
Foundation Grantee Lists
Other organizations found from
literature reviews
• IMLS administrative data

N/A

Other Stakeholder interview questions 4
and 9

1b. What is the universe of peer
funders to IMLS (government and
philanthropic funders of AA
museums on a national-level scope)?

• Foundation Center,
• Funders mentioned in qualitative
data collections
• Google using key search terms

G11. Use of AAHC to attract additional
funding

Funder interview question 2

G13. Awareness of similar funding

Applicant interview question 3
Grantee interview question 11
Grantee interview questions Cohort 2
specific module, New Grant Recipients
question 5
Funder interview question 2
Other Stakeholder interview question 2,
part b
IMLS staff interview question 4

35

Research questions

Secondary data sources

Survey questions
(G=grantee survey; N=nonapplicant
survey)

Interview questions

1c. Are there any key factors that
distinguish between grantees and
nonapplicants?

• Universe created in 1a
• Urban research on closing
likelihood post-recession
• 990 data
• Geospatial datasets of interest, e.g.
degree of segregation, nonprofit
density, and demographics.
• IMLS administrative data
• Publicly available information on
organizations

N3. Why hasn’t your organization ever
applied to AAHC program?

Funder interview question 3

• Universe created in 1a
• Urban research on closing
likelihood post-recession
• 990 data
• Geospatial datasets of interest, e.g.
degree of segregation, nonprofit
density, and demographics.
• IMLS administrative data
• Publicly available information on
organizations
• Review of IMLS administrative data

N/A

Applicant interview question 2

N3. Why hasn’t your organization ever
applied to AAHC program?

IMLS staff question 8e

1d. Are there any key factors that
distinguish between applicants and
grantees?

1e. What might IMLS do to better
connect to all potential applicants?
What tradeoffs might it need to
make to achieve this result given any
other competing objectives? What
benchmarks might exist?

N4. Have you every applied to, or
received, a grant from another IMLS
program?

N5. What type of funding support is
most useful to organizations such as
yours?

Other stakeholder interview question 8
IMLS staff interview questions 8 and 12

Funder interview questions 3a, 3c
Grantee interview questions AAHC’s Role in
Outreach and Engagement with African
American Museums and HBCUs and
Understanding the Universe of Prospective
Applicants question 21, 22, 23, 24
Other Stakeholder interview question 10

36

Research questions

Secondary data sources

Survey questions
(G=grantee survey; N=nonapplicant
survey)

Interview questions

1f. How has this program adapted to
evolving needs of eligible
organizations?

• Program documentation (program
descriptions, RFPs, NOFOs)

G2g. IMLS understanding of challenges

IMLS staff interview questions 8, part d and
12

G4. Projects ineligible for AAHC

Other stakeholder interviews question 3
Funder interview questions 4 and 5b and 5c

2a. How has this IMLS AAHC program
made a difference in the capacity of
the nation’s African American
museums and HBCU’s?

• Blogs on grantee performance
• Final and interim reports
• AAHC administrative data (to code
subgroups)
• 990 data

G2b. Access to training and knowledge
G2c. Connections with other grantees
G5. Projects ineligible for AAHC
G6. Extent to which grantees have
experienced 10 outcomes
G6. Attendance at an IMLS convening
G11. Use of AAHC to attract additional
funding

Grantee interview questions AAHC’s role in
building project-specific capacities among
grantees questions 1-6, AAHC’s role in
building general capacities among grantees
(including leveraging the AAHC grant)
questions 7-11, AAHC’s role in supporting
exhibits and public programs and the
visibility and recognition of grantees
questions 12-17.
Grantee interview questions for Cohorts 1,
2, and 3 that assess longer term impacts of
the AAHC grant.

G12. Funding alternative for project
2b. Are there certain parts of the
AAHC grant portfolio that have
performed better? For instance, how
has the performance of small grants
for small institutions compared to
large grants for large institutions?

• Blogs on grantee performance
• Final and interim reports
• AAHC administrative data (to code
subgroups)
• 990 data

[Answering this question will depend
primarily on an analysis of the
reported grantee outcomes (from
survey and secondary research),
broken down by subpopulations of
interest]

IMLS staff question 15a

37

Research questions

Secondary data sources

Survey questions
(G=grantee survey; N=nonapplicant
survey)

Interview questions

2ci. How have IMLS administrative
practices for the AAHC program
influenced participation of
applicants? Why? (For instance, does
it matter when the program makes
awards at the same time as other
IMLS grant programs?)

Review of the AAHC program’s
administrative standards and
procedures (including Notices of
Funding Availability)

G2a. Clarity of application process

Applicants interview question 2b

G2f. Responsiveness/ helpfulness of
IMLS staff
G3. Grant restrictions

Grantee interview question AAHC’s role in
outreach and engagement with African
American museums and HBCUs and
Understanding the Universe of Prospective
Applicants (Museums and HBCUs) question
24
Grantee interview questions - Cohort 2
module (2014-2019 grantees) questions
focusing on other funding opportunities
over the last 12 months.
IMLS staff interview questions 10, 12, and
13

2cii. How have IMLS administrative
practices for the AAHC program
influenced grantees’ implementation
of project awards? Why? (Are there
any barriers built into the design of
the program or the way in which
program staff oversee the program
that limit grantees’ ability to achieve
project outcomes?)

Review of the AAHC program’s
administrative standards and
procedures

2d. How can program evaluation be a
more integral part of the grant
program and not an optional or
additional feature for grantees?

• Conversations with other
evaluation experts at Urban
• Review of grantee reports and
administrative records

G2d. Reporting requirements

Grantee interview question 20

G2f. Responsiveness/ helpfulness of
IMLS staff

IMLS staff interview question 14

G3. Grant restrictions

Grantee interview questions in Cohort 2
(2014-2019) questions for new grant
recipients about the implementation phase
of the program.
Questions for grantees in Cohort 3 (repeat
grantees) questions that focus on
comparing the application and
implementation stages of the AAHC
program with other grant programs.

G2e. Guidance of measuring results
G7. Process for tracking results

Grantee interview question in Cohort 3
(repeat grant recipients) question 1
IMLS staff interview questions 14 and 15

38

Research questions

Secondary data sources

Survey questions
(G=grantee survey; N=nonapplicant
survey)

Interview questions

G8. Measuring, reporting, and using
data on results
G9. Importance of results data to IMLS
2e. What has happened to the
museums (and staff) that received
funding in the early iteration of the
program? What have they
accomplished post-grant?

• Names from AAHC Grantee
Applications/Reports,
• LinkedIn
• Review of association membership
and program data

3a. How has the AAHC grant
program, now in its 13th year,
performed overall in meeting its
legislative goals?

Summary and interpretation of all
primary and secondary data collected
by the evaluation (including IMLS
administrative data, interviews,
project reports, etc.).

G10. Improving use of results
N/A

G2g. IMLS understanding of challenges
G12. Funding alternative for project

Grantee interview questions in Cohort 1
module (Pre-2014 grantees) question 1 and
grantees in Cohort 3 nodule (repeat grant
recipients) questions 2 and 3

Grantee interview questions Cohort 1 and 3
modules
IMLS staff interview questions 7, 15, and 16
Funders interview questions 7a and 7b
Other stakeholder interview question 7

3b. What are the barriers to the
AAHC program achieving better
outcomes?

Summary and interpretation of all
primary and secondary data collected
by the evaluation (including IMLS
administrative data, interviews,
project reports, etc.).

G4. Ineligible projects
G14. Suggestions for improving
program

Grantee interview questions 20, 23
IMLS staff interview question 9
Other stakeholders interview question 11

N5. Most useful funding

39