IMLS NANHAN Evaluation

This letter will be sent by IMLS on behalf of the evaluation team to respondents, informing them of the purpose of the evaluation, Kituwah Services LLC’s role, and the forthcoming request for information collection (survey or survey and interview) from Kituwah Services LLC. This letter will be shared with all potential respondents for this evaluation that may have prior knowledge of IMLS as an actor in this space (e.g., grantees, unsuccessful applicants, eligible non-applicant, public agency or private foundation funders, service or intertribal organizations, and Tribal leadership).

This letter will be sent approximately one week before Kituwah Services plans to contact the stakeholder for information collection (see Appendix B).

Letter to All Potential Respondents and/or Participants in the Evaluation

To: [Email address for identified contact]

From: [IMLS contact]

Subject: Requesting your assistance in an evaluation of IMLS grant programs designed specifically for Native American, Native Hawaiian, and Alaska Native communities

Dear {First Name},

I hope this message finds you well.

Because you are a leader in the field of sustaining and celebrating Native American, Native Hawaiian, and Alaska Native history and culture, I am writing to invite you to participate in activities to evaluate four grant programs designed for Native American, Native Hawaiian, and Alaska Native communities offered by the Institute of Museum and Library Services (IMLS): Native American Library Services: Basic Grants; Native American Library Services: Enhancement Grants; Native American/Native Hawaiian Museum Services; and Native Hawaiian Library Services.

IMLS would like to increase the number of Native American, Native Hawaiian, and Alaska Native communities that are aware of, apply to, and receive grant funds from at least one these four grant programs. Your input will help us produce useful and actionable information to help improve our grant-making in the future.

IMLS has contracted with Kituwah Services LLC, a tribally-owned business, to conduct this evaluation. IMLS hosted an informational session in March 2022 to get input from Native American, Native Hawaiian, and Alaska Native communities for this evaluation. The Kituwah Services evaluation team incorporated that input into designing an approach that captures evaluation information from a range of stakeholders, including current and former grantees, applicants that did not receive an award, eligible entities that have never applied to the grant programs, tribal leadership, peer funders in this space, and service or intertribal organizations.

In the coming weeks, evaluators from Kituwah Services will be reaching out to you to learn about your views. Your participation is entirely voluntary and will have no impact on your relationship with IMLS or the Federal Government. Any information you provide to Kituwah Services will be reported to IMLS in aggregate form only. No individual responses will be reported.

We hope you are willing to participate in this effort, as it will enable us to better understand the performance of these grant programs and help improve the programs for future applicants and grantees.

If you have any questions or concerns about this evaluation, please contact Bill Medcalf at Kituwah Services LLC ([email protected]; 402-650-2744) or Emily Plagman at IMLS ([email protected]; 202-653-4763).

On behalf of the IMLS team, I thank you in advance for taking the time to share your knowledge and insights with us.

Best,

[IMLS STAFF MEMBER]

[Title]

[Office]

Institute of Museum and Library Services

955 L’Enfant Plaza North, SW, Suite 4000

Washington, D.C. 20024

Appendix B. Survey Instrument

This letter will be sent by Kituwah Services on behalf of the evaluation team to contacts among IMLS grantees and eligible non-applicants approximately one week after IMLS has sent the respondent contact letter to grantees and eligible non-applicants (Appendix A). Text in italics and brackets indicates customization of the instrument for the two respondent groups (grantees and eligible non-applicants) and will not appear to the respondent on the instrument.

Invitation Letter

To: [Emails of primary contacts for eligible applicant list]

From: [Kituwah email address]

Subject: We would like to hear from you: Your perspective on the IMLS grant programs designed specifically for Native American, Native Hawaiian, and Alaska Native communities

Dear {First Name},

I am part of a team of evaluators from Kituwah Services LLC, a tribally-owned business, working with the Institute of Museum and Library Sciences (IMLS) to learn more about their four grant programs designed specifically for Native American, Native Hawaiian, and Alaska Native communities: Native American Library Services: Basic Grants; Native American Library Services: Enhancement Grants; Native American/Native Hawaiian Museum Services Program; and Native Hawaiian Library Services. The goal of this evaluation is to help IMLS increase the number of Native American, Native Hawaiian, and Alaska Native communities that are aware of, apply to, and receive grant funds from at least one of these programs.

[for applicants: We invite you to complete this survey so that we may learn more about what has worked and what could be improved in these programs from the applicant perspective. Specifically, we are interested in learning about your experiences with the application process for at least one of the grant programs for which your tribe or nonprofit may be eligible, your expectations for the future direction of your organization, any specific challenges you may have, and contributions these IMLS programs might make or have made to your project or organization.]

[for eligible non-applicants: We invite you to complete this survey so that we may learn more about the views of tribes and nonprofits serving primarily Native Hawaiians that have never submitted an application for IMLS funding to any of the four grant programs. Specifically, we are interested in learning about your organization’s unique strengths, challenges, and funding priorities.]

Your responses will help us understand 1) your perceptions of the grant programs, 2) how the program(s) did or could contribute to your community and, more specifically its museum- and library-related goals, and 3) opportunities to strengthen the program(s).

This survey should take no more than 30 minutes to complete [for eligible non-applicants: 15 minutes]. You do not need to complete the entire survey in one sitting. Your input will help us produce useful and actionable information to help improve the programs for future applicants and grantees.

To access the survey, please click this link: [unique survey link]. Please do not share this link with others, as it is specific to your organization or department.

On behalf of the evaluation team, thank you in advance for taking the time to share your knowledge and insights with us.

Best,

[KITUWAH STAFF MEMBER]

[Title]

[Office]

[Address]

Survey Landing Page/Consent Language

Thank you for considering participating in this survey, developed by IMLS and administered by Kituwah Services LLC. The goal of this evaluation is to help IMLS increase the number of Native American, Native Hawaiian, and Alaska Native communities that are aware of, apply to, and receive grant funds from at least one of their four grant programs designed specifically for Native American, Native Hawaiian, and Alaska Native communities:

IMLS grant program	Entity Eligibility Criteria
Native American Library Services: Basic Grants	Federally Recognized Tribe AND Existing Library with Reg Hours, Staff, Materials for Users
Native American Library Services: Enhancement Grants	Federally Recognized Tribe AND Existing Library with Reg Hours, Staff, Materials for Users
Native Hawaiian Library Services	Nonprofit That Serves Primarily Native Hawaiians
Native American/Native Hawaiian Museum Services	Federally Recognized Tribe OR Nonprofit That Serves Primarily Native Hawaiians

Specifically, we are interested in learning more about your experiences with the application process (if applicable), the goals and future direction of your museum or library services project(s), any specific capacity challenges you may have when delivering these services, and contributions these four IMLS programs may have made to your project or organization (if applicable).

Please answer each question to the best of your knowledge based on your understanding of the grant program. The survey should take about [for applicants: 30 minutes to complete] [for eligible non-applicants: 15 minutes to complete]. If you are unable to complete the survey in one session, your answers will be saved, and you can return to the last page of the survey you visited by clicking on the original link provided. [for grantees: In addition, you may wish to keep a copy of your completed grant application or other grant materials at hand as you complete this survey.]

Informed Consent: Your participation is entirely voluntary, and you may choose to stop the survey at any time or skip questions for any reason. All responses received will be aggregated during the analysis process to protect individual identities. No individual or organizational names will be used in any reporting. We will not quote any of your responses unless we receive your express written consent. Your decision to participate or not will have no impact on the success of future applications to IMLS programs. The information you provide will be kept confidential and all personally identifiable information will be protected. Once this survey project is completed, the files and materials created and stored will be destroyed.

We greatly appreciate your help with this important survey. Please complete the survey by [date]. If you have questions or concerns before deciding to participate or while completing the survey, please contact Bill Medcalf at Kituwah Services LLC ([email protected]; 402-650-2744) or Emily Plagman at IMLS ([email protected]; 202-653-4763).

We want to be sure that you freely consent to participate in this survey and are aware that you are not obligated to answer any questions you do not wish to and that there will be no consequence to your organization should you decline to answer any question. Do you consent to participate in this survey?

Yes [if yes is checked, skip to Q2]
No [if no is checked, skip to end of the survey]

Instrument

Thanks for your time! We are asking you to complete this survey to learn about your organization's needs and capacity around grant funding -- specifically, your knowledge of and experiences with the four IMLS grant programs designated for federally recognized Native American tribes (including Alaska Native villages, regional corporations, and village corporations) or nonprofits that primarily serve and represent Native Hawaiians.

Organizational Characteristics

[This question section is for grantees and eligible non-applicants]

What is the name of [the Federally Recognized Indian Tribe that you serve/your Nonprofit Organization that serves primarily Native Hawaiians] ([“Tribe”/”Nonprofit”]):

How would describe the type of organization within your [Tribe/Nonprofit] in which you conduct your work? (Check one).

A library that has posted regular hours and makes available library services and materials to the public and is under the supervision of at least one permanent professional staff librarian
Museum
Other (please specify):

What is the name of the organization or department within your [Tribe/Nonprofit] in which you conduct your work?

Which of the following best describes your role within the organization? (Check one).
1. Executive Director
2. Department Director
3. Project Manager
4. Grants Manager/Coordinator
5. Board Member

Other (please specify):

How long have you been in this role? (Check one).
1. Less than one year
2. Between 1-3 years
3. Between 4-10 years
4. More than 10 years

In addition to the organization type referenced above, does your [Tribe/Nonprofit] operate any other organizations or departments that represent the work of libraries and/or museums?

If yes, please provide the name and organization.

What is the staff size of the organization where you work? (Check one in each row.)

Staff type	Number of People					NA
Staff type	Less than 5	5-9	10-24	25-49	More than 50	NA
Full-time
Part-time
Volunteer
Student intern
Other (please specify):

How would you classify your organization’s annual operating budget? (Check one.)

Less than $100,000
$100,000 to $249,999
$250,000 to $499,999
$500,000 to $999,999
$1,000,000 and greater

Think about the various funding categories for your programs and services. What is the estimated percentage of your total annual operating budget that comes from the following sources? (Please enter a number as a percentage of your budget for each source; enter 0 if it is not a source.)

Funding Stream	Percentage of your budget (%)
Tribal government
State or local government
Federal government
IMLS
Other government sources
Private foundations
Corporate sponsorship
Individual donors
Earned revenue (e.g., ticket sales, gift shop sales)
Other (please specify):

Familiarity with IMLS

[This question section is for grantees and eligible non-applicants]

How familiar is your organization with the Institute of Museum and Library Services’ (IMLS’s) four funding programs available to [Tribe\Nonprofit]? On a scale from 1, “Not at All Familiar,” to 10, “Very Familiar,” please rate your familiarity with the four IMLS funding programs.

[Scale 1-10]

Over the last five years, did your organization or department submit one or more applications for funding through any IMLS grant program?

Yes
No [respondent flagged as eligible non-applicant, skip to Organization Programs]
Not sure [skip to question 14]

[If yes] To what IMLS grant programs did your [Tribe/Nonprofit] submit an application for funding? (Check all that apply.)

Native American Library Services: Basic Grants
Native American Library Services: Enhancement Grants
Native American/Native Hawaiian Museum Services Program
Native Hawaiian Library Services
CARES Act Grants for Native American/Native Hawaiian Museums and Library Services
American Rescue Plan Act (ARPA)
Collections Assessment for Preservation
Community Catalyst Initiative
Digital Humanities Advancement Grants
CARES Act Grants for Museums and Libraries
Inspire! Grants for Small Museums
Laura Bush 21^st Century Librarian Program
Museum Assessment Program
Museum Grants for African American History and Culture
Museum Empowered
Museums for America
National Leadership Grants for Libraries
National Leadership Grants for Museums
Save America’s Treasures
None of the above
Not sure

[If yes, for applicants] Please share some details about your reasons for applying for IMLS funding.

[text box]

Has your [Tribe/Nonprofit] considered applying, or has it applied, to more than one grant program to the IMLS during the same fiscal year?

Yes
No [skip to Q17]
Not sure

[If yes] Please list the grant programs to which you most recently applied within the same fiscal year, and indicate whether an application was unsuccessful.

[text box]

Organization or Department Programs

[This question section is for grantees and eligible non-applicants]

Thinking about the quality of the work your organization performs, on a scale from 1, “Poor,” to 10, “Excellent,” please rate your activities related to the following. (Check one in each row.)

	Poor Excellent										NA
	1	2	3	4	5	6	7	8	9	10	NA
Community outreach
Children’s programs
Elders’ programs
Educational programs
Language programs
Digital/technology access
Exhibitions
Collections acquisition
Collections/archival management
Other (please specify):

Think about the work your organization performs. On a scale from 1, “Low Priority,” to 10, “High Priority,” please rate the priority of each of the following in your organization’s effort to advance its mission. (Check one in each row).

	Low Priority High Priority										NA
	1	2	3	4	5	6	7	8	9	10	NA
General operating support
Children’s programs
Elders’ programs
Adult education programs
Language programs
Broadband infrastructure
Collections/archives infrastructure
Collections development
Exhibition development
Other (please specify):

Think about the work your organization performs. On a scale from 1, “Major Barrier,” to 10, “Not At All A Barrier,” please rate internal barriers you experience that inhibit your organization’s ability to deliver the highest quality programs and services possible. (Check one in each row.)

	Major Barrier Not At All A Barrier										NA
	1	2	3	4	5	6	7	8	9	10	NA
Overall funding
Staff capacity
Ability to secure grant funding
Scope of programs and services
Broadband infrastructure
Internal grants process
Organizational governing bodies (e.g., Tribal government, nonprofit boards)
Transportation
Collections and program management
Other (please specify):

Website: Please think about your experience using the IMLS online resources on the IMLS.gov website. On a scale from 1,“Poor,” to 10, “Excellent,” please rate the strength of your experience with the following. (Check one in each row.)

	Poor Excellent										NA
	1	2	3	4	5	6	7	8	9	10	NA
Ability to find specific grant application information
Quality of content (e.g., materials are up-to-date, helpful)
Ability to accomplish what you want on the site
Accuracy of search results
Relevance to your area of need
Look and feel/visual appearance
Clarity of instructions in the Notice of Funding Opportunity
Sufficient detail of instruction to meet your needs
Other (please specify):

Experience with IMLS

[This question section is for grantees]

Information in Application Package: When you were preparing your application, how easy was it for you to locate and understand the information in the application package? On a scale from 1, "Very Difficult,” to 10, “Very Easy,” please rate the following. (Check one in each row.)

	Very Difficult Very Easy										NA
	1	2	3	4	5	6	7	8	9	10	NA
Grant program purpose
Grant program priorities
Selection criteria
Peer review process
Budget information and forms
Deadline for submission
Dollar limit on awards
Page limitations
Format and submission requirements
Program contact
Other (please specify):

Over the last five years, has your organization or department sought technical assistance from IMLS during the application stage? (Check one).

Yes
No [go to Question 23]
Not sure

Application Stage - Technical Assistance Provided by IMLS Staff: On a scale from 1, “Not At All Helpful,” to 10, “Very Helpful,” please rate your experience with the following. (Check one in each row.)

	Not At All Helpful Very Helpful										NA
	1	2	3	4	5	6	7	8	9	10	NA
Learning about IMLS program eligibility requirements
Preparing a competitive application
Reviewing panel comments
Other (please specify):

Grantee Documents: Think about the information you received from the IMLS after receiving notification of your award including the award notification, reporting requirements, instructions for requesting approval of a change, requesting payment, submitting financial and performance reports, and other relevant documents. On a scale from 1, “Poor,” to 10, “Excellent,” please rate the documents on the following dimensions. (Check one in each row.)

	Poor Excellent										NA
	1	2	3	4	5	6	7	8	9	10	NA
Clarity
General information from IMLS (e.g., staff emails, eGMS messages)
Sufficient detail to meet your program needs
Relevance to your program needs
Comprehensiveness in addressing the scope of challenges you encounter

Grantee Technical Assistance Provided by IMLS Staff: On a scale from 1, “Not At All Helpful,” to 10, “Very Helpful,” please rate your experience with IMLS Staff regarding the following. (Check one in each row.)

	Not At All Helpful Very Helpful										NA
	1	2	3	4	5	6	7	8	9	10	NA
Implementing a grant project
Requesting approval of a change
Closing out the grant
Other (please specify):

Performance Reporting: Think about the IMLS performance reporting requirements for your grant and rate the following on a scale from 1, “Poor,” to 10, “Excellent.” (Check one in each row.)

	Poor Excellent										NA
	1	2	3	4	5	6	7	8	9	10	NA
Clarity of reporting requirements
Ease of obtaining data for your required reports
Ease of submitting report(s) electronically
Availability of assistance to complete your report(s) (e.g., technical assistance and training)
Usefulness of the performance data to help you improve your grant project
Your understanding of how IMLS uses your data

Financial Reporting: Think about the IMLS financial reporting requirements for your grant and rate the following on a scale from 1, “Poor,” to 10, “Excellent.” (Check one in each row.)

	Poor Excellent										NA
	1	2	3	4	5	6	7	8	9	10	NA
Clarity of reporting requirements
Ease of obtaining data for your required reports
Ease of submitting report(s)
Availability of assistance to complete your report(s) (e.g., technical assistance and training)
Usefulness of the financial data to help you improve your grant project
Your understanding of how IMLS uses your data

Changes: During the last five years, did you request approval for a change, including an extension of the period of performance, for your grant? (Check one).

Yes
No [skip to Question 28]
Not sure

Think about the circumstances that required you to request a change. On a scale from 1, “Poor,” to 10,“Excellent,” please rate your experience in seeking approval of a change. (Check one in each row.)

	Poor Excellent										NA
	1	2	3	4	5	6	7	8	9	10	NA
Overall ease of filing request
IMLS staff responsiveness
Clarity of instruction
Submission process
Communication about the decision
Other (please specify):

Think about filing a request for approval of change in your grant. On a scale from 1, “Not a Factor,” to 10, “Significant Factor,” please rate the following factors for the reason you submitted the request. (Check one in each row.)

	Not a Factor Significant Factor										NA
	1	2	3	4	5	6	7	8	9	10	NA
Delayed project start
Staff transition
Partner issues
Funding
Other (please specify):

Organizational Capacity: As a result of receiving IMLS grant funding, which of the following factors changed within your organization or department? (Check all that apply).

New staffing
Improved organization systems - approaches to work
Expanded collections
Improved collections management
Additional professional development and skill building
Increased accessibility to the collection
Expanded use of technology
New partnerships
Expanded funding sources
Other (please specify) ______________________________

Grants and Fundraising

[This question section is for grantees and eligible non-applicants]

Think about the months leading up to the submission of an application, also known as the pre-application process. On a scale from 1, “Not At All Helpful,” to 10, “Very Helpful,” please rate the usefulness of the following services in preparing your application. (Check one in each row.)

	Not At All Helpful Very Helpful										NA
	1	2	3	4	5	6	7	8	9	10	NA
Support for project planning/ development
Grant writing workshop
Lessons from previous awardees
Guidance with program evaluation design
Review preliminary draft
Finding the information that you need on the IMLS website
Other (please specify):

Think about your organization or department’s fundraising efforts. On a scale from 1, “Poor,” to 10, “Excellent,” please rate the capacity of your organization or department to secure funding from the following. (Check one in each row.)

	Poor Excellent										NA
	1	2	3	4	5	6	7	8	9	10	NA
Federal agency grant programs
State agency grant programs
Private foundations
Corporate sponsorship
Individual donors
Other (please specify):

Think about the process in which your organization or department engages when seeking grant funds. On a scale from 1, “Poor,” to 10, “Excellent,” please rate the capacity of your organization or department to seek and secure grant funds along the following dimensions. (Check one in each row.

	Poor Excellent										NA
	1	2	3	4	5	6	7	8	9	10	NA
Staffing levels to research and identify potential grant opportunities
Staff skills necessary to develop competitive proposals
Experience with developing grant-funded projects
Organization’s ability to manage awarded grants
Experience with administrative and reporting requirements

Think about your organization’s or department’s need for technical assistance from IMLS staff when seeking and/or securing grant funds. On a scale from 1, “Not At All Useful,” to 10, “Extremely Useful,” please rate the usefulness of the following types of technical assistance. (Check one in each row.)

	Not At All Useful Extremely Useful										NA
	1	2	3	4	5	6	7	8	9	10	NA
Strategic/organizational planning
Project planning and development
Grant writing
Project management
Other (please specify):

How do you receive information regarding potential funding opportunities? (Check all that apply).

Funder Website
Grants.gov
Google or other search engine
Tribal Administrator/Government
Service Organization
Inter-Tribal Organization
Colleague in your professional network
Other (please specify):

Organizational Strengths & Wishes

[This question section is for grantees and eligible non-applicants]

Thinking about the work your organization or department performs, what are the three greatest strengths that help you advance your mission?

[text box]

Thinking about the best ways to advance your organization’s or department’s mission, describe three wishes.

[text box]

Would you be willing to provide your contact information in case the evaluation team has follow up questions? (Check one.)

No, thank you [go to closing survey message]
Yes [if checked display the following question]

Thank you! Please share your contact information below

Name: __________________
Phone number: _______________
Email: _______________

[Closing Survey Message]

Thank you!

Thank you for your time completing this survey! The information you provided will be used to help improve IMLS programs serving organizations such as yours. For more information about IMLS grant programs, please visit: https://imls.gov/

Appendix C. Semi-structured Interview Guides

This section provides the invitation emails that will be sent by the Kituwah Services LLC, inviting respondents to participate in semi-structured interviews. See Appendix A for the letter to all potential respondents and/or participants in the evaluation from IMLS, which will precede Kituwah's invitation letter below. In addition, this appendix includes the informed consent language that will be used at the start of every interview and the semi-structured interview protocols for all respondent groups. All the questions included in each protocol may not be asked directly or sequentially, as the interviewers will customize the questions and their sequencing, particularly if the respondent answered a given question indirectly earlier in the discussion. There is guiding language for interviewers to use under each question heading in italics within the semi-structured interview guides, as well as specific prompts italicized in square brackets for use as needed.

Invitation Letter

To: [List of primary contacts for each interview list]

From: [Kituwah email address]

Subject: We would like to hear from you: Your perspective on the IMLS grant programs designed specifically for Native American, Native Hawaiian, and Alaska Native communities

Dear {First Name},

I am part of a team of evaluators from Kituwah Services LLC, a tribally-owned business, working with the Institute of Museum and Library Sciences (IMLS) to learn more about their four grant programs designed specifically for [Federally Recognized Indian Tribe that you serve/your Nonprofit Organization that serves primarily Native Hawaiians]. The goal of this evaluation is to help IMLS increase the number of Native American, Native Hawaiian, and Alaska Native communities that are aware of, apply to, and receive grant funds from at least one of these programs.

We would like to interview you to learn more about your views as...

[for grantees: a grantee about what has worked and what could be improved relative to the IMLS programs designed specifically for [Federally Recognized Indian Tribe that you serve/your Nonprofit Organization that serves primarily Native Hawaiians]. Specifically, we are interested in learning more about your experiences with the application process, the success and sustainability of the project/organization, any specific capacity challenges you encountered, and contributions these IMLS programs may have made to your project or organization’s mission.]
[for unsuccessful applicants: an applicant who did not receive an IMLS award in the last five years, who understands the application process, and is aware of the strengths of and challenges facing organizations like your own. We believe your perspective will be invaluable in helping to improve the application process and better understand barriers for prospective applicants.]
[for eligible non-applicants: an eligible non-applicant who is aware of the strengths, challenges, and funding priorities of organizations like yours and has not applied for IMLS grant funds.]
[for funders: a public agency or private foundation funder in this space on the landscape of Native American, Native Hawaiian, and Alaska Native communities, their strengths, the challenges they face when seeking funds, and how targeted grants may best support them.]
[for service or intertribal organization leaders: a service or intertribal organization leader who is knowledgeable about the landscape of organizations in this space, the challenges they face, and the potential for programs like these four IMLS grant programs to help.]
[for a Tribal or Native Hawaiian leader: a Tribal or Native Hawaiian leader who is knowledgeable about the strengths, specific needs, and considerations of organizations striving to preserve the history and culture of Native American, Native Hawaiian, and Alaska Native communities.]

This interview is entirely voluntary and, should you choose to participate, will last no longer than one hour. We will be conducting interviews between [date] and [date] at a time that is convenient for you. If you are willing to participate, please indicate your preferred interview time by responding to this Doodle poll (in which you should please be sure to also record your name, where prompted) OR by responding to this email with preferred dates and times. Once we confirm a date and time, we will send you a calendar invitation to confirm the appointment, along with credentials for dialing in to the conference line that we will use for the call.

If there is someone else in your organization or department that you recommend that we interview instead of you (or in addition to you), we would be grateful for their name and contact information.

If you have questions or concerns, please contact Bill Medcalf at Kituwah Services LLC ([email protected]; 402-650-2744) or Emily Plagman at IMLS ([email protected]; 202-653-4763).

On behalf of the evaluation team, thank you in advance for taking the time to share your knowledge and insights with us.

Best,

[KITUWAH STAFF MEMBER]

[Title]

[Office]

[Address]

Introductory and Consent Language for Interview

The following will precede each interview.

The Institute of Museum and Library Services has asked Kituwah Services LLC (a tribally-owned business) to conduct an evaluation of its four grant programs designed specifically designed for [Federally Recognized Indian Tribe that you serve/your Nonprofit Organization that serves primarily Native Hawaiians]. The goal of this evaluation is to help IMLS increase the number of Native American, Native Hawaiian, and Alaska Native communities that are aware of, apply to, and receive grant funds from at least one of these programs. This is an evaluation of the IMLS programs themselves, not an evaluation or audit of grantee outcomes.

Your feedback is extremely important to the success of this evaluation because as:

a grantee, you/your organization or department has experience navigating the application process, implementing an IMLS grant, and observing and reporting outcomes related to that grant.
an applicant who has not received an IMLS award in the last five years, you/your organization or department has experience navigating the application process and are/is aware of some of the challenges facing organizations like your own.
an eligible non-applicant, you/your organization or department are/is knowledgeable about the strengths, challenges, and funding priorities for organizations like your own and has not applied for IMLS grant funds.
a public agency or private foundation funder in this space, you/your organization or department are/is knowledgeable about many of the eligible organizations in this space, the challenges they face and how targeted grants may best support them.
a service or intertribal organization, you/your organization or department are/is knowledgeable about many of the eligible organizations in this space, their strengths, the challenges they face, and the potential for IMLS programs like these to help organizations with whom you work closely.
a tribal or Native Hawaiian leader, you/your tribal government are/is knowledgeable about the strengths, specific needs, and considerations of organizations striving to preserve the history and culture of Native American, Native Hawaiian, and Alaska Native communities.
an IMLS staff member, you are knowledgeable about these four programs and can offer insights into their purposes, processes, and impacts on the field.

This interview is completely voluntary. You may terminate it at any point for any reason, and you may decline to answer any question. Your responses will be used to inform our understanding of the IMLS grant programs and their role in supporting grantee outcomes, the landscape of Native American, Native Hawaiian, and Alaska Native museums and libraries these grant programs are intended to serve, and opportunities for enhanced or improved services by IMLS and others.

We will be taking notes during this interview and would like to record the interview to ensure we are accurately capturing the information you provide. All interview data will be saved to an encrypted network drive, with password-controlled access limited to the survey evaluators and subcontractors, all of whom have signed confidentiality agreements. Access will only be available on-site or secure remote access, through password-protected computers. The files will be destroyed at the end of the evaluation.

Individual response data will not be shared with IMLS except in de-identified aggregate/group form. Before delivering any final datasets to IMLS, the evaluation team will remove any personally identifying information (PII)—such as name, address of respondents, their organizational affiliation, or any other characteristics that would make smaller subsets of eligible applicants identifiable — or that could permit disclosure or identification of respondents, directly or by inference. We will not quote anything you say unless we receive your express written consent.

Your decision to participate or not will have no effect on your efforts to apply for or perform under IMLS grant programs.

We want to be sure that you freely consent to participate in this interview and that you are aware that you are not obligated to answer any questions you do not wish to. Do you consent to participate in the interview?

[If yes] Do you consent to this interview being recorded?

[If yes, turn on recorder] [If no, I will take contemporaneous note that will be maintained as confidential and destroyed at the evaluation’s end.]

Interview Protocol for Grantees

Organizational Characteristics

We would like to start by learning a bit about your organization or department.

Describe your organization or department.
1. What is the organization type [prompts: department within Tribal government, 501c3 nonprofit]?
2. What is the organization’s annual budget?
3. How many full-time and part-time staff or volunteers do you have?
4. What are your core programs and services?
5. Whom do you consider your primary audience?
6. How is the organization funded?

Think about your visitor experience. Describe [Name of Organization] when it is at its best?

Think about your organization or department’s development over the next five years. What are the greatest opportunities for growth, raised visibility, or greater community impact? What are your most significant challenges? [prompts: What should you be doing more of? What should you be doing less of? What should you be doing that is completely different?]

Impact of IMLS Funding

We want to reflect on the project(s) that IMLS funds have supported.

Remember a standout experience associated with the IMLS-funded project(s).

Briefly describe the project and its core goals.
What happened?
Who was involved?
What did you contribute to the experience?
What were the key factors that made the project possible?
Tell your story describing the experience in detail.

How did the project advance your organization’s or department’s mission? Describe any changes to how you conduct your work as a result of IMLS funding. What efficiencies, gains or improvements, if any, were realized with IMLS funding? Did the project result in an increase in visitors or increased participation in programs?

What impact, if any, did the project have on organizational capacity? What organization systems were established? How did staff benefit from the project? What partnership(s) emerged from the project?

How did IMLS funding change the size and scope of the organization? How, if at all, has staffing changed? What impact has IMLS funding had on the organization’s budget? How will you sustain this change over time?

Sustainability

Transitioning slightly, we want to learn more about the sustainability of the project and what potentially would have happened without IMLS funding.

When the grant came to an end, were you able to sustain activities funded through IMLS [if so, how]? How did IMLS funding impact your organization’s or department’s earned income? Were you able to leverage IMLS funding for other grant funding or charitable support [if so, how]?

If IMLS funding was not awarded, would you have been able to conduct and/or complete the project? What changes would you have had to make to conduct and/or complete the project? What other funding sources would you have sought?

Grant Application Process and Program Administration

Think about your work in seeking grant funding from IMLS, the proposal development process, submitting a proposal, receiving reviewer feedback, executing your grant agreement, and managing the grant once the award was received.

What resources and/or support from IMLS did you find useful? Describe any resources, such as direct technical assistance from staff or online instruction, that were helpful during the following grant lifecycle stages:
1. Proposal development process
2. Application submission
3. Reviewer feedback
4. Executing the grant agreement
5. Managing and reporting on the grant
6. Closing out the grant

How could IMLS strengthen its support during the application process?

How could IMLS improve its outreach and communications about upcoming grants?

What are you’re the greatest strengths and greatest challenges in applying for and managing IMLS funding of your organization or department?

Grants and Fundraising

Part of this evaluation’s goal is to better understand the funding landscape for organizations like yours.

Describe the various sources of information you use to identify potential funding opportunities for your organization or department.
1. What role do service organizations play? [prompts: An example of a service organization is the Association of Tribal Archives, Libraries, and Museums (ATALM).]
2. What role do intertribal organizations (ATNI) play? [prompts: Examples of intertribal organizations are the United South and Eastern Tribes (USET) or the Affiliated Tribes of Northwest Indians.]
3. Are there other types of organizations that help identify potential funding sources? [prompts: State agencies.]

If you had to select two exemplary funders for your organization or department, what organizations would you choose? What is it about their funding practices makes them exemplary?

COVID-19

Over the last two years we all have been responding to the COVID pandemic.

Describe how COVID affected your organization or department. How did your programming and services change and for how long? What are changes made during COVID that you likely will continue long past the pandemic?

Closing Questions

We would like to end our time together with a couple of summary questions to help us better understand what recommendations to share with IMLS.

What questions would you like to ask IMLS staff to better understand their funding programs?

If you had three wishes for the continued development of [Name of Organization]'s work in the community, what would they be?

Interview Protocol for Unsuccessful Applicants

Organizational Characteristics

We would like to start by learning a bit about your organization or department.

Describe your organization or department.
1. What is the organization or department type?
2. What is the organization’s annual budget?
3. How many full-time and part-time staff and volunteers do you have?
4. What are your core programs and services?
5. Whom do you consider your primary audience?
6. How is the organization funded?

Think about your visitor experience. Describe {Name of Organization or Department} when it is at its best?

Think about the development of your organization or department over the next five years. What are the greatest opportunities for growth, raised visibility, or greater community impact? What are your most significant challenges? [prompts: What should you be doing more of? What should you be doing less of? What should you be doing that is completely different?]

Outcome of Proposed Projects

We want to reflect on the project(s) for which you sought IMLS funding support or that you may have developed with IMLS funding in mind.

Were you able to complete the project with other funding?
1. If yes, what were the key factors that made the project possible? What adjustment did you have to make to the project scope to meet the amount of funding available?
2. If no, what was the consequence of not undertaking the project?

Grant Application Process

Think about your work in seeking grant funding from IMLS, the proposal development process, submitting a proposal, and receiving reviewer feedback.

What resources and support from IMLS did you find useful? Describe any resources, such as direct technical assistance, from staff or online instruction, that were helpful during the following grant lifecycle stages:
1. Proposal development process
2. Application submission
3. Reviewer feedback

How could IMLS improve its outreach and communications process about its grant offerings?

What are the greatest challenges your organization or department faces in applying for IMLS funding?

Describe how the reviewer comments helped provide a better understanding of how best to strengthen future IMLS proposals or if they hindered your understanding of how to apply. How can IMLS better integrate reviewer comments to help improve the grant application process?

Grants and Fundraising

One of this evaluation’s goals is to better understand the funding landscape for organizations like yours.

Describe the various sources of information you use to identify potential funding opportunities for your organization or department.
1. What role do service organizations play? [prompts: An example of a service organization is the Association of Tribal Archives, Libraries, and Museums (ATALM).]
2. What role do intertribal organizations (ATNI) play? [prompts: Examples of intertribal organizations are the United South and Eastern Tribes (USET) or the Affiliated Tribes of Northwest Indians.]
3. Are there other types of organizations that help you identify potential funding sources? [prompts: state agencies]

If you had to select two exemplary funders for your organization or department, what organizations would you choose? What is it about their funding practices makes them exemplary?

COVID-19

Over the last two years, we all have been responding to the COVID pandemic.

Describe how COVID impacted your organization or department. How did your programming and services change and for how long? What are changes made during COVID that you will continue long past the pandemic?

Closing Questions

We would like to end our time together with a couple of high-level questions to help us better understand what recommendations to share with IMLS.

What questions would you like to ask IMLS staff to best understand their funding programs?

If you had three wishes for the continued development of [Name of Organization]'s work in the community, what would they be?

Interview Protocol for Eligible Non-applicants

Organizational Characteristics

We would like to start by learning a bit about your organization or department.

For the purposes of this interview, we define organization or departments affiliated with eligible entities by these three types:

A Tribe or Native Hawaiian nonprofit library accessible by community members and serves the information needs of patrons through a collection of books and other media for studying and/or borrowing. Some libraries provide computer access and public programs for patrons.
A Tribe or Native Hawaiian nonprofit museum is an organization that collects, stores, and exhibits objects with cultural, historic, or artistic value. Some eligible entities retain these collections and perform these functions at a cultural center.
A Tribe or Native Hawaiian nonprofit archive contains records of permanent value.

We recognize that some organization or departments consider themselves to be “cultural centers” that include multiple organization or department types, such as a museum and a library. We are interested in learning about the experiences and views on IMLS grants that support these organization or department types.

Describe your organization or department.
1. What is the organization type? [prompts: department within Tribal government, 501c3 nonprofit]
2. What is the organization’s annual budget?
3. How many full-time and part-time staff and volunteers?
4. What are your core programs and services?
5. Who do you consider your primary audience?
6. How is the organization funded? What is its primary source of funding? Do you seek funding from external sources such as public agencies like IMLS? Do you seek funding from private foundations?

Think about your visitor experience. Describe {Name of Organization} when it is at its best?

Think about your organization or department’s development over the next five years. What are the greatest opportunities for growth, impact, improvement? What are your most significant challenges? [prompts: What should you be doing more of? What should you be doing less of? What should you be doing that is completely different?]

Familiarity with IMLS

We would like to learn a bit more about your familiarity with IMLS and any considerations you might have in applying to their grant programs.

Are you familiar with IMLS? Have you ever considered seeking funds from the agency? Why or why not?

Grants and Fundraising

Part of this evaluation’s goal is to better understand the funding landscape for organizations like yours.

How would you describe the capacity of your organization or department to pursue external funding sources? What are the greatest constraints to your organization’s seeking and securing external funding?

What resources and support do you seek that may assist you with your proposal development? What are the greatest challenges your organization or department faces in applying for and managing externally-funded projects?

Describe the various sources of information you use to identify potential funding opportunities for your organization or department.
1. What role do service organizations play? [prompts: An example of a service organization is the Association of Tribal Archives, Libraries, and Museums (ATALM).]
2. What role do intertribal organizations (ATNI) play? [prompts: Examples of intertribal organizations are the United South and Eastern Tribes (USET) or the Affiliated Tribes of Northwest Indians.]
3. Are there other types of organizations that help identify potential funding sources? [prompts: state agencies.]

If you had to select two exemplary funders for your organization or department, what organizations would you choose? What is it about their funding practices that makes them exemplary?

COVID-19

Over the last two years, we all have been responding to the COVID pandemic.

Describe how COVID impacted your organization or department. How, if at all, did your programming and services change and for how long? What are changes made during COVID that you will continue past the pandemic?

Closing Questions

We would like to end our time together with a couple of high-level questions to help us better understand what recommendations to share with IMLS.

What questions would you like to ask IMLS staff to best understand their funding programs?

If you had three wishes for the continued development of [Name of Organization]'s work in the community, what would they be?

Interview Protocol for Public Agency or Private Foundation Funders

Organizational Characteristics

We would like to start by learning a bit about your organization or department.

Describe [Name of the Organization] funding programs, eligibility requirements, and range of grant amounts. What are [Name of the Organization] funding priorities and/or strategic initiatives?

In considering [Name of the Organization]'s funding portfolio, what is the estimated percentage of grantees that are Native American, Native Hawaiian, or Alaska Native organizations or that primarily serve Native American, Native Hawaiian, or Alaska Native constituents?

Outreach

After learning a bit more about your grant programs, it would be helpful to hear a little more from you on how you attract applicants via outreach.

Describe the outreach you conduct to attract Native American, Native Hawaiian, and Alaska Native applicants. What are the specific strategies you use to reach these potential applicants?

Applicant/Grantee Capacity and Technical Assistance

Given your knowledge of the field, we would love to learn a bit more from you about what you think the opportunities and challenges are for organizations in this space.

Describe the opportunities your organization or department provides to Native American, Native Hawaiian, and Alaska Native communities. Describe the greatest challenges these communities experience in accessing your funding and how best would they overcome those challenges.

How would you describe Native American, Native Hawaiian, and Alaska Native organizational capacity or readiness to secure grant funds? Describe the technical assistance and support your organization or department typically provides to applicants.

COVID-19

Over the last two years, we have all been responding to the COVID pandemic.

Describe how COVID impacted your organization or department. How did your programming and services change and for how long? What are changes made during COVID that you will continue past the pandemic? How do these changes impact how you fund or support Native American, Alaska Native, or Native Hawaiian communities?

Interview Protocol for Service or Intertribal Organizations

Organizational Characteristics

We would like to start by learning a bit about your organization or department.

Describe your programs and services related to museums, cultural centers, and/or libraries. What technical assistance and support around program development and identifying grant funding opportunities do you provide member organizations?

Capacity and Technical Assistance

Given your knowledge of the field, we would love to learn a bit more from you about what you think the opportunities and challenges are for organizations in this space.

Think about your members’ organizational development over the next five years. What are the greatest opportunities for growth, raised visibility, or greater community impact? What are your most significant challenges? How best can your organization or department help members realize opportunities and overcome challenges?

Describe your organization’s current ability to serve your members’ needs. What partnerships and collaborative relationships currently exist that can support member organizations to strengthen their cultural entities such as libraries, museums, and archives?

IMLS provides grant funds to Native American Tribes, Alaska Native communities, and nonprofits that serve primarily Native Hawaiians to advance libraries, museums, and archives that serve Indigenous communities. What recommendations might you have for IMLS to improve its programmatic reach and help build capacity in these communities?

COVID-19

Over the last two years, we all have been responding to the COVID pandemic.

Describe how COVID impacted your organization or department. How did your programming and services change and for how long? What are changes made during COVID that you will continue past the pandemic? How do they impact how you fund or support Native American, Alaska Native, or Native Hawaiian communities?

Interview Protocol for Tribal and Native Hawaiian Leadership

Organizational Characteristics

We would like to start by learning a bit about your community.

Which types of cultural institutions does [Name of the Tribe or Organization Name] support?
1. Describe the programs and services associated with that museum, archive, library, and/or cultural or community center.
2. What is the annual operating budget?
3. How are these programs funded?

Reflect on a recent experience at the museum, archive, library, and/or cultural or community center that stands out for its quality. Please consider only one event. Describe that experience.
1. What made it stand out?
2. Who was involved in making that experience memorable?
3. How could the organization improve on the experience?

Think about your [Name of Tribe, Organization Name]'s organizational development over the next five years. What are the greatest opportunities for growth, raised visibility, or greater community impact? What are your most significant challenges? [prompts: What should you be doing more of? What should you be doing less of? What should you be doing that is completely different?]

Community Engagement

We would love to get a better sense of what roles the institutions you are referencing play in your community.

Describe the role the museum, archive, library, and/or cultural or community center plays in the community. How engaged are [Tribal citizens/Native Hawaiians] in the programs and services offered? How does the museum, archive, library, and/or cultural or community center serve visitors to the community who are not [Tribal citizens/Native Hawaiians]?

Grants and Fundraising

Part of this evaluation’s goal is to better understand the funding landscape for organizations like yours.

Please explain the ways in which your staff seek grant funding for museums, archives, libraries, and/cultural or community centers. Do they seek funding from public funders, private foundations, or other sources? Do they solicit charitable contributions from individual donors?

COVID-19

Over the last two years, we all have been responding to the COVID pandemic.

Describe how COVID impacted your organization or department. How did your programming and services change and for how long? What are changes made during COVID that you will continue past the pandemic?

Closing Questions

We would like to end our time together with a couple of high-level questions to help us better understand what recommendations to share with IMLS.

If you had three wishes for the continued development of [Name of Organization]'s work in the community, what would they be?

Interview Protocol for IMLS Staff

Outreach and Technical Assistance

We would like to start by learning about IMLS approaches to outreach and applicant engagement.

Describe the nature of the outreach IMLS performs with Native American, Native Hawaiian, and Alaska Native potential applicants.
1. What outreach do you find works well to encourage Native American, Native Hawaiian, and Alaska Native organizations to apply?
2. How can IMLS improve its outreach to Native American, Native Hawaiian, and Alaska Native organizations?
3. How has IMLS worked with intertribal organizations and service organizations to reach potential applicants?
4. How has IMLS used social media and other forms of communication to promote its grant-making activities in connection with these four programs?
Describe how IMLS staff assist potential applicants with preparing proposals.
1. How often do potential applicants seek support and technical assistance in the pre-application stage? In what ways are the Native American, Native Hawaiian, and Alaska Native applicants different compared with other applicants? In what ways are they the same?
2. What do you consider to be the most valuable assistance IMLS staff can provide these potential or actual applicants?

Peer Review Feedback

We next would like to learn a bit more about the reviewer framework and how unsuccessful applicants receive feedback.

Describe how IMLS follows up with those applicants that were unsuccessful.
1. How often do applicants request consultation regarding the outcome of their grant application?
2. How does IMLS staff use reviewer comments to support technical assistance with applicants?

Program Changes

In order to best understand the programs in their current iterations, we would also like to learn more about program changes over time.

Describe how IMLS grantmaking to Native American, Native Hawaiian, and Alaska Native organization has evolved over time.
How does IMLS staff currently assess its grant-making activities? What does IMLS consider when making changes to its grantmaking?

Grantee Engagement

We want to learn more about the experience of grantees after they accept their awards.

When thinking about the post-award process, describe opportunities for grantees to better engage with IMLS staff to strengthen the impacts of their projects. What are the greatest challenges grantees face in implementing a successful project? How can IMLS improve its post-award engagement with grantees?

Grants and Fundraising

Part of this evaluation’s goal is to better understand the funding landscape for organizations served by IMLS programs.

Describe the landscape in which Native American, Native Hawaiian, and Alaska Native organizations develop and implement projects funded by IMLS. Describe how other public agency and/or private foundation funding is leveraged to successfully implement the projects.

Impact of IMLS Funding

Describe what you see in terms of how IMLS funding impacts Native American, Native Hawaiian, and Alaska Native communities. What can IMLS do differently to increase its impact on Native American, Native Hawaiian, and Alaska Native communities? What are the greatest opportunities available for Native American, Native Hawaiian, and Alaska Native organizations to strengthen their impact?

Closing Questions

We would like to end our time together with a one last question to help us develop recommendations from our findings.

Describe opportunities for IMLS to improve its grantmaking in Native American, Native Hawaiian, and Alaska Native communities.

[Closing language for all interviews]

Thank you for taking the time to speak with us and offer your thoughts and perspectives.

As we explained earlier, your much-appreciated contribution will help inform our broader evaluation IMLS grant programs for Native American, Native Hawaiian, and Alaska Native communities. Our findings will be published in early 2023, and we will be sure to share a copy of our report with you when it is publicly available. Please do not hesitate to reach out if you have questions after this call, about the interview or the broader evaluation approach, or to provide additional information that may be relevant. Thank you again for your time!

Appendix D. Outline of Virtual Convening of Eligible Applicants

A convening of 30 eligible applicants will incorporate large group discussion and small group breakout sessions. Using Appreciative Inquiry, the convening will surface organizational strengths, define outcomes for grantees resulting from grant funding, and construct a vision for an ideal grant program. Appreciative Inquiry provides a strengths-based, positive approach to organizational change that is participatory and focuses on opportunities to construct a shared vision for the future.

Consent Language for Participants

The following will be shared during the Welcome.

Your participation in this virtual convening is completely voluntary. You are welcome to leave at any point for any reason and you may decline to engage in any discussion. Your responses will be used to inform our understanding of the IMLS grant programs and their role in supporting grantee outcomes, the landscape of Native American, Native Hawaiian, and Alaska Native museums and libraries, and opportunities for further attention by IMLS and others.

We will be taking notes during this convening and would like to record the conversations to ensure we are accurately capturing the information you provide. We will take all possible steps to protect your confidentiality. Additionally, we will not quote anything you say unless we receive your express, written consent. The notes and the recording will be accessed only by the Kituwah Services evaluation team, all of whom have signed confidentiality pledges. Individual response data will not be shared with IMLS except in de-identified aggregate/group form.

Your decision to participate will not affect your interactions with the Federal Government, including IMLS grant programs.

We want to be sure that you freely consent to participate in this convening and that you are aware that you are not obligated to answer any questions you do not wish to. If you do not consent to participate, please leave at this time.

IMLS Appreciative Inquiry Summit Agenda

The following is the run-of-show outline of the virtual convening. Outreach will be conducted with interview and survey participants, informed by the first wave of information collection.

Welcome – 2 minutes
Appreciative Inquiry introductions – 15 minutes
Summit Overview, Outcomes and Process – 3 minutes
A Brief Introduction to Appreciative Inquiry and its 4-D Model of Discover, Dream, Design, and Destiny¹ – 10 minutes

Discovery – 40 Minutes

Evaluation Prompt	Activity type
Who are we at our best?	Small group discussion
Describe the outcomes associated with grant funding	Small group discussion
Positive Core Map² (themes from small group discussions)	Large group discussion

Break – 10 minutes
Dream – 30 minutes

Evaluation Prompt	Activity type
Consensus visioning for an ideal museum and library grant program	Large group discussion
Opportunity Map³ (themes from small group discussions)	Small group discussion

Design – 30 minutes

Evaluation Prompt	Activity type
High-impact program design elements (e.g., strategies, processes, best practices) best suited to Native American/Native Hawaiian organizations	Large group discussion

Destiny – 30 minutes

Evaluation Prompt	Activity type
Generation of possible actions	Small group discussion

Large Group Closing – 10 minutes

1 Appreciative Inquiry’s 4-D model of Discovery, Dream, Design, and Destiny anchors the process of inquiry and change. The Discovery phase identifies and appreciates the best of “what is,” focusing on peak times of organizational excellence. The Dream phase envisions the future that emerges from grounded examples from a positive past. The Design phase creates the ideal organization in order to achieve the dream. The Destiny phase marks the end of the Discovery, Dream, and Design phases and the beginning of an ongoing process of adopting an appreciative learning culture.

2 The Positive Core Map is part of the Discovery phase of Appreciative Inquiry and a process in which strengths are identified and mapped across a broader system.

3 An Opportunity Map is a process that considers strengths and identifies opportunities to enhance an organization and the broader system in which it operates.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Boyd, Claire
File Modified	0000-00-00
File Created	2022-06-23