IMLS Evaluation of Four Grant Programs Serving
Native American, Native Hawaiian, and Alaska Native Communities
IMLS has four active grant programs designed to serve Native American, Native Hawaiian, and Alaska Native communities: Native American Library Services: Basic Grants; Native American Library Services: Enhancement Grants; Native American/Native Hawaiian Museum Services Program; and Native Hawaiian Library Services. The agency has contracted with Kituwah Services, LLC, a tribally-owned business, to conduct a mixed-methods evaluation of all four programs to determine how well the agency's grantmaking aligns with the needs of communities served by these specific programs; to lay a foundation for improving the quality, reach, and impact of the agency’s grant programs in the future; and to inform efforts to increase the organizational capacity of eligible applicants to submit high-quality grant applications and of awardees to complete their award responsibilities successfully. This is a new information collection request, and the data to be collected are not available elsewhere. The information collection activities are planned for July 2022 through November 2022.
The evaluation is being undertaken in connection with IMLS’s statutory authority to conduct analyses, identify trends, and measure the contribution of its programs (20 U.S.C. § 9108; attached at the end of this document).
IMLS has established goals and objectives for each of these four grant programs, and these are in turn connected to agency-level strategic goals. All goals and objectives are articulated in the FY2022-2024 Notice of Funding Opportunity (NOFO) for each program, available through the links below:
Specific eligibility criteria vary by program, as do the amounts of funding available, cost-share requirements, and the anticipated periods of performance. These details are specified in each NOFO.
IMLS began awarding grants in its Native American Library Services: Basic Grants and Enhancement Grants and the Native Hawaiian Library Services programs in 1998, and its Native American/Native Hawaiian Museum Services Program in 2005. Over the course of these 24 funding cycles, IMLS has awarded is $110,226,316 in support of 6,128 projects (see Table A.1 for a grant program-specific breakdown).
Table A.1. Number of Grants and Funds Awarded by GRANT Program
IMLS Office |
Fiscal Years |
Grant Program |
Number of Grants Awarded |
Funds Awarded |
Office of Library Services |
1998-2021 |
Native American Library Services: Basic Grants |
5,322 |
$33,750,273 |
Office of Library Services |
1998-2021 |
Native American Library Services: Enhancement Grants |
368 |
$47,152,510 |
Office of Museum Services |
2005-2021 |
Native American/Native Hawaiian Museum Services Program |
392 |
$19,387,867 |
Office of Library Services |
1998-2021 |
Native Hawaiian Library Services |
46 |
$9,935,666 |
|
|
TOTAL |
6,128 |
$110,226,316 |
https://www.imls.gov/grants/awarded-grants. All publicly available awarded grants data (1998-2021) was aggregated for each of the four grant programs. Data were accessed on January 26, 2022.
This evaluation is designed to determine how well these four programs align with the needs of the communities they are designed to serve; to lay a foundation for improving the quality, reach, and impact of the agency’s grant programs in the future; and to inform efforts to increase the organizational capacity of eligible applicants to submit high-quality grant applications and of awardees to complete their award responsibilities successfully. This work will include identifying gaps in reaching eligible applicants; examining the differences among those who apply and are funded, those who apply and are not funded, and those who do not apply; and identifying ways to enhance engagement with applicants and grantees.
Stakeholders likely to be interested in the findings from this research are diverse and many, and their specific interests are varied. Likely audiences include those eligible to apply for the four grant programs, Congress, the Administration, other funders of eligible entities, local and national partners, and the general public. Relevant associations, such as the Indigenous Peoples Museum Network (IPMN); Association of Tribal Archives, Libraries, and Museums (ATALM); American Indian Alaska Native Tourism Association (AIANTA); American Alliance of Museums (AAM); American Library Association (ALA); and the American Association for State and Local History (AASLH), also are likely to be interested in the evaluation’s findings because of their direct relationship to Native communities, libraries, and museums. More specifically, we anticipate that Native American, Native Hawaiian, and Alaska Native communities will use the data to understand the capacity limitations of potential applicants and use that information to support improvements to their future applications. The evaluation is structured with these large, diverse stakeholder audiences in mind, and the evaluation team will look for opportunities to effectively convey insights and stories targeted for their consumption and use. IMLS will publicly post the final evaluation report on imls.gov and host at least one public virtual meeting to share and invite discussion of the findings.
This is not a research study; therefore, the intention is not to generalize these data and findings to the broad population of Native American tribes and nonprofit organizations serving primarily Native Hawaiians nor to Native American and Native Hawaiian museums or libraries outside those eligible to apply to these IMLS funding opportunities. This also is not an audit of grantees nor their individual performances.
Kituwah Services will use the SurveyMonkey® platform to administer the surveys. The surveys will be compliant with Section 508 of the Rehabilitation Act of 1973 and its amendments. Respondents will be given the option of receiving a mailed paper survey with a return envelope should they prefer that medium.
The Kituwah Services team will collect interview and virtual convening data via Microsoft Teams, with concurrent notetaking by an evaluation team member and audio recording through the telecommunications software. Before recording, the evaluation team will obtain each interviewee’s consent to be recorded.
All survey, interview, and virtual convening data will be saved to an encrypted network drive, with password-controlled access limited to the evaluation team. Access will be available only on-site or via secure remote access through password-protected computers. Audio files will be stored on a secure drive accessible only to the Kituwah Services evaluators and to subcontractors Seminole Heritage and Urban Institute. The files will be destroyed at the end of the evaluation.
The results of the project will be shared with existing and prospective grantees, policymakers, key partners, funders, academics, and others as well as the public via a dissemination strategy, consisting of publicly posting the final evaluation report on imls.gov and hosting at least one public virtual meeting to share and invite discussion of the findings.
IMLS works closely with the Association of Tribal Archives, Libraries, and Museums (ATALM), a leader in this space. ATALM produces critical field-building research in the form of annual reports and surveys that document the assets and needs of tribal museums and libraries. While these data are useful, their work does not address the efficacy of IMLS grantmaking in meeting community needs. For example, no existing data source includes information from eligible non-applicants about why they have chosen not to apply to IMLS funding opportunities.
The project is analyzing secondary data, particularly IMLS administrative data and project performance reports, and this has informed the evaluation team’s development of the proposed primary information collection instruments to avoid duplication of information. Performance reports provide important information about funded activities, challenges encountered in implementing projects, and outcomes achieved; however, they do not provide sufficient information to address the evaluation’s research questions and are not structured to address any consistent questions related to the IMLS’s contributions to project outcomes. To meet the goals of the evaluation, new information collection is required.
There are no small businesses involved in this information collection, but we do expect some respondents to represent small libraries or museums with limited personnel capacity. To not burden these entities unnecessarily, the evaluation will provide clear, concise instructions and ensure the information collection process limits the imposition on personnel time, including the need to follow up with requests for additional information. The instruments are designed to minimize respondent burden: the grantee survey is designed to take 30 minutes (0.50 hr), the eligible non-applicant survey is designed to take 15 minutes (0.25 hr), and the interviews will each take no more than 60 minutes. The virtual convening is designed to last no more than 3 hours and will include few small entities. Only one person from each organization will be asked to complete a survey, participate in an interview, or participate in the virtual convening.
The proposed evaluation is the first comprehensive effort to assess all four of IMLS’s grant programs designed to serve federally recognized tribes and nonprofit organizations serving primarily Native Hawaiians. Without this evaluation, IMLS will continue to work without comprehensive, reliable information to answer key questions posed by eligible organizations, policymakers, and other partners.
The proposed information collection activities are consistent with the guidelines set forth in 5 C.F.R. Part 1320 (Controlling Paperwork Burdens on the Public). There are no special circumstances that require deviation from these guidelines.
On January 14, 2022, a 60-Day Federal Register Notice was published at 87 FR 2464-2465. No comments were received.
The evaluation design, including information collection instruments, has been developed in collaboration with an Urban Institute team that brings experience conducting federal evaluations of programs that specifically serve Native American, Native Hawaiian, and Alaska Native communities. The evaluation plan and draft instruments have also been reviewed by subject-matter experts on Native American, Native Hawaiian, and Alaska Native communities, who provided recommendations to improve the availability of additional data, the clarity of questions and instructions, and the need for specific data elements. In March 2022, IMLS hosted a public meeting of federally recognized Native American tribes, Alaska Natives, and nonprofit organizations that serve primarily Native Hawaiian communities and included a session about process and content to direct the inquiry of the information collection and raise awareness of the evaluation to facilitate information collection outreach. This review led to the revision and consolidation of several questions to improve the survey’s clarity and reduce the time burden on respondents.
The analysis of the survey data received will also be informed by an advisory group of subject-matter experts who will provide guidance on interpreting preliminary results of both the primary research (e.g., web-based survey, interviews, virtual convening) as well as the secondary research (e.g., literature reviews, administrative data, census data). The subject-matter expertise solicited will enrich the final product and ensure appropriate framing.
No incentives or other payments or gifts will be offered to survey or interview participants.
The evaluation team will provide respondents with a written explanation of the purpose of the evaluation, assurance of confidentiality, and intended use of the collected data at the beginning of each survey or virtual engagement. Individual responses will be de-identified and publicly reported in aggregate. Evaluators will ask for explicit, written permission from respondents for use of any quotes in the report, although these quotes will not have attribution to an individual or the individual’s affiliated organization. For the full text of the informed consent language, please see Appendix B and Appendix C.
As noted in our response to Question A3, all survey, interview, and virtual convening data will be saved to an encrypted network drive, with password-controlled access limited to the Kituwah Services evaluators and to subcontractors Seminole Heritage and Urban Institute, who have signed confidentiality agreements. Access will only be available on-site or secure remote access, through password-protected computers. The files will be destroyed at the end of the evaluation.
Before delivering any final datasets to IMLS, the evaluation team will remove any personally identifying information (PII)—such as name, address of respondents, their organizational affiliation, or any other characteristics that would make smaller subsets of eligible applicants identifiable—that could permit disclosure or identification of respondents, directly or by inference. Kituwah Services will destroy all personally identifiable information at the end of the evaluation.
No questions of a sensitive nature will be included.
The information collection will use three instruments: web-based surveys, interviews, and a virtual convening. For all three instruments, the burden to 418 respondents is estimated to total 265.50 hours at a cost of $8,991.05. Please see the supporting documentation submitted with this request for approval.
Web-Based Surveys: We expect that 338 respondents, representing grantees and eligible non-applicants, will spend 0.50 and 0.25 hour each, respectively, participating. This results in a total burden of 125.5 hours and a cost of $3,863.52. The burden estimate is based on the following.
IMLS made awards to 331 institutions between FY 2015 and FY2021, and we will contact each of them. We expect a response rate of 50 percent, or 164 respondents, representing museums and libraries in equal numbers.
We estimate that 579 organizations are eligible but have never applied for an award in any of the four IMLS grant programs. We will contact each of them and expect a response rate of 30 percent, or 174 respondents, representing museums and libraries in equal numbers.
Dollar figures are based on U.S. Bureau of Labor Statistics (May 2021) (BLS) figures for Museum Curator and Librarian mean hourly wages.
Interviews: We will conduct 50 interviews, distributed among our external respondent groups. At 1.0 hour each, the interviews will create a burden of 50.0 hours and a cost of $2,356.88. The dollar figure is based on a range of BLS figures of $30.71 to $73.62, reflecting the mean hourly wages of likely respondents.
Virtual Convening: The virtual convening will include 30 participants, representing grantees, unsuccessful applicants, and eligible non-applicants, each spending 3.0 hours. This results in a total burden of 90.0 hours and a cost of $2,770.65, based on BLS figures for Museum Curator and Librarian mean hourly wages.
There are no additional cost burdens to respondents beyond the labor cost of burden-hours described in item A12 above.
The total one-time estimated cost to the Federal Government for this evaluation is $10,216.46, reflecting the contracting agreement with Kituwah Services for the surveys, interviews, and virtual convening).
This is a new information collection request, and there have been no program changes or cost adjustments
The start of information collection will be preceded by sending an introductory Respondent Contact Letter from IMLS to respondents to explain the importance of the evaluation, participation in data collection, and the role of the evaluator (see Appendix A). This will be sent in July 2022, or as soon as OMB approval is received. Table A2 presents the full project timeline.
Table A.2. Project timeline
Project Phase |
Task |
Date |
Launch and evaluation design |
Launch project |
October 2021 |
Design evaluation |
November 2021- March 2022 |
|
Primary information collection |
Submit PRA request to OMB |
May 2022 |
Launch web-based survey and begin interviews |
July 2022 |
|
Complete survey and interviews |
October 2022 |
|
Analyze survey and interview results |
November 2022 |
|
Host virtual convening |
November-December 2022 |
|
Draft interim report and presentation material due to IMLS (primary data) |
January 2023 |
|
Submit an interim report to IMLS |
January 2023 |
|
Deliverables & dissemination |
Deliver final draft report and draft presentation |
March 2023 |
Integrate IMLS feedback into the final report draft |
March-June 2023 |
|
Prepare final report for publication |
July 2023 |
|
Post the final report on imls.gov; host at least one public virtual meeting to share and invite discussion of the findings |
August-September 2023 |
The evaluation team will conduct the survey once, beginning in July 2022 or as soon as OMB approval is received and ending approximately six weeks later.
The evaluation team will contact target grantees, unsuccessful applicants, and eligible non-applicants to initiate the survey, providing information reiterating the purpose of the survey and the use of their responses (see Appendix B). This email message will direct respondents to the survey on an online survey platform (SurveyMonkey®) and will include introductory language and consent language. Respondents must indicate their consent to complete the survey in order to participate. This outreach will briefly explain the purpose of the survey, the use of the data, the value/benefit to respondents, and the short time commitment. This email will also include a link to the survey. Other methods of disseminating the survey will be used, such as leveraging personal connections (through word-of-mouth) of the project’s subject-matter experts and IMLS’s partners including the social media presence of relevant partners. Survey results will be analyzed following completion of the survey, and preliminary analysis should be completed approximately 6 weeks after the end of the survey information collection.
Interviews will begin in July 2022 or as soon as OMB approval is received and end approximately three months later.
The evaluator will contact target respondents to participate in an interview, providing information reiterating the purpose of the interview, explaining the use and safeguarding of their responses, and describing the types of questions that will be asked and inviting them to participate by indicating their availability (see Appendix C). Interview data will be analyzed on a rolling basis and analysis is expected to be completed approximately eight weeks after the end of the interview information collection period.
In January 2023, draft findings from the primary information collection will be included in an interim report (the project’s second interim report) that will be submitted to IMLS and formally presented at a Microsoft Teams briefing and discussion.
The findings from both interim reports (i.e., findings from both the primary and secondary research) will be presented in a final project report, designed to address all the evaluation’s research questions. A draft of this final report will be shared with IMLS in March 2023 followed by an in-person briefing and discussion of findings. After two rounds of review and revision, a final report will be shared with IMLS in July 2023, ready for publication. IMLS will work with Kituwah Services in disseminating the report and findings in August.
IMLS will display the expiration date of OMB approval and OMB approval number on all instruments associated with this information collection, including forms and questionnaires.
No exceptions are necessary for this information collection.
Attachment: United States Code Title 20 – Education, Chapter 72 – Museum and Library Services
Attachment
United States Code Title 20 – Education
Chapter 72 – Museum and Library Services
[As of December 31, 2018, P.L. 115-410. Official version can be found at 20 U.S.C. § 9101, et seq.]
Sec. 9108. Policy research, data collection, analysis and modeling, evaluation, and dissemination
(a) In general
The Director shall regularly support and conduct, as appropriate, policy research, data collection, analysis and modeling, evaluation, and dissemination of information to extend and improve the Nation’s museum, library, and information services.
(b) Objectives
The objectives of the policy research, data collection, analysis and modeling, evaluation, and dissemination of information carried out under this section include the following:
(1) To enhance and expand the capacity of museums, libraries, and information services to anticipate, respond to, and meet the evolving needs of communities and the public, including by identifying trends and developments that may impact the need for and delivery of services.
(2) To provide information and data on the role, value, and impact of museum, library, and information resources, including the identification of trends and potential gaps in the availability and use of museum and library services by their communities and the public;
(3) To measure the effectiveness of museums, libraries, and information services throughout the United States, including the impact of Federal programs authorized under this Act.
(4) To identify indicators and outcomes that can be used to create enhancements to the efficiency and efficacy of museum, library, and information services.
(5) To promote advancement and growth in museum, library, and information services through sharing of best practices and effective strategies in order to better serve the people of the United States.
(6) To facilitate planning for, and building of, institutional capacity in order to improve—
(A) museum, library, and information services at the national, State, local, and regional levels; and
(B) international communications and cooperative networks.
(7) To support and enhance collaborative professional networks and consortia that use shared, meaningful, and actionable data analysis and modeling to advance museum, library, and information services and address community needs.
(c) Authority to contract and enter into other arrangements
The Director is authorized to enter into grants, contracts, cooperative agreements, and other arrangements with Federal agencies, public and private organizations, and other entities with expertise the Director determines appropriate, to further the objectives described in subsection (b) and to carry out the responsibilities under subsection (f).
(d) Consultation and Public Engagement
In carrying out subsection (a) and in furtherance of the objectives described in subsection (b), the Director—
(1) shall conduct ongoing collaboration (as determined appropriate by the Director) and consult with—
(A) State library administrative agencies; and
(B)National, State, tribal, and regional museum and library organizations; and
(2) may also collaborate or consult with –
(A) cooperative networks of geographic- or discipline-based museums and libraries; and
(B) Other applicable agencies, organizations (including international organizations), entities (including entities with expertise in the fields of data collection, analysis and modeling, and evaluation), and community stakeholders.
(e) Assistance to museums and libraries
The Director shall provide technical support and assistance (and other resources, to the extent practicable) to ensure consistency in data reporting and help the museum and library fields with meeting the objectives of this section.
(f) Dissemination
(1) In general –
Each year, the Director shall widely disseminate, as appropriate to further the objectives described in subsection (b) --
(A) the results, data, reports, findings, studies, surveys, and other information obtained under this section;
(B) the means and approaches by which the objectives described in subsection (b) were accomplished; and
(C) information regarding the manner and extent to which collaboration and consultation were conducted, as required by subsection (d).
(2) Formats to be used --
The information described in paragraph (1) shall be shared in formats that facilitate access and ease of use and are searchable.
(g) Authorization of appropriations
(1) In general
There are authorized to be appropriated to carry out this section $3,500,000 for each of the fiscal years 2020 through 2025.
(2) Availability of funds
Sums appropriated under paragraph (1) for any fiscal year shall remain available for obligation until expended.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Eldridge, Matthew |
| File Modified | 0000-00-00 |
| File Created | 2022-06-23 |