IMLS_Museums_for_America_Evaluation_Study_JustificationA72710FINAL

IMLS Museums for America Evaluation Study

Part A. Justification

A1 Circumstances Requiring Collection of Information

Need

Through the Museums for America Program, the Institute of Museum and Library Services awards grants to a range of scientific, historical, youth, and general museums to support museum projects and activities that enhance their capacity to serve their communities. From 2004 to 2007 IMLS awarded grants in three categories: a) Serving as Centers of Community Engagement, b) Sustaining Cultural Heritage, and c) Supporting Lifelong Learning. Based on feedback from applicants, institutions were unclear of the distinctions among the categories and IMLS staff found overlap in the project applications, and three new grant categories were established in 2008. The three new categories are designed to support museum efforts in a) engaging communities, b) stewarding collections and holdings, or c) building institutional capacity. These categories are nested within the overall goals of the Museums for American Program, which are to strengthen the ability of museums to serve the public more effectively by supporting high-priority activities that advance their missions and strategic goals.

At the broadest level, anecdotal evidence suggests that many applicants, even those who do not ultimately receive awards, experience benefits in completing the Museums for America grant application. The process of clarifying their mission, conducting strategic planning, and conceptualizing programs and activities in terms of the grant categories appears to have some positive effect on institutional capacity and may enable applicants to secure funding from other sources. Data on the Museums for America application process may help the Institute refine the application process to provide broad value to all applicants.

Beyond understanding the application process, the Institute seeks to understand implementation processes such as partnerships established, effects of cost-sharing requirements, and grant-related activities undertaken. Questions to this end will be addressed to the subset of actual grantees within the larger pool of applicants, about 644 institutions. These data will inform IMLS of similarities and differences that may have emerged over time. To investigate how grant-enabled activities have affected the grantee organizations themselves and their local communities, a series of effects-related questions will be addressed to the subset of grantees with completed projects. Data on implementation processes, activities conducted, and related effects will help the Institute in future grant-making.

To complement these data with greater context and depth and to provide insights into the study’s research questions not amenable to quantitative investigation, RMC will conduct six on-site case studies, two in each of the three grant categories. The research team has established selection criteria to narrow the field of potential sites and will seek to balance final selection across characteristics that will admit exploration of a full range of contextual factors such as geographic region, size of the community served by an institution, and institution size. Final selection of case study site visits will be made in consultation with the IMLS program staff.

A2 Indicate How, by Whom, and for What Purpose the Information Is to Be Used

The Institute for Museum and Library Services will be using the information gathered from this study to inform, refine, and improve future applications and agency grant-making and to describe the effects of grants, consistent with the Office of Management and Budget’s call for increased emphasis on rigorous, independent program evaluations (OMB M-10-01). Ultimately, this information will be used to strengthen the design and operation of the program to achieve intended effects as well as possible. IMLS will be the sole user of information gathered from this study.

A3 Use of Information Technology to Reduce Burden

RMC will use existing administrative data to collect data from 2008-2010 applicants, including grantees; this will eliminate the need to ask questions of applicants that they have answered elsewhere. For example, RMC’s review of these data will permit analyses of applicants based on characteristics such as geographic location, type of projects proposed, institution size, and so on. Further, pre-populating the online survey with these administrative data will aid respondents in recalling information relevant to their application experience, and reduce the burden of completing the survey.

To solicit data directly through the online survey, RMC will use traditional and electronic methods. Initially, all applying institutions will receive letters addressed to the most current IMLS contact name but also include “Current Director” in the event the contact person is no longer there. The postcard will describe the study and encourage participation, and direct the appropriate staff member to the online survey with a protected password. Non-respondents will receive email notification of the online survey code and password. A pilot test will be conducted to reduce technical problems with the survey administration, which will reduce data entry errors, costs, and time associated with paper survey forms. Stored data will be converted to PASW (formally SPSS software) files for analysis.

RMC’s information technology director performs a security self-assessment patterns on baseline security assessments as outlined in the National Institute of Standards and Technology guidelines (Special Publication 800-53). Security requirements include updating all applicable security controls, continuous monitoring, third-party vendor security considerations, authorizing connections, physical security considerations, creating off-site backups, and purging retired equipment. An independent security review entails a vulnerability scan of the internal network servers, routers, and selected workstations and an assessment of perimeter security including firewall configuration and administration.

A4 Avoidance of Duplication

Although all Museums for America grantees submit annual reports on their activities, no formal or systematic process currently exists to analyze the extent or effects of grantee activities.

Making use of existing IMLS database, which contains data on types of applications, demographic details on institution sizes, and the dates of applications and awards requested will enable an analysis of administrative data which the IMLS does not have the resources to conduct. The data to be collected on grantee activities will be new data and will not duplicate information available elsewhere.

Currently, no data are collected on the Museums for America application process. Again, the only data to be collected will be new data and will not duplicate information available elsewhere.

A5 Methods to Minimize Burden on Small Entities

If requested, RMC will mail hard (paper) copies of the online survey to entities that do not have computer or Internet access.

A6 Consequences of Not Collecting Information

The educational and social role that non-profit museums play is not fulfilled by other kinds of institutions. Improving the grant-making process and effectiveness of grants through Museums for America is a beneficial service.

A7 Explain Any Special Circumstances

None of the special circumstances apply to these data collections.

A8 Consultation outside the Agency

RMC is not consulting with any other agencies in this study. No entity is currently studying or collecting these Museums for America data.

A9 Payment of Gifts

No payments or gifts will be made to museum personnel who cooperate with RMC collecting materials for this study. Their participation is strictly voluntary and uncompensated. However, responding grantees will receive a copy of the executive summary of RMC’s final report to the IMLS.

A10 Assurances of Confidentiality

All survey respondents will be assured of confidentiality in the survey instructions. Online survey data will not be reported on an individual or single entity; all data will be aggregated. Respondents will provide information to RMC’s secure website where it will be downloaded onto a password-protected service in RMC’s office in Portsmouth, New Hampshire.

A11 Justification of Sensitive Questions

No questions of a sensitive nature will be asked of or about any individuals. Data collected will concern program activities, not individual actions.

A12 Estimates of Respondent Hour Burden

The section of the survey concerning the application process (addressed to approximately 644 applicants from the 2008 to 2010 funding cycle) is estimated to take most respondents about 15 minutes and an additional two minutes to respond to questions regarding unfunded applications, if applicable.

The section concerning implementation process, particularly grant activities (addressed to the 761 institutions awarded grants from 2004 to 2009) is estimated to take most respondents 10 minutes. The section concerning short term effects (addressed to a subset of grantees, totaling 589 institutions with closed grants) is estimated to take 8 minutes. The final section concerning long terms effects (addressed to a yet smaller subset of grantees, totaling 304 institutions) and including those with grants awarded and completed from 2004 to 2007) is estimated to take most respondents an additional 8-10 minutes dependingon the length of their responses to three open-ended questions.

Sixty-six institutions will be asked to complete all sections of the survey (including both perceptions of recent application process and completed grants), and is estimated to take 45 minutes.

A13 Estimates of Annual Cost Burden to Respondents

This one-time collection of information will not entail direct monetary costs to respondents for their participation other than the time required to locate and submit the requested information electronically. If electronic submission creates problems, respondents might incur a small cost for faxing or mailing information in.

We anticipate a range of different response rates (30-70%) depending on an institution’s recent history with the MFA grant program and the feasibility of museum staff being knowledgeable about the targeted awarded project. Within the museum community, receiving a grant from IMLS is viewed as prestigious. Generally, IMLS experiences high response rates from any data collection efforts from their grantees. Keeping IMLS informed of museum needs, activities and operations ultimately serves the museum community by ensuring IMLS is fulfilling their mission to support museums and libraries. Although there is little incentive for museums who have never received funding from IMLS to respond to the survey, their voices are just as important to improve the MFA grant making program. The table below presents the various sample groups, the anticipated response rate, and the rationale behind the estimated rate.

Description of Sample Pool	Est. Response Rate	Rationale
Applied 2008-2010 funding cycles and at least one unfunded	30%	Very little interest due to no previous MFA awards from 2004-2009 and at least one recent application was unfunded
Did not apply 2008-2010 funding cycles; Have a completed grant finished in 2007 or before	40%	Little interest since the museum has not recently applied for a MFA grant and possible staff changes since the targeted project was completed
Applied 2008-2010 funding cycles and at least one unfunded; have a completed grant finished in 2007 or before.	40%	Little interest since recent application(s) were unfunded and possible staff changes since the targeted project was completed
Applied 2008-2010 funding cycles and all funded; have a completed grant finished in 2007 or before.	50%	General interest since recently applied but possible staff changes since the targeted project was completed
Applied 2008-2010 funding cycles and at least one unfunded; only grant ever received is still in process	60%	High interest due to an award project that is currently in progress
Applied 2008-2010 funding cycles and at least one unfunded; have a recent grant completed	60%	High interest due to recent application(s) and recent completed award project
Did not apply 2008-2010 funding cycles; have grant project in process	60%	High interest due to an awarded project that is currently in progress
Did not apply 2008-2010 funding cycles; Have a recent grant completed	60%	High interest due to an awarded project recently completed
Applied 2008-2010 funding cycles and all funded; have recent project completed	70%	High interest since applied and funded recently and had a recent project completed
Applied 2008-2010 funding cycles and all funded; have grant project in process	70%	Very high interest due to recent funded applications and one project in process

The table below presents the number of museums to be contacted and the estimated response rate for each sample group. The response rate is applied to the sample groups and the estimated number of respondents is indicated in the appropriate survey sections. The last table column presents the estimated total time it will take a respondent to complete the survey.

			Application Study (2008-2010 funding cycles Applicants only)		Project Activities and Effects Study on ONE Awarded Project
Description of Sample Pool	Number of Museums Contacted/ Est. Response Rate	A. Museum Back-ground	B. Perception of Application Process	C. Perception of Unfunded Application(s)	D. Project Activities	E. Short Term Effects	F. Long Term Effects	Est. total time to complete survey sections (in min.)
Applied 2008-2010 funding cycles and at least one unfunded	250/30%	75	75	75				20
Applied 2008-2010 funding cycles and at least one unfunded; only grant ever received is still in process	34/60%	20	20	20	20			30
Applied 2008-2010 funding cycles and at least one unfunded; have a recent grant completed only	45/60%	27	27	27	27	27		38
Applied 2008-2010 funding cycles and at least one funded; have a completed grant finished in 2007 or before.	66/40%	26	26	26	26	26	26	45
Applied 2008-2010 funding cycles and all funded; have grant project in process	106/70%	72	72		72			27
Applied 2008-2010 funding cycles and all funded; have recent project completed	80/70%	56	56		56	56		35
Applied 2008-2010 funding cycles and all funded; have a completed grant finished in 2007 or before.	63/50%	32	32		32	32	32	43
Did not apply 2008-2010 funding cycles; have grant project in process	32/60%	19			19			12
Did not apply 2008-2010 funding cycles; Have a recent grant completed	160/60%	96			96	96		20
Did not apply 2008-2010 funding cycles; Have a completed grant finished in 2007 or before	175/40%	70			70	70	70	28
TOTAL Museums	1011	493	308	148	418	307	128

It is estimated that a respondent will take 12-45 minutes in order to complete the appropriate survey sections. The estimated number of respondents and time to complete the survey specified in the above table results in a total of 228 hours. Using $30 per labor hour, calculates the total cost to be $6,842.

A14 Estimates of annual Cost Burden to Federal Government

This is a one-time survey. No costs will be ongoing.

A15 Program Changes in Burden and Cost Estimates

This request is for new information collection. No changes have yet been needed.

A16 Plans and Schedules for Tabulation and Publication

Reports will be prepared for publication based on data from the online survey and analysis of six individual case studies. Online survey data will be presented in aggregated tables with possible breakouts by such variables as institution size and project activities. Case study data will not be tabulated. Reports will be publically available and accessible on IMLS’s website.

A17 Expiration Date Omission Approval

Not applicable. All data collection instruments will include the OMB data control number and data collection expiration date.

A18 Exceptions

Not applicable. No exceptions are requested.

4