Supporting Statement A.BR Audio Guide

Supporting Statement—Section A

Evaluation of the Impact of the

National Endowment for the Arts’ and the Institute of Museum and Library Services’

Distribution of Big Read Audio Guides to Public Libraries

Background and Context

The Big Read is an initiative of the National Endowment for the Arts (NEA) and the Institute of Museum and Library Services (IMLS) in partnership with Arts Midwest, designed to revitalize the role of literature in American culture by providing citizens with the opportunity to read and discuss a single work of fiction within their communities. Through Arts Midwest, the NEA awarded $5,000-$20,000 grants to 189 communities around the country in Phase I of this ongoing program, 72 in Cycle 1 (January-June 2007), and 117 in Cycle 2 (September-December 2007). The first cycle of Phase II is scheduled to begin in January 2008.

In addition to direct grants to participating communities, the Big Read also provides NEA-developed program resources such as a web site, Reader's and Teacher’s Guides, and Audio Guides for each title with commentary from public figures, artists, and educators. Preliminary findings from the Cycle 1 external evaluation (OMB Control Number: 3135-0121) indicate that these Big Read resources were extremely well-received, and that they lent credibility, status, and support to communities and helped them attract audiences to events and reading activities. The audio guides provided rich, contextual information on books and authors, in a form accessible to both readers and non-readers. Findings also indicated that public libraries play a pivotal role in the Big Read program.

To take full advantage of that role, and the vital role libraries traditionally play in local communities, the Institute of Museum and Library Services (IMLS) has contributed additional funds for Phase II of the Big Read (January-June 2008) for the distribution of Big Read Audio Guides to 16,500 public libraries across the U.S. This widespread dissemination has the potential to raise awareness of the Big Read among public libraries and communities throughout the country, make Big Read materials available to a larger audience, and enable public libraries and communities to participate in the Big Read program without formally applying for a full grant.

The study proposed here is designed to assess the impact of the distribution. It includes a brief initial survey for all public libraries, to be included in the Big Read Audio Guide mailing; a follow-up online survey for those returning the initial survey; and a brief telephone interview with two small sub-samples: one of non-respondents to the initial survey, and one of non-respondents to the online survey. The non-respondent surveys seek to understand whether non-responding libraries had different characteristics (e.g., size, location, etc.) and patterns of Audio Guide usage than those that responded.

1. Justification

A.1. Necessity of Information Collection

As federal agencies, the National Endowment for the Arts and the Institute for Museum and Library Services evaluate their programs on a systematic basis. The NEA, pursuant to its mandate to “support projects and productions that will encourage public knowledge, education, understanding and appreciation of the arts” (20 USC, Section 954), entered into an agreement with Arts Midwest to sponsor an evaluation of this arts initiative. A private evaluation firm, Rockman et al (REA), was selected to conduct the program evaluation of the Big Read initiative.

REA was selected to conduct this companion study. The study will provide specific feedback on the reception and use of the Audio Guides, and the impact of this auxiliary effort in promoting the Big Read and drawing readers to Big Read titles and authors. These data will in turn contribute to the broader national study of the Big Read and its success in expanding the audience of those who read literary works for pleasure and enlightenment, and in supporting libraries and other community organizations in bringing communities together to foster literary reading.

A.2. Needs and Uses of the Data

Initial Survey (paper). In collaboration with the IMLS and the NEA, REA has developed a brief paper survey to be included in the distribution to all libraries, along with a stamped return envelope addressed to REA. This brief survey will ask the respondent group whether they have heard about the Big Read and how they might use the Audio Guides. The instrument also requests basic descriptive information about the library—whether, for example, it serves a predominately urban, suburban or rural area; whether it is a main or branch library; and how respondents would characterize typical patronage and circulation rates. Responses will verify receipt of the materials, and through a request for descriptive and contact information, provide a sample for a follow-up survey.

Follow-Up Survey (online or email). To find out how libraries actually use the Audio Guides, REA will invite, via email, a stratified sample of those who completed the initial survey to take a brief online follow-up survey, two-three months after receipt. Forced-choice items will ask whether respondents added the guides to their CD collection, paired them with books, books on tape, or movies by or about the same authors and titles, shared them with schools, used them in book discussion groups and reading programs, or displayed them in the library. Items will also ask whether respondents observed any greater-than-normal interest in the Big Read titles, to what extent patrons checked out Audio Guides and companion books, books on tape, or DVDs; and whether respondents accessed any other Big Read resources (e.g., visiting the website, downloading Reader’s Guides). The goal is to determine the impact of the distribution on libraries, as well as on patrons reached indirectly by Big Read efforts.

Telephone Follow-up. REA will also telephone two small samples composed of non-respondents to the initial and follow-up online surveys. These interviews will allow us to understand whether the libraries that did not respond to our surveys had similar characteristics to those that did and whether their patterns of Audio Guide usage were similar.

A.3. Uses of Information Technology

The follow-up survey will be conducted using online survey technology. The survey will be hosted on the REA server, with links on the Big Read national website.

A.4. Efforts to Identify Duplication

This is the only evaluation of the NEA’s and IMLS’s Big Read Audio Guide distribution to public libraries. Therefore there are no existing efforts that this study might duplicate. We have also taken care not to duplicate any information that libraries that are Big Read grantees are required to report to Arts Midwest and the NEA as part of their routine record-keeping and reporting.

A.5. Minimizing Burden in Small Businesses or Small Entities

The respondents to the evaluation include no small businesses. We are minimizing the burden on all public libraries, including smaller libraries, by asking that they only complete one, or in some cases two, very brief surveys.

A.6. Consequences of Less Frequent Data Collection

This is a one-time data collection effort to determine the effectiveness of the distribution of Big Read Audio Guides to public libraries and is critical to determining whether such programs are pursued in the future. All instruments will be brief and the burden on respondents will be modest.

A.7. Special Circumstances

None of the special circumstances listed in the instructions for completing this supporting document apply to the evaluation of the IMLS Big Read Audio Guide Distribution to Public Libraries.

A.8. Public Comment and Consultations outside the Agency

A notice was published in the Federal Register on Aug. 8, 2007 (page 44589)to solicit comments on the evaluation of the IMLS Audio Guide distribution prior to submission of this OMB clearance request. No public comments were received at the NEA in response to the notice.

A.9. Payment to Respondents

No payments are planned.

A.10. Assurance of Confidentiality

To ensure confidentiality of information, only site and respondent ID numbers will be housed with each form of data for the purposes of matching data across instruments. No names or other identifying information will be kept with the data that will be used for analysis. A separate file with names of individuals or sites and matching ID numbers will be kept on a separate computer under password protection. The external evaluator will compile all data, which will be stored in a secure location. No identifying information will be used in the reporting of findings or provided to any grantee or host organization.

A.11. Questions of a Sensitive Nature

There are no questions in the attached instruments that respondents would consider sensitive.

A.12. Estimates of Hour Burden for Data Collection

Data Collection Protocol	Method of Administration	Burden Hour Estimate
Initial Survey	Paper, included, with return envelope, in Audio Guide mailing	3 minutes x 12,375 (16,500 X 75%)= 619 hours
Follow-up Survey	Online	8 minutes x 1,500 (2000 X 75%)= 200 hours
Non-Respondent Telephone Interviews	Telephone	4 minutes x 90 (120 X 75%) = 6 hours

A. 13.

There are no costs to respondents other than that of their time to respond. Paper surveys will include stamped, addressed envelopes.

A. 14.

The total cost of this evaluation to the Federal Government is $38,375.

A.15

Not applicable

A.16. Timeline and Publication

1. Develop evaluation instruments for impact and implementation evaluation. Proposed instruments include:

1. Audio Guide Initial Survey

2. Audio Guide Follow-up Survey

3. Follow-up interview Protocols

2. Administer surveys. All 16,500 public libraries will be sent Audio Guides on Jan. 15, 2008 and asked to return the enclosed, informational survey. Online follow-up surveys will be administered two-three months afterward, to a sample of those (N=2,000) who complete the initial surveys and provide email addresses; respondents will receive the URL to the survey via email.

REA will also call two additional small sub-samples of non-respondents, first of libraries (whose contact information is readily available through online sources) that did not return the initial survey (N=90); and, second, of libraries that completed the initial survey but not the online follow-up (N=30).

3. Conduct quantitative and qualitative analyses. The primary method of data analysis for the initial and follow-up online surveys and for quantitative indices on the telephone interviews will be descriptive. Analysis will include calculating means, standard deviations, and frequencies of responses; where appropriate, we may conduct simple cross-tabs, mean comparisons, or correlations to assess relationships among particular items.

For any open-ended questions or voluntary responses during the interviews, we will look for recurrent themes and code the qualitative data accordingly. For analysis purposes, we will sample responses to open-ended items on the initial and follow-up surveys since the analysis of large numbers of open-ended data are time and cost-prohibitive. We will sample every 10th response and create codes for the sampled responses. To ensure representativeness of the sample to the pool of responses, we will compare the characteristics of the libraries whose responses were sampled to those of the pool as a whole. 

4. Submit status report. REA will submit a brief status report to the NEA and IMLS following analyses of the initial survey returns. This report will include summaries of progress to date, counts and characteristics of survey returns, and a summary of proposed uses of the Audio Guides. In the case of very low response rates, REA will make recommendations to the NEA and IMLS about other avenues of reminding libraries to return surveys—e.g., postings from professional organizations (e.g., ALA) or in their newsletters.

5. Write final report. The final report, submitted in June 2008, will include the following elements:

• Executive Summary of major findings, developed in collaboration with the NEA and IMLS.

• Description of methodology, data collection goals, research questions, and sampling procedures.

• Context of the Big Read auxiliary initiative.

• Findings, including the following:

• public libraries’ overall response to the Audio Guide distribution;

• general awareness of the Big Read (based on initial survey responses indicating whether libraries had heard about the program);

• the use of the resources by libraries, including an analysis of use based on size, patronage, location, and other factors;

• the effectiveness of the distribution in generating interest among patrons in Big Read titles and related materials;

• the effectiveness of the distribution in generating interest among libraries in the Big Read and other Big Read resources;

• implications and recommendations for using widespread distribution as a means of raising awareness of the Big Read, making materials available to non-grantee audiences, and defining a role for participation aside from formal grants.

A.17. Exemption to Display Certification

Not seeking exemption

A.18. Exceptions to Certifications

There are no exceptions to the certifications set forth in Item 19.