Supporting Statement-Section A
Evaluation of the National Endowment for the Arts Big Read Initiative
This request is for clearance to conduct the 2007-2008 program evaluation of the NEA’s Big Read initiative. This evaluation is aimed at assessing the design of the 2007 Big Read program and to assess the program’s impact on literary reading habits in participating communities. Specifically, we are seeking clearance approval for all data collection instruments included in this submission.
We are asking for clearance for the evaluation from March 2007 through June of 2008.
The Big Read is an initiative of the National Endowment for the Arts (NEA), in partnership with the Institute of Museum and Library Services (IMLS) and in cooperation with Arts Midwest, designed to revitalize the role of literature in American popular culture by providing citizens with the opportunity to read and discuss a single book of fiction within their communities. As part of The Big Read, selected communities will receive grants for innovative reading programs, and support and resources that include materials for discussing 20th-century American classics, an extensive Website, and a national awareness campaign.
Through Arts Midwest, the NEA, in partnership with IMLS, will award $5,000-$20,000 grants to approximately 200 community organizations to conduct month-long community-based Big Read programs that encourage citizens to read for pleasure and enlightenment. First-cycle programs, with grants awarded in November 2006, will run from January to June 2007. A second cycle will take place from September to December 2007. Each grantee is required to produce a comprehensive community-wide read focusing on one or more Big Read titles, and including collaboration among libraries, schools, local government, military bases, and the private sector. Each community will develop its own program of activities, such as a keynote session, special events, classroom activities, and book discussions aimed at a diverse range of audiences.
In addition to direct grants, the Big Read will provide each grantee with NEA-developed program materials that include: Reader's Guides and Teacher's Guides for each Big Read title; audio guides for each book with commentary from public figures, artists, and educators; an online organizer's guide for running a successful Big Read program; and a comprehensive Web site. In addition, grantees will receive promotional materials to encourage broad participation, including television public service announcements and radio programming. The program will provide additional professional development for key managers from each grantee to help build local capacity for partnerships and effective Big Read program delivery.
In 2006, the NEA and Arts Midwest conducted a pilot program of the Big Read in ten communities nationwide. Each participating community organization conducted a local evaluation, including the collection of basic data on program effectiveness and participation levels. The current evaluation is aimed at assessing the design of the 2006-07 Big Read program and to assess the program’s impact on literary reading habits in participating communities.
As federal agencies, the National Endowment for the Arts and the Institute for Museum and Library Services evaluate their programs on a systematic basis. The NEA, pursuant to its mandate to “support projects and productions that will encourage public knowledge, education, understanding and appreciation of the arts” (20 USC20, Section 954 (q)), has entered into an agreement with Arts Midwest to sponsor an evaluation of this arts initiative. A private evaluation firm, Rockman et al was selected to conduct the program evaluation of the Big Read initiative. The audience for most of the evaluation deliverables will be the Big Read partners’ staffs and executives; some may be distributed widely to key stakeholders and national partners.
The Big Read evaluation has the following overarching goals:
To assess the effectiveness of the Big Read program in meeting its key goals of:
revitalizing the role of literature in American popular culture;
bringing community organizations together to foster literary reading; and
strengthening skills in community organizations, particularly of library staff, for community partnerships and audience outreach.
To assess the impact of the Big Read on American literary reading habits in participating communities, following up on the NEA Reading at Risk report, in particular on:
expanding the audience of those who read literary works for pleasure and enlightenment, and
expanding participation in community-wide arts and cultural activities related to literature.
As a national study, the Big Read Evaluation will serve as a sound base from which to make estimates of the impact of the initiatives on partnering organizations, communities, and individuals. The Big Read evaluation data will also provide information on the characteristics of those who participate in the initiative and the degree to which the initiative is reaching previously under-represented groups.
A.2. Needs and Uses of the Data
We plan to collect data using several different instruments, and two samples. One set of instruments will be administered to all sites participating in the Big Read initiative after April1, 2007, and another to a sub-set of case study sites for more in-depth data on the implementation and impact of the Big Read. Copies of all instruments are included with this ICR. The specific data sources and the associated data collection methods are described below. Within the description of each instrument, we address the purpose and need for each data source.
Our data collection activities generally meet the following objectives of the Big Read evaluation:
to understand the outcomes of the Big Read for participating community organizations and for specific target audiences within these communities, such as students, teachers, and readers new to literary fiction;
to facilitate the consistent collection of quantitative and qualitative data from participating communities at the local and national levels;
to strengthen active 2007 programs based on interim project data, for example using data from grantee community organizations whose programs take place early in the cycle;
to ascertain which activities and characteristics of local programs most successfully attract participation by different community segments (e.g., age groups; cultural sectors; professional categories such as teachers, employers, or private funders);
to ascertain which activities and characteristics of local programs contribute most effectively to increase literary reading;
to ascertain which organizational combinations and project partnership structures contribute most effectively to participation in the Big Read and to increase literary reading; and
to ascertain the extent to which Big Read professional development resources strengthen the capacity of local organizations, particularly among library staff, to build partnerships and attract a broad spectrum of participants to local arts and cultural activities related to literary reading.
Data Collection Activities for All Big Read Sites
We propose to collect data from three sources in all Big Read sites:
Post-implementation online survey for Big Read lead organizations/grantees
Post-implementation online survey for Big Read participants
Follow-up participant telephone surveys/interviews with a sub-sample of online survey respondents
Post-Implementation Online Surveys of Grantees. A post-implementation survey will be used to collect information from grantees that is not otherwise available in their proposal documents or other record-keeping and reporting forms. The survey will ask about grantees’ success in reaching targeted audiences; the value of educational, professional development, and promotional materials; and capacity-building outcomes for their organizations. The survey, which will be administered online with links hosted by The Big Read website, will help gauge which resources, activities, and partnerships were most effective in attracting participation by different community segments and in increasing literary reading.
Post-Implementation Online Participant Survey. Big Read participants will be asked to complete an online survey at the conclusion of their local Big Read. The survey will help us collect formative feedback on Big Read events, assess participants’ reading habits and attitudes, and estimate the characteristics of the population participating in the Big Read. It will include items about Big Read events, resources, and reading habits; items related to specific books; and expanded demographic items matched to Reading at Risk items. Rockman will administer the survey online through a link on local and national Big Read websites, and use a variety of ways to direct participants to the link—announcements at events, printed website survey links on materials, telephone or email contacts with participants who provided contact information and permission to contact them, and announcements on national and local websites.
Follow-up Interviews with Participants. Three months after the completion of local Big Reads, we will contact a sub-sample of participants to respond to a brief telephone interview. The purpose of the follow-up interview is to assess any changes in reading habits and attitudes and any lasting impact of Big Read participation. The interview sample will be drawn from those participants who completed the online survey and will represent approximately 20% of that sample. To obtain a representative sub-sample across sites, we will use stratified random sampling procedures and the Big Read site as the stratification criteria.
Data Collection from Case Study Sites
We will conduct case studies with approximately 15% of the grantees (15 per cycle/30 in all). Sites will be selected, in collaboration with Big Read partners, based on book choice, demographics, participant characteristics (e.g., schools, first-time literary readers, military personnel), scope and timing of events, and partnerships. In Cycle 1, we will focus on grantees whose programs take place after April 1, 2007. The goal of the case studies is to gather more in-depth data, including follow-up data collected 3 months following the program. Case-study data collection will include:
Grantee Interviews/Surveys. We will ask grantees in case study sites to take part in a Big Read interviews/surveys (by phone or, for sites visited, in person) to learn more about their efforts, partnerships, and use and effectiveness of the Big Read materials. We will examine partnership roles and effectiveness of implementation models, the capacity-building outcomes for organizations, and the effect on target audiences such as readers new to literary fiction.
Partner Interviews/Surveys. We will adapt interviews for certain partners, gathering data from schools, e.g., on classroom activities, teachers’ and students’ use of the guides and audio CDs, etc.; or, from vendors, bookseller, or cultural institutions, on changes in sales or attendance during the Big Read. These interviews will be conducted during site visits.
Site Visits. For approximately half of the case-study sites (5-8 per cycles/10-15 in all), REA will conduct face-to-face activities (e.g., grantee and partner interviews, observations of Big Read events, and focus groups with selected participants, arranged by the community organizations). We have included the focus group protocol and interview protocol to be used with the participants at the sites we visit. The purpose of the site visits is to collect more in-depth information about how the Big Read initiative is implemented in the community and its impact on participants.
Three Month Follow-Up Interviews with Grantees and Local Partners. A 3-month follow-up telephone survey will explore any longer-term changes in patronage and circulation and literature-related events and partnerships. The telephone interview will be conducted with grantees and, where appropriate, selected partners in the 10-15 case-study sites the evaluation team visits.
A.3. Use of Information Technology
When possible, all surveys will be conducted using online survey technology. The data collected from online surveys will be housed on the REA server. Links to the online surveys will be hosted on the Big Read national website, as well as local Big Read websites.
A.4. Efforts to Identify Duplication
This is the only evaluation effort of the Big Read initiative currently being implemented. Therefore, there are no other existing efforts to collect the data described above. We have taken care to not duplicate information asked on the surveys with any information grantees are asked to provide as a matter of routine record-keeping and reporting.
A.5. Minimizing Burden in Small Businesses or Small Entities
We are minimizing the burden of data collection on the part of the grantee organizations by creating brief instruments that supplement, not duplicate, the data collection efforts they will be making as part of their routine record-keeping and reporting requirements for their Big Read grant.
A.6. Consequences of Less Frequent Data Collection
Most grantees and participants will be surveyed only once, at the conclusion of their community’s Big Read implementation. A sample of participants will also be asked to take part in the 3-month follow-up interview/survey. In the case study sites, some participants may also take part in focus groups, and the lead organization or grantee representative may be surveyed or interviewed twice, once on site or during their Big Reads, and during a follow-up interview 3 months following.
A7. Special Circumstances
None of the special circumstances listed in the instructions for completing this supporting document apply to the program evaluation of the Big Read Initiative.
A.8. Public Comment and Consultations Outside the Agency
A notice was published in the Federal Register NOVEMBER 24, 2006 to solicit comments on the evaluation of the Big Read Initiative prior to submission of this OMB clearance request. (A copy of this notice is attached.) No public comments were received comments were received at NEA in response to this notice.
No payments are planned will be made to the respondents in the evaluation other than the contractual agreements for disbursing grant funds.
.
A.10. Assurance of Confidentiality
The researchers will obtain written informed consent from all adult participants who complete surveys or interviews. Participants will be briefed on the procedures, risks, benefits, and other human subjects protections, and asked to sign an informed consent agreement to participate and have data used in the study. The researchers will also explain anonymity and confidentiality procedures as well as data security. No personally identifying information will be associated with survey or interview responses in the reporting of the findings.
To ensure confidentiality of information, only site and respondent ID numbers will be housed with each form of data for the purposes of matching data across instruments. No names or other identifying information will be kept with the data that will be used for analysis. A separate file with names of individuals or sites and matching ID numbers will be kept on a separate computer under password protection. The external evaluator will compile all data, which will be stored in a secure location. No identifying information will be used in the reporting of findings or provided to any grantee or host organization.
A.11. Questions of a Sensitive Nature
There are no questions in the attached instruments that respondents would consider sensitive.
A.12. Estimates of Hour Burden for Data Collection
Data Collection Protocol |
Method of Administration |
Approximate Time for Administration |
All Sites |
||
Grantee survey |
Online |
1620 minutes (x 200 organizations) |
Participant survey |
Online |
810 minutes (x 12,0002000 respondents |
Follow-up participant interview |
Telephone |
5 minutes (x 1,800200 respondents) |
Case Study Sites |
||
Grantee interview |
Telephone/in person |
1620 minutes (1 grantee x 30 sites) |
Partner interview |
In person |
1620 minutes (1-2 x 10 15 sites) |
Follow-up grantee and partner interviews |
Telephone |
1215 minutes (1 grantee x 30 sites; 1-2 partners x 10 15 sites) |
Site visits |
In person |
2 days per site (x 14-2010-1515 sites) Focus groups 20 minutes Intercept interviews 5 minutes |
A.13
No commentary (if needed—input from Tom and Sunil)
There are no costs to respondents other than that of their time to respond.
No commentary (if needed—input from Tom and Sunil)
The total cost of this program evaluation to the Federal government is $314,000.
Not applicable
Partner interviews
Participant focus groups
Executive Summary of major findings to be developed in collaboration with NEA and Big Read partners that will target specific stakeholders and audiences to whom the partnership wishes to disseminate findings.
Description of methodology, data collection goals, research questions, and sampling procedures.
Context of the Big Read initiative including a description of the local sites and the participants
Findings including the following:
The effectiveness of Big Read professional development resources in building capacity of community organizations, particularly library staff, to attract and manage community partnerships and to attract diverse audiences;
The impact of Big Read participation on libraries, librarians, and other community organization participants, including the ways in which these organizations build coalitions and partnerships;
The effectiveness of the program in attracting participation by diverse populations and changing their attitudes and behaviors towards literary reading;
The varying models used by different communities in implementing the program and their relative success in attracting participation, building community partnerships, and changing patterns of literary reading
Implications and recommendations for scaling up and sustaining the Big Read initiative across and within communities and the issues and challenges associated with implementation.
Table 2. Proposed Big Read Month-by-Month Work Plan
Evaluation Activities |
Initial Planning, Cycle 1 |
Cycle 2, Final Data Analysis and Report |
||||||||||||||||||
Nov 06 |
Dec 06 |
Jan 07 |
Feb 07 |
Mar 07 |
Apr 07 |
May 07 |
June 07 |
July 07 |
Aug 07 |
Sept 07 |
Oct 07 |
Nov 07 |
Dec 07 |
Jan 08 |
Feb 08 |
Mar 08 |
Apr 08 |
May 08 |
June 08 |
|
Develop relational database and reporting mechanisms |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Develop evaluation instruments for impact and implementation evaluation |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Conduct case studies |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Administer surveys |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Conduct quantitative and qualitative impact and implementation analyses |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Conduct analysis and extrapolation of Reading At Risk survey items |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Submit status reports |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Write final report |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
A.17 Exemption to Display Certification
Not applicable
A.18 Exceptions to Certifications
There are no exceptions to the certifications set forth in Item 19.
| File Type | application/msword |
| File Title | OMB SUPPORTING STATEMENT |
| Author | Terri Akey |
| Last Modified By | neaprofile |
| File Modified | 2007-01-31 |
| File Created | 2007-01-31 |