This is a request for approval to collect information about assessment of student learning in the arts. This research study, Improving the Assessment of Student Learning in the Arts (SLA), will collect and analyze data on current practices and trends in the assessment of K-12 student arts learning and identify assessment models that might be most effective in various learning environments. Arts educators who provide instruction in schools and communities are looking for models of assessment in all art forms. Creating, identifying, and implementing effective assessment tools that support quality arts instruction is a major challenge for arts educators, and this project is a major step in addressing this issue.
This survey will supplement what is learned through the literature review by collecting information directly from arts providers, researchers, and evaluators about: 1) what arts organizations are doing to assess student learning in the arts; 2) the types of assessments arts organizations and evaluators are using to measure student arts learning; and 3) what assistance these organizations need to improve assessment of student learning in the arts. Additionally, this survey will allow us to identify and collect non-published research conducted by arts organizations. We are aware that arts organizations conduct research and collect data that does not necessarily culminate in a publication or publicly distributed report. Arts organizations engage in these activities for a variety of reasons, from fulfilling grant requirements to program improvement, and in some cases the findings are not published or disseminated. This information is nonetheless important to understanding the current state of arts assessment.
To maximize the value and efficiency of the survey, it was developed after an initial review of the literature and consultation with experts in the field. Thus, questions reflect what was learned from the preliminary literature review, particularly as it related to state and national trends in assessment of student learning in the arts. This allowed us to better target questions that probe deeper into what arts organizations are doing and allow us to collect richer data on trends and practices. Further, the survey serves as a formative tool to help identify the needs of arts organizations and formulate recommendations on ways in which we can help improve the state of assessment of K-12 student learning in the arts.
The data from this collection will be used to prepare a report that will include: a) a brief history of arts learning assessment in the country, and b) an analysis of current trends and practices including identification of practices that hold the greatest potential for success in a varitey of arts learning environments. We will disseminate the results gathered through the project to the arts education field to help advance student assessment in the arts.
Evaluation and assessment are becoming more prominent in the arts, and local arts organizations are now often required to demonstrate results related to student learning. Assessment of student learning in the arts is dependent on such factors as the arts discipline, the age level, skill level, and instructional setting. This is compounded by an overall lack of availability and knowledge of appropriate assessment resources and models, as well as a lack of understanding of how to effectively evaluate and assess student learning in the arts. This data will provide much needed information about how agencies collect and use assessments, and will be used to push the field forward and provide better information to arts organizations.
WestEd, our cooperator, will use a web-based survey to collect data from respondents. The survey will be structured to allow users to enter information in multiple sessions so respondents can schedule time for the survey accordingly. Since the proposed respondents have ready access to the Internet and are accustomed to communicating via email, the electronic dissemination of the survey is appropriate for this population.
During the data collection and reporting period, WestEd will establish a dedicated email address to allow respondents to ask questions or request assistance during the survey period. The e-mail address will be featured on all pages of the survey and in any related correspondence to grantees.
WestEd conducted a review of the literature and interviewed advisory committee members to identify already available information and to inform survey development. Additionally, multiple people reviewed the survey to avoid redundancy. These efforts helped ensure only the most vital and necessary questions were included on the survey.
Multiple methods will be used to minimize burden on small entities. Participating in the survey is voluntary so small entities may opt out of the study with no consequence. Ample time has been allocated for completing the survey allowing small entities to complete it without feeling rushed to do so. The survey will be administered electronically so respondents may complete the survey in multiple settings at a time convenient to them. Additionally, the survey includes skip patterns so that respondents will not see or be asked to answer questions that do not apply to them. We believe these efforts will minimize the burden on all respondents.
As previously described, the purpose of the data collection is to determine the current state of assessment of student learning in the arts. The literature review and interviews provide some data to address this issue, but lacks the breadth and depth needed for a thorough examination. Often studies and evaluations related to arts learning go unpublished or are limited in circulation. Currently there is no repository for art studies, evaluation designs, or assessment tools, creating a need for these agencies to create and develop their own from scratch rather than learning from previous efforts by their peers. By working to compile such information and identify how agencies collect and use assessments, we will be able to push the field forward and provide better information to arts organizations.
There are no special circumstances that require the collection to be conducted in a manner inconsistent with the guidelines in 5 CFR 1320.5.
A pre-clearance notice about the study was published in the Federal Register on January 9, 2009 (Volume 76, Number 6, page 963). A notice about the study will also be published when the final OMB package is submitted.
We spoke with potential respondents about the availability of data and lack of current data. We hired WestEd as our cooperator to conduct interviews with an advisory group to collect their views about the availability of data and to inform survey development and data collection efforts.
There are no payments or gifts to respondents.
This collection does not involve collecting any confidential information. Therefore, no assurances of confidentiality are required.
This project does not include any questions of a sensitive nature. As required by the Paperwork Reduction Act of 1995, information will be provided to respondents about the purpose of the data collection and how we will use the information.
We anticipate sending the survey to 130,244 respondents. The estimated time to complete the survey is 30 minutes, and this is a one-time survey. Therefore the total estimated burden for completing the survey is 65,122 person hours.
There are no additional respondent costs aside from those outlined in section A12.
The total cost to the Federal Government for the data collection is $40,000. These costs are associated with 1) developing the survey, 2) analyzing the survey data and combining these with data from literature review and expert interviews, and 3) preparing a written research report.
Exhibit 1: Estimated Annual Costs of Data Collection, by Expense Type
Expense Type |
Annual Cost (in dollars) |
Percent of total cost |
Developing the survey |
$10,500 |
26% |
Analyzing the survey data and combining these with data with data from the literature review and expert interviews |
$17,500 |
44% |
Preparing a written research report |
$12,000 |
30% |
TOTAL |
$40,000 |
100% |
This is a new request for clearance.
Survey data collection will occur between June and October, 2009 (pending OMB approval). The data will be aggregated and descriptive statistics will be computed. Data may be disaggregated by specific sub-group (such as local arts organization or education staff) as appropriate. WestEd will produce a report based on the outcomes and findings from this data collection by January 29, 2010. The report will be made available on our website and will be distributed to all interested parties, including survey respondents.
No request is being made for exemption from displaying the OMB expiration date.
This collection of information involves no exceptions to the Certification for Paperwork Reduction Act Submissions.
The study population for this data collection is the universe of members from 16 specific arts agencies, education agencies, cultural organizations, and research organizations. A census (all members of this defined universe) will be asked to participate in this data collection. This respondent universe consists of 130,244 members from these 16 agencies/organizations. We anticipate a response rate of 40%.
Individuals are members on the listserv of large organizations involved in the arts. The agencies fall into the following categories: a) national arts agencies, b) national and state education agencies, c) national community guild, and d) cultural organizations.
We are using pools of individuals to collect rich data and describe the processes in which these individuals have used arts assessment tools. We recognize and understand the limitations of the data we are collecting and are not intending to obtain precision in defining the universe, nor are we planning to generalize or make statements about all arts educators and their use of assessment.
Respondents will complete the web-based survey online. Membership agency representatives will send an email to their entire membership, and each member (i.e., potential respondent) will be given a survey access code. The email will indicate participation is voluntary and will include the purpose for the survey, a link for accessing the survey, and contact information should respondents have any questions or want additional information about the data collection.
During the data collection period, WestEd will also establish a dedicated email address to allow respondents to ask questions, report problems, or request assistance. The e-mail address will be featured on all pages of the survey and in any related correspondence to respondents.
We anticipate a response rate of 40% for the data collection, and will provide information and support before and during the data collection process to facilitate participation. We will provide advance notice of the survey via email prior to data collection to allow respondents to gather the necessary information. The supporting documents that accompany this advance notice will also explain the purpose and importance of the survey for the study.
During data collection, WestEd will provide support as described in the section above, and also will track the response rate as respondents login and complete the survey. The electronic administration of the survey allows for instant access to the data and also allows for instant identification of nonrespondents. The membership agencies will send targeted reminder emails to nonrespondents two weeks after the initial administration, and a second targeted reminder email one month after initial administration. We will provide the text for the email reminders, and agencies can modify them as needed.
Throughout the data collection period, WestEd will provide regular updates to our staff on the response rate and the status of the data collection. In cases where the response rate is unexpectedly low, or where issues are identified regarding the validity of the initial responses, WestEd will convene a conference call with our staff to make any necessary amendments to the data collection plan.
The survey instrument went through a series of reviews before being submitted for OMB approval. WestEd worked closely with our staff throughout the design and revision process, and enlisted key leaders in the area of arts assessment to review the survey as well.
WestEd also pilot tested the survey with a group of three to four respondents to assess item comprehension, the effectiveness of the proposed strategies for gaining participation, and the length of time (i.e., 30 minutes) for respondents to answer the survey questions. Such information helped determine the actual burden associated with the survey.
WestEd will pilot test the survey with one of the agencies to ensure we are receiving the intended information, including the coding and categorization of qualitative data, before continuing the use of the survey.
The SLA study is led at WestEd by Treseen McCormick, a Senior Project Director in the Evaluation Research program. McCormick and her team of WestEd staff will be responsible for developing the data collection instruments, collecting and analyzing the survey data, and reporting the results to our staff.
Improving the Assessment of
Student Learning in the Arts page
| File Type | application/msword |
| File Title | PART A |
| Author | Flaherty |
| Last Modified By | neaprofile |
| File Modified | 2009-06-30 |
| File Created | 2009-06-30 |