2700-0114 GLOBE Program Evaluation Form Supporting Statement_12_18_06

2700-0114 GLOBE Program Evaluation Form Supporting Statement_12_18_06.doc

GLOBE Program Evaluation

OMB: 2700-0114

Document [doc]
Download: doc | pdf

A


. JUSTIFICATION

  1. Circumstances Making Information Collection Necessary

GLOBE is an ambitious science and education program that continues to evolve even as it penetrates thousands of classrooms in the United States and around the world. The University Corporation for Atmospheric Research (UCAR) took over the management of the GLOBE Program in 2003, and worked closely with SRI to move into the next phase of program evaluation. Currently, GLOBE materials development and professional development training are being organized in a phased approach, with a new "E-guide" that was completed in 2005 and new protocol development and associated activities and learning modules under development in 2006-2008 in response to the NASA/NSF MOU that resulted in 4 new Earth System Science Projects (ESSPs) for the GLOBE Program. Individuals responsible for developing and/or revising these materials and for managing the new NSF ESSPs need to know how GLOBE Partners are facilitating teachers to use GLOBE materials, which parts of the program are working well, and which parts require improvement. In addition, the program needs basic information concerning how many students and teachers at particular grade levels and settings are participating in the program. Program managers need to understand the number of students and teachers being affected by GLOBE and the impact that the program is having on teaching and learning, to make optimal resource management decisions.

  1. How, and by Whom, and for What Purpose Information Is to Be Used

SRI conducted program evaluation for the first 10 years of the program and set the stage for future program evaluation in an organized fashion (1995-2005). The proposed 2006-08 study will allow GLOBE to both measure whether GLOBE is effective and the conditions under which it is effective as a means of achieving self-sustainability in the near future through a comprehensive annual evaluation. The Annual Partner Survey collects, initiated in 2005, gathers information regarding partnership funding and sustainability, programmatic implementation and capacity building, trainers and trainer support, teachers and teacher support, communication and administrative support and education and environmental issues relevant to their state and/or country. The GLOBE Annual Evaluation reports on multiple constituencies, each of whom is likely to use the evaluation for their own purposes as shown in Table 2.

T


able 2. Application of Evaluation Information

Likely Application of

Evaluation Findings

Constituency


Purpose of Application

Build a framework that promotes collaboration between all members of the GLOBE community and facilitates capacity building and self-sustainability

GLOBE Program Office

To assist Partners to build strong models of regionalization to build infrastructure and promote future sustainability of the program around the world.

Adjustment of train-the-

trainer offerings and partner

support

GLOBE’s International and U.S. Partners

To refine the GLOBE

program and improve

partnerships' effectiveness; promote capacity building.

Adjust training and post-

training support offerings to

teachers

GLOBE's International and U.S. Partners

To adjust offerings to

reflect findings about most

effective services partners

can offer to teachers.

Design new approaches to

structuring collaborative

relationships with schools

GLOBE Community: GLOBE Science PIs, Partners, Teachers, Students; GLOBE Program Office

To improve data collection

and reporting and the application of both to student research that leads to

advances in science.

Align GLOBE with standards and policies to make it easier for teachers to adopt inquiry‑

based science curricula

District- and state-level

administrators

To facilitate GLOBE implementation in the U.S. and abroad.

Share new strategies for

implementing GLOBE

effectively in classrooms

GLOBE Community: GLOBE Science PIs, Partners, Teachers, Students; GLOBE Program Office

To improve student

achievement within

GLOBE.

Online Evaluation needs; ease of data entry and mechanisms supporting student research

GLOBE Community: GLOBE Science PIs, Partners, Teachers, Students; GLOBE Program Office

To improve the functionality of the GLOBE Web site.



  1. Use of Improved Information Technology to Reduce Burden

The GLOBE Partner survey is an online instrument developed in 2005 and housed within the GLOBE Partner Administrative section of the GLOBE Web site. This survey is administered in January of each year. The survey gathers data on all activities related to GLOBE implementation for the year prior to administration of the survey. Therefore, the 2005 GLOBE Partner Survey was administered in January 2006 and the 2006 GLOBE Partner Survey will be administered in January 2007, gathering information for all GLOBE community activities that occurred during 2006.

  1. Efforts to Identify Duplication

SRI no longer serves as the external evaluator of The GLOBE Program. At this time, the GLOBE Program Office is the only entity conducting surveys, interviews, and assessments of the impact of GLOBE world-wide. All Annual GLOBE Surveys distributed to the GLOBE community, including the optional online surveys regarding Web site redesign, school participation in the new NSF ESSP projects and evaluation of GLOBE events, are directed by the GLOBE Program Office (GPO).


The GPO has found no duplication of these efforts.


  1. Impact on Small Entities

No small business entities are proposed as respondents. All possible efforts have been made to prepare and carry out the surveys, interviews, and assessments in an online format as well as through the GLOBE Partners and Teachers so as to minimize the burden on GLOBE students and their schools, which are mostly entities associated with local governing bodies.


  1. Consequence If Collection Conducted Less Frequently

The GLOBE Program has a built-in flexibility that enables it to make necessary changes as it develops. These changes will be made on the basis of reliable information that is gathered from the GLOBE community whose constituents include Country Coordinators, U.S. Partners, Science PIs, Teachers and their Students (partners serving as the voice for the teachers and students that they work with). This is one of the main purposes of the evaluation.

There is an ongoing need of the program to understand the impact of GLOBE on student learning, because so many new students are exposed to the program each year. In the past, GLOBE has either measured program effectiveness using a global measure of implementation (e.g., data reporting), investigated aspects of implementation (e.g., data analysis, protocol implementation) and employed SRI to conduct teacher/student surveys.

The change to one (1) required Annual Evaluation for all GLOBE Partners ensures that the GLOBE Program Office will be provided with timely information that adequately represents the international nature of the program which is implemented in over 100 countries world-wide.

  1. C


    ollection Compliant with OMB Guidelines

Data collection is fully in compliance with OMB data collection guidelines in 5 CFR 1320.6 (Controlling Paperwork Burdens on the Public-General Information Collection Guidelines).

  1. Consultation with Persons outside the Agency

A number of professional researchers outside the federal agencies that have been involved in GLOBE have been consulted in the design of the current online evaluation and in the development of surveys, interview protocols, and assessments used by the GLOBE Program. They include:

Evaluation Project Development by SRI. Dr. William Penuel, Senior Education Researcher, served as co-PI on the GLOBE evaluation project ending in 2005 and has led a number of large-scale evaluations of educational technology programs, including the Department of Education's Community Technology Centers Program. He also has experience in developing assessments of science inquiry, and is PI for an NSF-funded project focused on the development of formative assessment tools to improve project-based science. Dr. Penuel and his team from SRI were involved in all aspects of the development of the GLOBE Annual Survey currently in use.

  1. Payment of Gifts to Respondents

No payment of gifts is provided to respondents of GLOBE Program evaluations.

  1. Assurance of Confidentiality

Respondents work exclusively inside the GLOBE Program Administrative pages of the GLOBE Web site when responding to the GLOBE Annual Evaluation. Each GLOBE Partner records their programmatic information in their secured space on the Web site and each Partner has the ability to print out a report of their survey responses for their own use. All Partners have been provided with information explaining the purpose for data collection and a summary document of all information obtained through the survey each year is provided to the entire GLOBE community at each GLOBE Annual Conference.


  1. Justification for Questions of Sensitive Nature
    No items of a sensitive nature are contained in these instruments.

  2. Estimates of Respondent Burden

The respondent burden for the recipients of the instruments is described in this section. These instruments include a required online Annual Partner Implementation survey and optional online materials development and review forms. Table 3 reports the information used to calculate the respondent burden, including the type of instrument, number of respondents, the estimated response rate, the estimated time each respondent will spend completing the task. These burden estimates are based on sample sizes and completion rates provided in Item 1,


Section B of this document—Description of Respondent Universe and Sample and Procedures for Collection of Information.

Table 3. Calculation of Respondent Burden

Annual GLOBE Partner Survey

Respondents

Hours

Total Burden

International Country Coordinators

108

2

216

U.S. Partners

150

2

300

TOTAL

258

2

516



  1. E


    stimated Costs to Respondents

There are no respondent costs other than labor hours; this cost estimate is shown in Table 3.


  1. Estimated Costs to Federal Government

2006 Partner Survey Preparation time and cost


Staff


Tasks

Hours worked

Cost

Teresa Kennedy

Survey text development

60

$5,904

Paula Robinson

Survey text development

20

1,343

Karen Milberger

Technical support

60

3,271





$10,518


There are no estimated costs to GLOBE Partners. They complete the survey on a voluntary basis.

  1. Reasons for Change in Burden or Program

The total annual response hours requested has been reduced using several techniques:

  • By constructing online survey and assessment instruments; and

  • By limiting the surveys to Partners and not including teachers and their students.


  1. P


    ublished Results

A summary of the data gathered through the Annual Partner Survey is provided to all Partners at each Annual Conference. Individual responses to any of the items on any of the instruments are not published, except as part of summary reports and papers about the GLOBE Program and its implementation. If, as part of a description of GLOBE at a particular school, an individual is named in association with a particular response, that person's full consent will have been obtained ahead of time, and they will be given a chance to review the quotation in the context of the report in which it is presented. Absolutely no student names will be associated with individual responses at any point in the evaluation. In addition, Partners are encouraged to provide their specific results in the form of Partnership Reports published in the GLOBE Annual Conference Proceedings.

  1. Non-display of Expiration Date

N/A.

  1. Exceptions to the Certification Statement

N/A.



B


. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL
METHODS

1. Description of Respondent Universe and Sample and Procedures for Collection of Information


Large-scale GLOBE Partner Survey. All GLOBE Partners are required to complete the annual GLOBE evaluation form in January of each year.

2. Schedule and Procedures for Collection of Information

October-December 2005: Notification of survey

January 2006: Implementation survey covering 2005 activities

February 2007: Follow-up of non-respondents.


October-December 2006: Notification of survey

January 2007: Implementation survey covering 2006 activities

February 2008: Follow-up of non-respondents.


October-December 2007: Notification of survey

January 2008: Implementation survey covering 2007 activities

February 2009: Follow-up of non-respondents.


  1. M


    aximizing Response Rates; Issues of Non-response

As shown above, a rigorous schedule has been developed for notifying respondents prior to the survey and for following up on non-respondents. Our experience has shown that providing notification of a survey before it is conducted and following survey dissemination with up to 3 mail, e-mail, or telephone reminders produces a very high rate of response. Because of these follow-up procedures in place, the high morale among the GLOBE partners and their teachers, and the low burden entailed in completing the online form, response rates for active GLOBE partners is 100%.


  1. Tests of Procedures and Methods

Large-scale GLOBE partner surveys have been conducted in the past two years. Survey items have undergone minor modifications for each new use, reflecting lessons learned from prior years' data and changes in GLOBE Program content.

  1. Persons Responsible for Data Collection Design and Development

Dr. Teresa Kennedy worked with Dr. William Penuel and his team from of SRI International to create the GLOBE Annual Partner Survey. She will lead all activities related to this survey.

Dr. Teresa Kennedy

Deputy Director, The GLOBE Program

University Corporation for Atmospheric Research (UCAR)

3300 Mitchell Lane

Boulder, Colorado 80301

303-497-2643

[email protected]



File Typeapplication/msword
File Title12-12-06 GLOBE Program Office
AuthorLMIT-ODIN
Last Modified ByWalter Kit
File Modified2006-12-19
File Created2006-12-19

© 2024 OMB.report | Privacy Policy