31450158 Supporting Statement Part A

REQUEST FOR RENEWAL OF OMB 3145-0158

PROJECT MONITORING SYSTEM FOR

INFORMAL SCIENCE EDUCATION (ISE) PROGRAM

of the National Science Foundation

Directorate for Education and Human Resources

Division of Learning in Formal and Informal Settings

Section A.

A. Introduction

This request for Office of Management and Budget (OMB) review asks for renewal of OMB 3145-0158. This collection is the project monitoring system for the Informal Science Education (ISE) Program, of the Directorate for Education and Human Resources (EHR) at the National Science Foundation (NSF). This system was developed and has collected project level data during the last three years. Continued collection of information from ISE projects will enable NSF to continue to assess the extent to which individual project grants have attained their anticipated outcomes.

A.1. Circumstances Requiring the Collection of Data

Informal education is the life-long learning process in which every person acquires knowledge, skills, attitudes, and values from daily experiences and resources in his or her environment. Informal learning is self-directed, voluntary, and motivated mainly by intrinsic interests, curiosity, exploration, and social interaction. The purpose of the ISE program is to support projects designed to increase public interest in, understanding of, and engagement with science, technology, engineering, and mathematics (STEM). The outcome of all ISE projects is an informed citizenry that has access to the ideas of science and engineering and understands their role in enhancing their quality of life and the health, prosperity, welfare, and security of the nation.

All ISE projects have as their primary audience the informal learner -- from young child to senior citizen. Informal learning, in contrast with formal learning, refers to activities that are not primarily for school use or part of an ongoing school curriculum. Informal learning presumes voluntary participation as opposed to the mandatory participation of a credited school activity.

The goals of the Informal Science Education program are to encourage and support projects that:

• engage the interest of children and adults in STEM in daily life so that they develop capabilities: scientific and technological literacy, mathematical competence, problem-solving skills, and the desire to learn;

• bring together individuals and organizations from the informal and formal education communities, as well as from the private and public sectors, to strengthen STEM education in all settings; and

• develop and implement innovative strategies that support the development of a socially responsible and informed public, and demonstrate promise of increasing participation of all citizens in STEM.

To assure that the projects chosen for funding in this program achieve these broad program goals, it is important that the program monitor the progress of each project to determine whether the individual projects are meeting their individual goals. Additionally, it is important for NSF to be able to report on the progress of its programs to a wide variety of government audiences.

A.2. Purposes and Use of the Data

The ISE Online Project Monitoring System (OPMS) is designed to obtain information about each ISE-funded project and its grant recipient and partner organizations, participants, activities, deliverables, and impacts.¹ It allows ISE Principal Investigators and/or Project Directors to input various data pertaining to their respective projects. These data are used by program officers to evaluate the collective impact of the ISE portfolio of funded projects, to monitor project-related activities and projects’ progress over time, and to obtain information that can inform the design of future ISE projects.

All information collected are being used by the Division of Research on Learning in Formal and Informal Settings (DRL) for program monitoring, to support program impact assessment and evaluation needs, and to provide analytical and policy support to the EHR directorate. The information is being used and disclosed in accordance with the Education and Training System of Records, 63 Fed. Reg. 264, 272 (January 5, 1998). NSF is making the data available only to EHR and DRL staff, EHR and DRL contractors with responsibility for impact database management or program impact assessment and evaluation, and the NSF program managers and their staff.

The OPMS is comprised of a series of survey items that are completed at different points in time during the grant award period. An initial submission, completed by respondents at the beginning of the grant award period, obtains information about a project’s characteristics, anticipated activities, and desired outcomes. More specifically, it consists of the following six sections:

Section 1: Information about the Lead Organization (i.e., organization name; division/department name; description of organization; Congressional district; prior financial support; and contact information).

Section 2: Information about Individual Members of the Project Team (i.e., individual name(s); affiliation; role of individual on project; contact information; and anticipated level of effort).

Section 3: Information about Organizational Partners and Contractors (i.e., organization name; division/department name; description of organization; contact information; and role of organization on project).

Section 4: Information about the ISE Project (i.e., project title; abstract; anticipated end date; primary and secondary focus; and various information on target audience(s), how these will be reached, and anticipated numbers of individuals reached).

Section 5: Information about Deliverables (i.e., deliverable name; description; type of deliverable; primary or secondary focus; and projected status of deliverable in subsequent years).

Section 6: Information about Project Impacts (i.e., specification of impact; benchmarks or targets used to determine whether impact is achieved; type of impact; approaches, data collection techniques, and mode(s) of analysis used to demonstrate impact is attained; anticipated date by which impact will be attained; and methods used to disseminate information about the project).

An annual report, completed by respondents at the end of each year of the grant award period, updates information from the initial submission and obtains information about the actual activities that occurred and the number of individuals reached during the previous 12 months. A closeout report, completed at the end of the last year of the grant award period (i.e., in lieu of annual report), obtains information about the outcomes achieved over the course of the project, as well as an assessment of future plans and lessons learned that would be of use to future ISE projects.

The primary purpose for this data collection is program planning and management, also known as program monitoring, at the project and program levels. Monitoring the ISE program yields a better understanding of how the program is being implemented and its impact. NSF uses the data to monitor the annual activities and associated outcomes of individual projects. It also provides important information for NSF to report on program progress for a host of requests.

All information collected is and will continue to be used to provide analytical and policy support to EHR, assisting NSF to make decisions about future funding and other program initiatives to improve STEM education.

As of June 2009, the OPMS has collected initial submissions from 47 ISE-funded projects. Westat, the contractor responsible for developing and administering the OPMS, is currently developing a report that will summarize the information provided by these projects. The report will provide the following information:

• Total number of projects (e.g., media; exhibit; research), by audience type (i.e., public only; professional only; both public and professional);

• Type of lead organizations (e.g., college/university; museum), by audience type and by project type;

• History of prior NSF funding received by the lead organization, by audience type;

• Type of organizational partners and contractors, by audience type and by project type;

• Primary and secondary STEM content foci, by audience type and by project type;

• Type of project deliverables, by audience type and by project type;

• Anticipated target audiences, by project type;

• Total number of anticipated participants, by project type;

• Geographic reach of projects, by project type; and

• Categories of project impacts (i.e., knowledge/comprehension; engagement/interest; behavior; skills; attitude; other), by audience type and project type.

These 47 projects will provide annual reports throughout their grant award period, followed by a closeout report at the end of the final year of the grant award period. In addition, initial submissions will be collected from more recent ISE awardees in September 2009.

A.3. Use of Information Technology to Reduce Burden

The ISE OPMS makes maximum use of computer technology to minimize the response burden. Projects enter and submit the required data over the Internet using the specially developed ISE online monitoring system software. EHR favors Web-based systems because they facilitate respondents’ data entry across computer platforms. One feature of the system is the thorough editing of data for completeness, validity, and consistency prior to final submittal. Editing is performed as they are entered. Questionable or incomplete entries are called to respondents’ attention before they are submitted to NSF. Features such as checkboxes and standard menus facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden.

The ISE online data collection software facilitates respondents’ data entry, and employs automatic data validation (such as limiting response range for certain data entry cells) to ensure correct and complete data submissions, thus reducing the need for edit follow-ups. In all annual and closeout reports, the Web database displays data previously submitted. Data on members of the project team and organizational partners that were submitted by the project for the previous year, for example, are shown on the following year’s form so these data can be easily reviewed and updated. In addition, information on project deliverables and impacts entered in the initial submission is pre-filled in subsequent years. Since most program participation is on a multi-year basis, updating prior years’ data is far easier and less burdensome than providing the data again.

A.4. Efforts to Identify Duplication

The online management system is the only current means of collecting these data. No duplicate data are being collected—and no similar data currently exist. Data collected via the ISE OPMS will be used, where possible, to pre-fill survey items in subsequent years to further minimize overall response burden.

A.5. Small Business

No information is to be collected from small businesses.

A.6. Consequences of Not Collecting the Information

If the information is not collected, NSF will be unable to document the effectiveness, impacts or outcomes of the ISE program. It will not be able to meet its accountability requirements or assess the degree to which projects are meeting their goals. Moreover, NSF will be unable to comply fully with the congressional mandate that the Foundation evaluate its STEM education programs.

A.7. Special Circumstances Justifying Inconsistencies With Guidelines in 5 CFR 1320.6

The data collection is in compliance with 5 CFR 1320.6.

A.8. Federal Register Notice and Consultation Outside the National Science Foundation

This data collection was published in Federal Register May 5, 2009 (74 FR 20740). One comment was received that suggested that this information collection was already being conducted by the Department of Education. NSF assures OMB that the Information Science Education Program is not also duplicated by the Department of Education.

In June 2005, seven Principal Investigators (PIs) of ISE-funded projects were contacted to pretest a paper-based version of this monitoring system. This sample was taken from a list of PIs provided by EHR and was designed to represent a broad range of projects that receive funding from ISE (i.e., community-based, media-based, and museum-based projects). The PIs responded well to the collection, felt that the information being asked of them was reasonable, and provided only a few suggestions for changes. Selected items were subsequently revised based on respondents’ comments. Feedback received from respondents during the initial round of data collection in 2007-08 was subsequently used to enhance existing training materials.

A.9. Payments or Gifts to Respondents

No payments or gifts will be provided to respondents.

A.10. Assurance of Confidentiality

The data collected via the online system are kept confidential. This information is maintained in accordance with the requirements of the Privacy Act of 1974. Data collected are available to NSF officials and staff, evaluation contractors, and the contractors hired to manage the data and data collection software. Data are processed according to Federal and State privacy statutes. Detailed procedures for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). That system limits access to personally identifiable information to authorized users. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c.

A.11. Questions of a Sensitive Nature

The types of questions asked in this monitoring system are not considered sensitive. The only individually identifiable information collected by the surveys is the name and contact information for persons completing the surveys and/or supplying the data reported. This information is needed to allow staff and project evaluators to follow-up with any necessary clarifying questions.

A.12. Estimates of Response Burden

As mentioned above, the ISE OPMS collects both quantitative and qualitative data on an annual basis that allows for comparisons both within and among NSF-funded projects over time. In keeping with NSF’s ISE program monitoring goals, the instrument used is administered via an online system and is designed to collect data that are easily accessible to respondents.

A.12.1 Number of Respondents, Frequency of Response and Annual Hour Burden

The estimated annual response burden is 2,508 person-hours. The estimated annual response burden has increased from the original request for clearance in 2006. In 2006, the annual burden was estimated at 1,920 hours. The reason for this change is the inclusion of additional ISE project types in the OPMS that were not included in prior rounds of data collection. Whereas only full-scale development and broad implementation projects were included in the 2006 estimates, the ISE program has added several new project types to its project portfolio, including Pathways, EAGER, and RAPID awards, all of which will be asked to complete the OPMS. In addition, the program has decided to include several categories of pre-existing ISE awards (i.e., Research, CRPA, Conferences) in the OPMS for the first time, which represents a change in policy from prior years. The inclusion of these categories will allow the ISE program to generate an exhaustive set of information about ISE-funded projects, regardless of their size or scope, on an annual basis.

In all cases, the respondents are PIs on ISE-supported projects. Burden hours per response are estimated on the basis of discussions with NSF program officers, a sample of ISE PIs, and experience in administrating similar surveys. The calculations used to determine the overall response burden of 2,508 hours are shown in Exhibit 1. The differences in the number of hours per respondent type shown in Exhibit 1 are due to the fact that Research, Pathways, CRPA, Conference, EAGER, and RAPID awards are exempt from the section in the OPMS pertaining to project impacts.

Exhibit 1: Calculations used to estimate overall response burden for the ISE online collection2
Respondent Type	Number of Respondents	Hours Per Respondent Type	Annual Person Hour Total
Initial submission
PIs – Full-scale and Broad Implementation	23	24	552
PIs – Research, Pathways, CRPA	17	16	272
PIs – Conferences, EAGER, RAPID	15	16	240
Annual report
PIs – Full-scale and Broad Implementation	69	8	552
PIs – Research, Pathways, CRPA	51	4	204
PIs – Conferences, EAGER, RAPID	0	N/A	0
Closeout report
PIs – Full-scale and Broad Implementation	23	16	368
PIs – Research, Pathways, CRPA	17	10	170
PIs – Conferences, EAGER, RAPID	15	10	150
Total	230		2,508

A.12.2 Hour Burden Estimates by Each Module and Aggregate Hour Burdens

The hour burden estimates by type of module are presented in Exhibit 1.

A.12.3 Estimates of Annualized Cost to Respondents for the Hour Burdens

The overall annualized cost to the respondents is estimated to be $116,371.20. The hourly wage rate is based on information found in the U.S. Department of Labor, Bureau of Labor Statistics, National Compensation Survey (Table 1, National Compensation Survey: Occupational Wages in the United States, July 2002, June 2003, Summary 03-02), adjusted for inflation. Calculations are shown in Exhibit 2.

Exhibit 2: Calculations used to estimate cost burden for the ISE OPMS
Respondent Type	Calculation (# of respondents x hourly wage x hours to complete)	Total Hours
Initial submission
PIs – Full-scale and Broad Implementation	23 respondents x $46.40/hour x 24 hours to complete	$25,612.80
PIs – Research, Pathways, CRPA	17 respondents x $46.40/hour x 16 hours to complete	$12,620.80
PIs – Conferences, EAGER, RAPID	15 respondents x $46.40/hour x 16 hours to complete	$11,136.00
Annual report
PIs – Full-scale and Broad Implementation	69 respondents x $46.40/hour x 8 hours to complete	$25,612.80
PIs – Research, Pathways, CRPA	51 respondents x $46.40/hour x 4 hours to complete	$9,465.60
PIs – Conferences, EAGER, RAPID	N/A (no respondents)	N/A
Closeout report
PIs – Full-scale and Broad Implementation	23 respondents x $46.40/hour x 16 hours to complete	$17,075.20
PIs – Research, Pathways, CRPA	17 respondents x $46.40/hour x 10 hours to complete	$7,888.00
PIs – Conferences, EAGER, RAPID	15 respondents x $46.40/hour x 10 hours to complete	$6,960.00
Total		$116,371.20

A.13. Estimate of Total Capital and Startup Costs/Operation and Maintenance Costs to Respondents or Record Keepers

There is no overall annual cost burden to respondents or record-keepers that results from ISE other than the time spent responding to the survey instruments that are attached as appendices to this request.

It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested in aggregate form is from records that are maintained as part of normal educational practice. Furthermore, the respondents are active participants in programs or projects funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, the PIs who are the respondents to the individual data collections tasks for ISE make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance cost under OMB 3145-0058, and software (e.g., Microsoft Explorer or Mozilla Firefox) to respond.

A.14. Estimates of Costs to the Federal Government

The total estimated cost to the government of all data collection, analysis, and reporting activities for this study is approximately $313,096. This estimate is based on the actual annual cost of the collection in 2008 (shown in Exhibit 3).

Exhibit 3: Estimated Annual Cost to the Federal Government of Collection (based on 2008 expenditures)
Personnel	$102,267
Travel and field expenses	$829
Computing	$6,419
Copying, telephone, and postage	$489
Overhead	$111,260
G&A and Fee	$91,832
Total Costs	$313,096

A.15. Changes in Burden

No changes to the response burden estimates for individual respondents presented above are anticipated. As noted previously, the annual number of respondents has increased to reflect the number of ISE awards and the addition of new project types to the ISE OPMS.

A.16. Plans for Publication, Analysis and Schedule

The data are being collected for internal review purposes and to monitor the ISE projects, as well as for reporting to Congress. Reports to NSF management, PIs, and Congress dealing with the characteristics and performance of the ISE Program include tables and charts generated from the database.

After the products are delivered, NSF determines whether the quality of the products deserves publication verbatim by NSF (i.e., NSF is the exclusive publisher of the information being gathered). Often it is only after seeing the quality of the information delivered by the study that NSF decides the format (raw or analytical) and manner (in the NSF-numbered product Online Document System (ODS) or simply a page on the NSF website) in which to publish.

Before the conclusion of the study, both NSF and the funded ISE projects may use preliminary data to improve management and performance. For example, data generated by this study are expected to be inputs to other internal and external NSF reports.

The next round of data collection through the OPMS is scheduled for September 2009; at this time, an initial submission will be completed by recent ISE awardees (i.e., those receiving initial funding in 2008-09), and an annual report (or closeout report for those projects with a short duration) will be completed by the 47 projects from which the OPMS has already collected an initial submission.

A.17. Approval to Not Display Expiration Date

Not applicable.

A.18. Exceptions to Item 19 of OMB Form 83-I

No exceptions apply.

1 The system does not measure actual impacts; rather, we use the term “impacts” here in a manner specific to the nomenclature of the ISE program.

2 Estimates are provided for a typical annual collection cycle. Hours per respondent type figures and annual person hour totals are based on an average duration of 3.5 years for all award types excluding Conference, EAGER, and RAPID awards (i.e., 40 awards per year). Conference, EAGER, and RAPID awards each have an average duration of 1 year. Therefore, the 69 Full-scale and Broad Implementation respondents listed as completing an annual report are comprised of those projects who received Full-scale and Broad Implementation awards (23 each year) during the prior three years. Similarly, the 51 Research, Pathways, and CRPA respondents listed as completing an annual report are comprised of those projects who received these types of awards (17 each year) over the prior three years.

13