REQUEST FOR CLEARANCE
National Science Foundation
Directorate of Education and Human Resources
Division of Undergraduate Education
Transforming Undergraduate Education in Science, Technology,
Engineering, and Mathematics
Monitoring System (TUES)
Attachment J
This request for Office of Management and Budget (OMB) review is part of the renewal process for the National Science Foundation (NSF) Directorate for Education and Human Resources (EHR) Generic Clearance, OMB 3145-0136, which will expire on March 31, 2011. The EHR Generic Clearance includes collections of information about NSF's education and training (E&T) activities. This particular request addresses management or monitoring for the Transforming Undergraduate Education in Science, Technology, Engineering, and Mathematics (TUES) program within EHR's Division of Undergraduate Education (DUE).
The TUES program seeks to improve the quality of science, technology, engineering, and mathematics (STEM) education for all undergraduate students. This program especially encourages projects that have the potential to transform undergraduate STEM education, for example, by bringing about widespread adoption of classroom practices that embody understanding of how students learn most effectively. Thus transferability and dissemination are critical aspects for projects developing instructional materials and methods and should be considered throughout the project's lifetime. More advanced projects should involve efforts to facilitate adaptation at other sites.
The program supports efforts to create, adapt, and disseminate new learning materials and teaching strategies to reflect advances both in STEM disciplines and in what is known about teaching and learning. It funds projects that develop faculty expertise, implement educational innovations, assess learning and evaluate innovations, prepare K-12 teachers, or conduct research on STEM teaching and learning. It also supports projects that further the work of the program itself, for example, synthesis and dissemination of findings across the program. The program supports projects representing different stages of development, ranging from small, exploratory investigations to large, comprehensive projects.
Data collected from TUES awards through the monitoring system are needed by NSF for project and program monitoring, to fulfill policy and program reporting needs, and to serve as preliminary work for future impact assessment and evaluation activities.
The information that will be collected in this task is required for effective administration, communication, and program and project monitoring; for meeting reporting requirements; for measuring attainment of NSF's program, project, and strategic goals as laid out in NSF’s Strategic Plan; and as a baseline for future program evaluations.
The primary purpose of this data collection is to provide data and information for effective program management and monitoring of program activities. This data collection activity will be designed to track the extent to which TUES awards meet the objectives of the program. Within the DUE division, this information will be used to administer and monitor the progress of the program. The findings will be used to recommend, among other things, administrative changes in program functions, level of award support, individual program focus and emphasis, and recruiting efforts.
In recent guidance from the Director of OMB, M-10-32, the need for rigorous evaluations and the objectives of evaluations of programs were clearly outlined, including the use of evaluation resources. Because the collection of data contained in this monitoring effort will contribute to the formal evaluation of the program and will provide regular measures of program performance by accumulating operating information from each project in the program, this guidance is particularly pertinent to this request.
“Improving and coordinating the use of existing evaluation resources. In addition to the voluntary evaluation initiative, agencies should continue to carefully assess, report on, and allocate the base funds and resources that the agencies have for conducting evaluation. Agencies are encouraged to share information beyond what is requested in guidance and consult with OMB’s Resource Management Offices (RMOs) to coordinate and improve the design, implementation, and utilization of evaluations.”
These directives establish an ongoing need for NSF to engage in an interactive process of collecting information and using it to improve program services and processes.
The TUES program will also use the data to fulfill reporting requirements. As a part of its performance assessment activities, NSF relies on the judgment of external experts to maintain high standards of program management. Directorate and Office advisory committees (ACs) meet twice a year, while Committees of Visitors (COVs) for divisions or programs meet once every three years. Data collected in the TUES monitoring system may be used to report to these committees on program activities. In addition, NSF is required to measure the attainment of its program, project, and strategic goals, as identified by the President’s Accountable Government Initiative, the Government Performance and Results Act (GPRA) Modernization Act of 2010, and the NSF’s Strategic Plan. Data collected in the TUES monitoring system will help NSF management examine their progress towards the Foundation’s goals and respond to these reporting requirements.
Finally, the data can also be used as a preliminary step in more detailed future evaluation efforts. EHR makes these data available to NSF staff, EHR contractors with responsibility for the collection, and DUE program managers and their staff and contractors. Information collected may also be disseminated, in aggregate form, to current and prospective applicants to the TUES program and to the broad science, technology, engineering, and mathematics education community upon request to and approval from the NSF.
The two award types for which data will be collected are summarized below:
The general categories of data that are collected are summarized below:
Like other tasks under this generic clearance request, TUESWeb will use advanced electronic information technology to minimize data duplication and respondent burden. EHR tends to favor Web-based systems because they can facilitate respondents’ data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough editing of all submitted data for completeness, validity, and consistency. Editing is performed as data are entered. Most invalid data cannot enter the system, and questionable or incomplete entries are called to respondents’ attention before they are submitted to NSF. TUESWeb data fields will be marked with out-of-range indicators, warning respondents to check their data if they appear to be out-of-range. Web-based systems also employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus, and predefined charts and graphics. All of these features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden. Other efforts to reduce burden will include automatically entering each principal investigator’s (PI's) contact information and the amount and dates of their award(s).
The TUESWeb data collection will not duplicate other NSF efforts. Comparable data are not currently collected on an annual basis for the TUES program. In addition, the collection will be coordinated with the NSF FastLane Project Reports system (OMB 3145-0058) to ensure that the two collections do not collect similar data. Aggregate data will be shared with NSF-funded researchers as appropriate, thereby minimizing the possibility that the other researchers will duplicate these efforts in their own future collections.
Only a small amount of data will be collected from any small business organizations, with the total small business response burden being less than one percent of the total response burden. Based on current data, fewer than five small businesses would be affected by this data collection. Together these businesses hold fewer than five awards in total and each small business would spend no more than 15 hours responding per award.
Without this information, NSF would be restricted in managing and reporting on the activities of awards in the TUES program. Without this feedback, NSF would have no way of making systematic modifications to the TUES program (e.g., adequacy of funding amount, duration of award, and institutional supports needed). These data will ensure that NSF makes informed decisions about future directions of the TUES program. The information requested here is not available elsewhere.
Additionally, without this information NSF would find it difficult to meet GPRA and OMB reporting requirements and would be unable to comply fully with congressional and presidential mandates that the Foundation asses its STEM education programs.
The data collection will comply with 5 CFR 1320.6.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register November 9, 2010, Volume 75, Number 216, pages 68829-68830. No comments were received.
The PIs who will be using the TUESWeb system will be consulted in its design and planning. A pilot test (involving no more than nine participants) will be conducted before the system launches in summer 2011. Feedback from the system users also be solicited at the annual TUES PI meeting, usually held in the winter, and user comments submitted during the collection periods also will be taken into consideration for future system improvements.
No payments or gifts will be provided to respondents.
Respondents will be advised that any information on specific individuals will be maintained in accordance with the Privacy Act of 1974. Data collected are available to NSF officials and staff, evaluation contractors, and the contractors hired to manage the data and data collection software. Data are processed according to Federal and State privacy statues. Detailed procedures for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). That system limits access to personally identifiable information to authorized users. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c. The information requested may be disclosed to qualified researchers and contractors in order to coordinate programs and to a Federal agency, court or party in a court, or Federal administrative proceeding, if the government is a party.
The opening screen on TUESWeb will contain language similar to other EHR data collection systems, which state the following:
Information from this data collection system will be retained by the National Science Foundation (NSF), a Federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). All individually identifiable information supplied by individuals or institutions to a Federal agency may be used only for the purposes outlined in the system of records notice and may not be disclosed or used in identifiable form for any other purpose, unless otherwise compelled by law. These are confidential files accessible only to appropriate NSF officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form, or data explicitly requested as “for general use,” will be made available to anyone outside of the NSF for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c.
The exact language has not yet been determined because NSF has no plans for TUESWeb to collect data based on gender, race, ethnicity, or disability for anyone.
TUES will request data that are considered of a private nature, including the name, phone number, and e-mail address only from the PIs, faculty members, and other project personnel who will use TUESWeb. Individuals' data will be provided only to TUES program staff and consultants conducting studies using the data as authorized by NSF. TUESWeb will collect all other data about individuals in aggregated form. Any public reporting of the data will be in aggregate form.
The total average number of annual respondents is approximately 1,100 and the average annual burden hour total is 16,500. The Web-based collection will be an annual activity of the TUES program beginning in summer 2011. There are approximately 1,100 active respondents at a given point in time, and new respondents enter at approximately the same rate that respondents leave the program or projects as their funding expires.
|
Type of |
Average Number of Respondents |
Burden Hours per Respondent |
Annual Burden Hours |
|
PIs |
1,100 |
15 |
16,500 |
|
Total |
1,100 |
15 |
16,500 |
As mentioned above respondents will be project PIs. The estimated total annual response burden is 16,500 hours. The annual burden by form was calculated as follows:
|
Form Type |
Respondent Type |
Number of Respondents |
Burden Hours per Respondent |
Total Annual Burden Hours |
|
TUES data collection form |
PIs |
1,100 |
15 |
16,500 |
|
Total |
|
1,100 |
15 |
16,500 |
The overall annualized cost to the respondents is estimated to be $660,000. The following table shows the annualized estimates of costs to PI respondents, who are generally university professors. These estimated hourly rates are based on a report in the April 16, 2010, edition of The Chronicle of Higher Education (2010). (“What Professors Earn.” The Chronicle of Higher Education, 56(31), A10, Washington, D.C.: The Chronicle of Higher Education, Inc.) According to the report, the average salary of an associate professor across all types of doctoral-granting institutions (public, private, church-related) was $83,511. When divided by the number of standard annual work hours (2,080), this calculates to $40 per hour.
|
Respondent Type |
Number, Rate, and Burden |
Costs |
|
PIs |
(1,100 x $40/hour x 15 hours) |
$660,000 |
|
Total |
(1,100 x $40/hour x 15 hours) |
$660,000 |
There is no overall annual cost burden to respondents or record-keepers that results from the monitoring of the TUES program other than the time spent responding to the data collection depicted by the wireframe sketches attached as appendix A to this request.
It is usual and customary for individuals involved in implementing a TUES award to keep descriptive records. The information being requested is from records that are maintained as part of normal practice at higher education institutions. Furthermore, the majority of respondents are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, PIs and other project personnel who are the respondents to the TUES data collection task make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance costs under OMB 3145-0058, and free software (e.g., Microsoft Explorer or Mozilla Firefox) to respond. Thus, there are no capital and startup costs or operation and maintenance costs to respondents or record-keepers.
|
Operational Activities |
Cost Over Three Years |
|
System Development (includes initial development of the database and Web-based application, and later changes requested by the program, e.g., increased reporting tools, additional validations) |
$612,000 |
|
System Maintenance, Updates, and Technical Support (the system requires updates each year before opening the collection; maintenance is required to keep the system current with technology, e.g., database servers, operating systems) |
$264,000 |
|
Data Collection Opening and Support (e.g., online and telephone support to respondents and contacting respondents to encourage completion of the questions), Reporting (as defined by DUE), and Followup Activities (e.g., providing data to other consultants) |
$324,000 |
|
Three-Year Total for All Operational Activities |
$1,200,000 |
The annualized cost was computed as one-third of the total three-year costs; thus, the annualized cost to NSF for the TUES data collection is $400,000.
This data collection will be launched for the first time in summer 2011. Therefore, there are no changes in burden.
The data from this collection will be used for internal review purposes and to monitor the TUES awards, as well as for reporting to Congress and OMB. Reports to NSF management, PIs, and Congress dealing with characteristics and performance of the TUES program could include statistical tables and charts generated from the database. At this time NSF has no set timeline for publishing interim reports from this study.
The first data collection is scheduled to begin in summer 2011 and remain open for two months. NSF program officers could potentially extend the deadline upon request of the respondents. Once the data collection has been completed, agency staff will have access to the data through the online system as needed.
Not applicable
No exceptions apply.
The sample size is the entire universe of PIs responsible for reporting on TUES awards. There will be an estimated average of 1,100 active PIs each year, and data will be collected on each of their awards. The individual respondents come from both the individual project's lead institution and other collaborating and partnering institutions of higher education.
|
Population |
Estimated Universe Size |
Sample Size |
|
Awardees |
1,100 |
1,100 |
This data collection will use a Web-based instrument. Each respondent will provide answers each year during the duration of their NSF funding.
This data collection is a census, so no sampling is required.
Not applicable
Not applicable
Not applicable
Not applicable
Similar collections in the EHR portfolio have had 100 percent response rates and NSF expects the same rate for this data collection. Considerable effort will be made for follow up for non-response and incomplete responses. Biweekly during the collection, the contractor will send e-mail messages and make telephone calls to project staff that have not logged into the system, and will notify all award sites that are still entering data when the system closing date is one week away. The collection is part of reporting required of awardees; PIs will be responsible for ensuring that data are collected.
This system is currently in development with a planned launch of summer 2011. As such, there have been no tests of procedures or methods.
Agency
Connie Della-Piana, Program Director, National Science Foundation, (703) 292-5309
Contractors
ICF Macro will be responsible for data collection and analysis under the direction of Lea Mesner, (301) 657-3070.
| File Type | text/html |
| File Title | Supporting Statement |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |