REQUEST FOR CLEARANCE
National Science Foundation
Directorate of Education and Human Resources
Division of Undergraduate Education
Scholarships in Science, Technology, Engineering, and Mathematics Program
Monitoring System (S-STEM)
Attachment G
This request for Office of Management and Budget (OMB) review asks for a renewal of clearance for the monitoring data collection for the Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM) program (formerly known as the Computer Science, Engineering, and Mathematics Scholarship program) that is part of the National Science Foundation's (NSF) Directorate for Education and Human Resources (EHR) Generic Clearance (OMB 3145-0136), which expires on March 31, 2011. The S-STEM program is administered by the EHR Division of Undergraduate Education (DUE).
The S-STEM program supports scholarships for low-income, academically talented students, enabling them to pursue associate, baccalaureate, or graduate-level degrees in computer science, computer technology, engineering, engineering technology, or mathematics. Academic institutions apply for the awards to support scholarship activities and are responsible for selecting scholarship recipients.
The program was established by NSF in accordance with the American Competitiveness and Workforce Improvement Act of 1998 and subsequent appropriations. The Act reflects the national need to increase substantially the number of American high technology workers and to develop high-quality professionals in these fields.
Data collected from S-STEM awards through the monitoring system are needed by NSF for project and program monitoring, to fulfill policy and program reporting needs, and to serve as preliminary work for future impact assessment and evaluation activities. Screenshots from the system can be found in appendix A.
The information collected in this task is required for effective administration, communication, and program and project monitoring; for meeting reporting requirements; for measuring attainment of NSF’s program, project and strategic goals as laid out in NSF’s Strategic Plan; and as a baseline for future program evaluations.
In recent guidance from the Director of OMB, M-10-32, the need for rigorous evaluations and the objectives of evaluations of programs were clearly outlined, including the use of evaluation resources. Because the collection of data contained in this monitoring effort contributes to the formal evaluation of the program and provides regular measures of program performance by accumulating operating information from each project in the program, this guidance is particularly pertinent to this request.
“Improving and coordinating the use of existing evaluation resources. In addition to the voluntary evaluation initiative, agencies should continue to carefully assess, report on, and allocate the base funds and resources that the agencies have for conducting evaluation. Agencies are encouraged to share information beyond what is requested in guidance and consult with OMB’s Resource Management Offices (RMOs) to coordinate and improve the design, implementation, and utilization of evaluations.”
These directives establish an ongoing need for NSF to engage in an interactive process of collecting information and using it to improve program services and processes.
The primary purpose of this collection is to provide data and information for effective program management and monitoring of program activities. The data collection activity is designed to track scholars in biological sciences (except medicine and other clinical fields); physical sciences, including physics, chemistry, astronomy, and materials science; mathematical sciences; and information computer science; geosciences; engineering; technology areas associated with the proceeding fields (for example biotechnology, chemical technology, engineering technology, information technology, etc), and establish whether the S-STEM project has been successful in attracting students. Within DUE this information is used to monitor the administration of the program in the various institutions, including the number and academic status of scholarship recipients in each institution that has an award. The information also is used to determine the types of individuals receiving scholarships, their academic status and standing, and their academic interest in the fields of interest to the S-STEM program. The primary focus of the collection is on descriptive data of scholars receiving support under the program provided by a single award to an institution. The intent is to determine the types of individuals, in the aggregate, that receive scholarships, and the extent of the support in subsequent decisions to further their education and/or pursue work in those fields. Findings are used to recommend, among other things, administrative changes in program functions, level of scholarship support, individual program focus and emphasis, and recruiting efforts.
The S-STEM program also uses the data to fulfill reporting requirements. As a part of its performance assessment activities, NSF relies on the judgment of external experts to maintain high standards of program management. Directorate and Office advisory committees (ACs) meet twice a year, while Committees of Visitors (COVs) for divisions or programs meet once every three years. Data collected may be used to report to these committees on program activities. In addition, NSF is required to measure the attainment of its program, project and strategic goals by the President’s Accountable Government Initiative, the Government Performance and Results Act (GPRA) Modernization Act of 2010, and the NSF’s Strategic Plan. Data collected helps NSF management examine their progress towards the Foundation’s goals and respond to these reporting requirements.
Finally, the data can also be used as a preliminary step in more detailed future evaluation efforts. EHR makes these data available to NSF staff, EHR contractors with responsibility for the collection, and DUE program managers and their staff and contractors. Information collected may also be disseminated, in aggregate form, to current and prospective applicants to the S-STEM program and to the broad undergraduate science, technology, engineering, and mathematics education community upon request to and approval from the NSF.
EHR tends to favor Web-based systems because they can facilitate respondents’ data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough editing of all submitted data for completeness, validity and consistency. Editing is performed as data are entered. Programmed data validations prevent most invalid data from entering the system, and questionable or incomplete entries are called to respondents’ attention before they are submitted to NSF. Web-based data collection instruments employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus and predefined charts and graphics. All these features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden.
The data for this monitoring effort are collected by 508-compliant Web-based data collection instruments. The instrument structure allows respondents to move between a menu screen and screens addressing individual topics. The question format is composed of text boxes, radio buttons, and checkboxes. Respondents may enter and leave the instrument as often as they desire and continue to change their responses as necessary. Respondents have access to detailed online help material to assist them in completing their responses. Additionally, since the collection is Web-based, minor changes in wording and displays can be easily made in response to user feedback. See appendix A for a copy of the data collection instrument.
The S-STEM data collection form does not duplicate other NSF efforts and the data requested are not available elsewhere. Whenever possible, EHR will use internal sources of information rather than request the same from participants.
For all multiyear grants (including both standard and continuing grants), the principal investigator (PI) at each institution must submit an annual project report to the cognizant Program Officer at least 90 days before the end of the current budget period. As part of the reporting responsibility, the PI is expected to provide NSF, using the attached form, with an accurate list of all scholarship recipients and their demographic characteristics within 30 days of the beginning of each semester/quarter, and to ensure that appropriate documentation (e.g., verification of eligibility, educational progress) on the scholarship recipients is maintained throughout the life of the program.
No information is to be collected from small businesses.
Without this information, NSF would be restricted in managing and reporting on the activities of awards in the S-STEM program. Without this feedback, NSF would have no way of making systematic modifications to the program (e.g., adequacy of funding amount, duration of award, and institutional supports needed). These data will ensure that NSF makes informed decisions about future directions of the S-STEM program. The information requested here is not available elsewhere.
Additionally, without this information NSF would find it difficult to meet agency data requests, as well as GPRA and OMB reporting requirements and would be unable to comply fully with congressional and presidential mandates that the Foundation asses its STEM education programs.
The data collections will comply with 5 CFR 1320.6.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register November 9, 2010, Volume 75, Number 216, pages 68829-68830. No comments were received.
Modifications have been made in the application over the time it has been in operation in response to user comments and suggestions. An updated version of the instrument was beta tested with seven award PIs in the summer of 2009, and their feedback informed further updates.
No payments or gifts will be provided to respondents.
Respondents will be advised that any information on specific individuals will be maintained in accordance with the Privacy Act of 1974. Data collected are available to NSF officials and staff, evaluation contractors, and the contractors hired to manage the data and data collection software. Data are processed according to Federal and State privacy statues. Detailed procedures for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). That system limits access to personally identifiable information to authorized users. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c. The information requested may be disclosed to qualified researchers and contractors in order to coordinate programs and to a Federal agency, court or party in a court, or Federal administrative proceeding, if the government is a party.
The Privacy & Public Burden Statements screen of the S-STEM system states the following:
Information from this data collection system will be retained by the National Science Foundation (NSF), a Federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). All individually identifiable information supplied by individuals or institutions to a Federal agency may be used only for the purposes outlined in the system of records notice and may not be disclosed or used in identifiable form for any other purpose, unless otherwise compelled by law. These are confidential files accessible only to appropriate NSF officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form, or data explicitly requested as “for general use,” will be made available to anyone outside of the NSF for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c. Date of birth will be maintained in accordance with the requirements of the Privacy Act of 1974.
The S-STEM system collects data that may be considered of a private nature, including the name, address, date of birth, and grade point average (GPA) of participating students. These data are collected in order to monitor the sites, assess the success of the program, and track recipients of funding. Individualized data are provided only to NSF program staff and to consultants conducting studies using the data as authorized by NSF, and any public reporting of the data is in aggregate form. Respondents have the option of not providing information that they consider privileged, such as disability status, by marking the “not reported” option on the form. Some S-STEM awards have asked their scholars to sign a release/consent form. All information will be maintained in accordance with the requirements of the Privacy Act of 1974.
The total number of annual respondents is 500. The estimated annual response burden is 6,500 hours. The frequency of response is an average of two times per year.
Respondents are principal investigators of S-STEM awards. There is an average of 500 active awards each year, with 500 total PIs (1 per award) and an average of 40 scholars per award. PIs must report on each student receiving S-STEM scholarship support for each semester or quarter, depending on the system used by their institution, for an average of two responses per year per PI. Because of the nature of the project, PIs will have most of the data on scholars readily available and will need to spend an average of only 10 minutes per semester or quarter entering data on each scholar, for a total annual burden of approximately 6.5 hours per PI. The burden was calculated as follows:
|
Respondent Type |
No. of Respondents |
Burden Hours per Respondent |
Responses per Year |
Annual BurdenHours |
|
PIs |
500 |
6.5 |
2 |
6,500 |
|
Total |
500 |
|
|
6,500 |
The S-STEM data collection consists of one form. As mentioned above, respondents will be faculty serving as project PIs. The estimated total annual response burden is 6,500 hours. Burden is minimized by the fact that the Web-based screens request data in simple data entry fields, including radio buttons, drop-down menus and text boxes, so little if any time is required for familiarization with the system. The annual burden by form was calculated as follows:
|
Form |
No. of Respondents |
Burden Hours per Respondent |
Responses per Year |
Annual Burden Hours |
|
S-STEM Data Collection Form |
500 |
6.5 |
2 |
6,500 |
|
Total |
500 |
|
|
6,500 |
The overall annualized cost to the respondents for the S-STEM data collection is estimated to be $260,000. The following table shows the annualized estimates of costs to PI respondents, who are generally university professors. The estimated hourly rate is based on a report in the April 16, 2010, edition of The Chronicle of Higher Education (2010). (“What Professors Earn.” The Chronicle of Higher Education, 56(31), A10, Washington, D.C.: The Chronicle of Higher Education, Inc.). According to the report, the average salary of an associate professor across all types of doctoral-granting institutions (public, private, church-related) was $83,511. When divided by the number of standard annual work hours (2,080), this calculates to approximately $40 per hour.
|
Type of Respondent |
No. of Respondents |
Burden Hours per Respondent |
Responses per Year |
Average Hourly Rate |
Estimated Annualized Cost |
|
PI |
500 |
6.5 hrs. |
2 |
$40/hr. |
$260,000 |
|
Total |
500 |
|
|
|
$260,000 |
There is no overall annual cost burden to respondents or record keepers that results from the S-STEM data collection other than the time spent responding to the instrument attached as an appendix to this request.
It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested is from records that are maintained as part of normal educational or training practice. Furthermore, respondents are recipients of awards funded by NSF. In order to receive this funding institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, S-STEM PIs make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance costs under OMB 3145-0058, and free software (e.g., Microsoft Explorer or Mozilla Firefox) to respond. The information requested is typical of educational and research portfolios and would be maintained as part of normal practice. Thus, there are no capital and startup costs or operation and maintenance costs to respondents or record-keepers.
Computing the annualized cost to NSF for the S-STEM data collection was done by taking the budgets for three years and calculating the costs for each of the following operational activities involved in producing, maintaining, and conducting the S-STEM data collection:
|
Operational Activities |
Cost Over Three Years |
|
System Development (includes initial development of the database and Web-based application, and later changes requested by the program, e.g., increased reporting tools, additional validations) |
$534,377 |
|
System Maintenance, Updates, and Technical Support (the system requires updates each year before opening the collection; maintenance is required to keep the system current with technology, e.g., database servers, operating systems) |
$267,188 |
|
Data Collection Opening and Support (e.g., online and telephone support to respondents and contacting respondents to encourage completion of the questions), Reporting (as defined by DUE), and Followup Activities (e.g., providing data to other consultants) |
$322,853 |
|
Three-Year Total for All Operational Activities |
$1,113,285 |
The annualized cost was computed as one-third of the total three-year costs; thus, the annualized cost to NSF for the STEP data collection is $371,095.
This burden of 6,500 hours represents an increase of 3,300 hours over the previously requested burden of 3,200 hours. This change is due to an increase in the average number of active awards and associated scholars. There have been no programmatic changes.
The S-STEM data collection is on-going, with respondents submitting data each semester/quarter that an award is active and we anticipate this schedule to continue.
Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. ICF Macro, the contractor that conducts this data collection on behalf of NSF, is forbidden contractually from publishing results unless NSF has made a specific exception. In short, all products of the collections are the property of NSF and NSF is the exclusive publisher of the information being gathered. Often it is only after seeing the quality of the information delivered by the study that NSF decides the format (raw or analytical) and manner (in the NSF-numbered product Online Document System (ODS) or simply a page on the NSF Web site) in which to publish.
The data from this collection will be used for internal review purposes and to monitor the S-STEM awards, as well as for reporting to Congress and OMB. Reports to NSF management, PIs, and Congress dealing with characteristics and performance of the S-STEM program will include statistical tables and charts generated from the database. At this time NSF has no set timeline for publishing interim reports from this study.
Not applicable
No exceptions apply.
The S-STEM data collection will involve approximately 500 PIs at 500 awards sites annually. As this represents the entire universe, no statistical sampling will be employed.
|
Population |
Estimated Universe Size |
Sample Size |
|
S-STEM PIs |
500 |
500 |
This data collection uses a Web-based system. The PIs will provide answers each year during the duration of their NSF funding. The latest program solicitation for the S-STEM program can be found here.
NSF understands the limitations of this data collection, particularly in terms of using the data to determine program effectiveness. Data collected through the S-STEM system are not used to determine the ultimate effectiveness of its STEM educational interventions, but are used in program planning and management, to report on agency activities and goals, and to lay the groundwork for future evaluations.
This data collection is a census, so no sampling is required.
Not applicable
Not applicable
Not applicable
Not applicable
The collection is part of reporting required of awardees. Submitting these data is required by the terms of the awards; a response rate of greater than 90 percent is expected. PIs are contacted if they fail to enter data for each semester/quarter in which their award is active.
The system underwent an update in the summer of 2009, and a beta test was conducted with seven S-STEM PIs. Further refinements to the system were made in response to the beta test results, and the program continues to make changes to the application in response to suggestions for improvements from users.
Agency Unit
Joyce Evans, National Science Foundation, (703) 292-5098
Dennis Davenport, National Science Foundation, (703) 292-4659
Bert Holmes, National Science Foundation, (703) 292-5128
Connie Kubo Della-Piana, National Science Foundation, (703) 292-5309
ICF Macro of Bethesda, MD will be responsible for data collection and analysis under the direction of Lea Mesner, (301) 657-3070.
| File Type | text/html |
| File Title | Supporting Statement |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |