Request For Clearance: National Science Foundation, Directorate of Education and Human Resources, Divisions of Research, Evaluation and Communication and Undergraduate Education
Attachment I
This request for Office of Management and Budget (OMB) review asks for a renewal of clearance of the program monitoring data collection for the National Science Foundation (NSF) Division of Undergraduate Education (DUE) Science, Technology, Engineering, and Mathematics Talent Expansion Program (STEP) under the Directorate for Education and Human Resources (EHR) Generic Clearance OMB 3145-0136, which expires on January 31, 2008. The EHR Generic Clearance is for the collection of monitoring data on education and training activities supported by the NSF.
The Science, Technology, Engineering, and Mathematics Talent Expansion Program (STEP), authorized under the National Science Foundation (NSF) Reauthorization Act of 2002 (P.L. 107-368), responds to the critical need for a highly trained and educated workforce in established or emerging science, technology, engineering, and mathematics (STEM) disciplines (Type 1) and for educational research on retention and degree attainment in STEM (Type 2). For further information about the NSF STEP program, please go to http://www.nsf.gov/funding/pgm_summ.jsp?pims_id=5488&org=DUE&from=fund.
STEP seeks to increase the number of students (U.S. citizens or permanent residents) receiving associate's or baccalaureate degrees in established or emerging STEM fields. Increasing the number of undergraduate students obtaining degrees in STEM fields will provide a workforce that is prepared to ensure a healthy economy, respond to demands for national security, and maintain and elevate the quality of life and standard of living in the United States through technological and scientific advancements.
The program awards grants to institutions of higher education in the United States and its territories, to consortia thereof, or to nonprofit entities that have established consortia among such institutions of higher education. The institutions of higher education must offer either associate's degrees or baccalaureate degrees in STEM. Associate's degree-granting institutions with a demonstrated record of articulation to STEM baccalaureate programs need not necessarily grant associate's degrees in STEM fields in order to be eligible for this program.
The STEP program supports the NSF strategic outcome goal of cultivating "a world-class, broadly inclusive science and engineering workforce," and expanding "the scientific literacy of all citizens," labeled as 'Learning' on page 5 of the FY 2006–2011 Strategic Plan, http://www.nsf.gov/pubs/2006/nsf0648/NSF-06-48.pdf. In particular, the program will help promote NSF's Learning-related investment priorities to "develop methods to effectively bridge critical junctures in STEM education pathways" and "prepare a diverse, globally engaged STEM workforce" (reference page 7 of the plan). The information collected by the program monitoring system will inform how the STEP program is contributing to the NSF plan.
The STEP Program Monitoring System will be the primary source of data on the extent to which project goals are being achieved. The monitoring system consists of a Web-based data collection instrument that will be completed by the principal investigator (PI) of the lead institution for each STEP award and by the institution leader for each partner institution associated with the award.
The information collected in this task is required for effective administration, communication, and program and project monitoring; for meeting reporting requirements; for measuring attainment of NSF's program, project, and strategic goals as laid out in NSF’s Strategic Plan; and as a baseline for future program evaluations.
The primary purpose of this collection is program management, also known as program monitoring. This data collection activity is designed to track the extent to which STEP projects meet the objectives of the program. Within the DUE division, this information is used to administer and monitor the progress of the program. The findings are used to recommend, among other things, administrative changes in program functions, level of award support, individual program focus and emphasis, and recruiting efforts.
The STEP program also uses the data to fulfill reporting requirements. As a part of its performance assessment activities, NSF relies on the judgment of external experts to maintain high standards of program management. Directorate and Office advisory committees (ACs) meet twice a year, while Committees of Visitors (COVs) for divisions or programs meet once every three years. Data collected in the STEP monitoring system may be used to report to these committees on program activities. In addition, NSF is required to measure the attainment of its program, project, and strategic goals by the President's Management agenda as represented by the Office of Management and Budget's (OMB) Program Assessment Rating Tool (PART), by the Government Performance and Results Act (GPRA) of 1993, and by the NSF’s Strategic Plan. Data collected in the system help NSF management examine their progress towards the Foundation’s goals and respond to these reporting requirements.
Finally, the data can also be used as a preliminary step in more detailed future evaluation efforts, such as the sort of rigorous evaluations described in the May 2007 Report of the Academic Competitiveness Council (ACC), which was established by the Deficit Reduction Act of 2005 (P.L. 109-171) to serve as a multi-agency effort to identify Federal STEM education programs and establish their effectiveness. The full ACC report can be accessed at http://www.ed.gov/about/inits/ed/competitiveness/acc-mathscience/index.html.
Under the STEP Program Monitoring System, each principal investigator (PI)/institution leader of a STEP award is required to provide annual data using the Web-based data collection system (see appendix A). The following is an overview of the types of information collected:
A crosswalk of data elements can be found in Appendix B. Appendices A1 and A2 include copies of the offline data collection forms for 2-year and 4-year institutions (The offline data collection forms are Microsoft Excel documents with the questions that are provided to respondents prior to opening the online data collection system to assist them with early data collection efforts. These forms can be e-mailed to the various individuals who will be involved in preparing the award's response to inform them of the kinds of data to be collected.)
Web-based systems can facilitate respondents' data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough editing of all submitted data for completeness, validity, and consistency. Programmed data validations prevent most invalid data from entering the system, and questionable or incomplete entries are called to respondents' attention before they are submitted to NSF. Web-based data collection instruments employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus and predefined charts and graphics. All of these features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden.
The data for this monitoring effort are collected by 508-compliant Web-based data collection instruments. The instrument structure allows respondents to move between a menu screen and screens addressing individual topics. The question format is composed of text boxes, radio buttons, and checkboxes. Respondents may enter and leave the instrument as often as they desire and continue to change their responses until they complete their authorized final submission. Respondents have access to screen-specific help material as well as a Help & Instructions feature, which includes a discipline crosswalk and definitions for each race/ethnicity category, to assist them in completing their responses. Additionally, since the collection is Web-based, minor changes in wording and displays can be easily made in response to user feedback.
This system does not duplicate other NSF efforts. Comparable data are not currently being collected on an annual basis for the STEP program. In addition, the collection is coordinated with the NSF FastLane Project Reports system (OMB 3145-0058) to ensure that the two collections do not collect similar data. Aggregate data will be shared with NSF-funded researchers as appropriate, thereby minimizing the possibility that other researchers will duplicate these efforts in their own future collections.
No information is to be collected from small businesses.
Without this information NSF would be unable to document the effectiveness or output of the STEP program. The Foundation would be unable to disseminate information to other projects and institutions about successful approaches to increasing the number of STEM graduates or STEM majors transferring to 4-year STEM programs. Additionally, without this feedback NSF would have no way of making systematic modifications to the STEP program (e.g., adequacy of funding amount, duration of award, institutional supports needed). Moreover, NSF would be unable to comply fully with the congressional mandate that the Foundation evaluate its science, technology, engineering, and mathematics education programs.
In addition, the enabling legislation for the STEP program (Public Law 107–368, National Science Foundation Authorization Act of 2002) stipulates the following:
(D)(i) In order to receive a grant under this paragraph, an eligible applicant shall establish targets to increase the number of students studying toward and completing associate’s or bachelor’s degrees in science, mathematics, engineering, or technology.The data collection will comply with 5 CFR 1320.6.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register August 24, 2007, Volume 72, Number 164, page 48692. No comments were received.
During the initial system development the narrative questions were sent to eight STEP award PIs; seven of these responded, and their responses were evaluated to ensure that the questions were eliciting the desired information. (A list of the pilot test participants is included below.) Further revisions to instruction and question wording were made based on respondent feedback from the first data collection cycle. Macro International Inc. will maintain the online data collection system and databases, and will provide technical support to respondents as needed.
No payments or gifts will be provided to respondents.
Respondents will be advised that any information on specific individuals will be maintained in accordance with the Privacy Act of 1974. Data collected are available to NSF officials and staff, evaluation contractors, and the contractors hired to manage the data and data collection software. Data are processed according to Federal and State privacy statues. Detailed procedures for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). That system limits access to personally identifiable information to authorized users. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c. The information requested may be disclosed to qualified researchers and contractors in order to coordinate programs and to a Federal agency, court or party in a court, or Federal administrative proceeding, if the government is a party.
The opening screen of the STEP system states the following:
Information from this data collection system will be retained by the National Science Foundation (NSF), a Federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). These are confidential files accessible only to appropriate NSF officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form, or data explicitly requested as “for general use,” will be made available to anyone outside of the National Science Foundation for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c.
The STEP Program Monitoring System collects the name, telephone and fax numbers, and e-mail address of each PI/institution leader. The instrument does not collect information on individual students, only aggregated counts of majors, degrees granted, and transfers to 4-year institutions, broken out by gender and race/ethnicity. This information is collected in order to monitor the award sites, to provide continuity, and to assess the success of the award programs. The respondents will be PIs (for lead institutions) and institution leaders (for partner institutions) who will presumably obtain the student demographic data from their institution's Institutional Research offices. Individualized (institution-specific) data are provided only to STEP program staff and to contractors authorized by NSF. Any public reporting of the data will be in aggregate form.
The total number of average annual respondents is 177 and the estimated average annual response burden is 3,295 person-hours. The Web-based data collection is an annual activity of the STEP program. The population for the first year of the clearance will consist of 77 STEP awards, including awards granted in FY 2003, FY 2004, FY 2005, and FY 2006. Data will be collected from each lead institution and its corresponding partner institution(s); the respondents will be award PIs (for lead institutions) and institution leaders (for partner institutions). For the first year of the clearance, there will be 77 PIs and 59 institution leaders, or 136 total respondents. For the second year of the clearance 22 additional awards, including 22 lead institutions/PIs and 19 partner institutions/institution leaders, will be surveyed. In the third year of the clearance we anticipate that an additional 22 awards, including 22 lead institutions/PIs and 19 partner institutions/institution leaders, will be surveyed. The total number of annual respondents was calculated as follows:
|
Collection/Clearance Year
|
Type of Respondent
|
Number of Respondents
|
Burden Hours Per Respondent
|
Annual Person-Hours
|
|
One
|
New award PIs/institution leaders
|
46
|
30 hours
|
1,380 person-hours
|
|
One
|
Returning award PIs/institution leaders
|
90
|
15 hours
|
1,350 person-hours
|
|
Two
|
New award PIs/Institution leaders
|
41
|
30 hours
|
1,230 person-hours
|
|
Two
|
Returning award PIs/institution leaders
|
136
|
15 hours
|
2,040 person-hours
|
|
Three
|
New award PIs/institution leaders
|
41
|
30 hours
|
1,230 person-hours
|
|
Three
|
Returning award PIs/institution leaders
|
177
|
15 hours
|
2,655 person-hours
|
|
Three-Year Total
|
|
531
|
|
9,885 person-hours
|
|
Average Annual
|
|
177
|
|
3,295 person-hours
|
The overall annualized cost to the respondents for the STEP data collection is estimated to be $121,915. The following table shows the annualized estimates of costs to PI/institution leaders respondents, who are generally university professors. The estimated hourly rate is based on a report in the April 20, 2007, edition of The Chronicle of Higher Education (2007. "What Professors Earn." The Chronicle of Higher Education, 53(33), Washington, D.C.: The Chronicle of Higher Education, Inc.) According to the report, the average salary of an associate professor across all types of doctoral-granting institutions (public, private, church-related) was $76,639. When divided by the number of standard work hours (2,080), the average hourly wage is approximately $37 per hour.
|
Collection/Clearance Year
|
Type of Respondent
|
Number of Respondents
|
Burden Hours Per Respondent
|
Average Hourly Rate
|
Estimated Annualized Cost
|
|
One
|
New award PIs/institution leaders
|
46
|
30 hours
|
$37
|
$51,060
|
|
One
|
Returning award PIs/institution leaders
|
90
|
15 hours
|
$37
|
$49,950
|
|
Two
|
New award PIs/institution leaders
|
41
|
30 hours
|
$37
|
$45,510
|
|
Two
|
Returning award PIs/institution leaders
|
136
|
15 hours
|
$37
|
$75,480
|
|
Three
|
New award PIs/institution leaders
|
41
|
30 hours
|
$37
|
$45,510
|
|
Three
|
Returning award PIs/institution leaders
|
177
|
15 hours
|
$37
|
$98,235
|
|
Three-Year Total
|
|
|
|
|
$365,745
|
|
Average Annual
|
|
|
|
|
$121,915
|
There is no overall annual cost burden to respondents or record-keepers that results from the distance monitoring of the STEP program other than the time spent responding to the data collection instrument.
It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested is from records that are maintained as part of normal educational or training practice. Furthermore, the majority of PIs/institution leaders are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, PIs/institution leaders who are the respondents to the STEP data collection make use of standard office equipment (e.g., computers) and Internet connectivity that are already required as a startup cost and maintenance cost under the NSF GPG. The information requested is typical of educational and research portfolios and would be maintained as part of normal practice.Computing the annualized cost to NSF for the STEP data collection was done by taking the budgets for three years and calculating the costs for each of the following operational activities involved in producing, maintaining, and conducting the STEP data collection:
|
Operational Activities |
Costs Over Three Years |
|
System development (includes initial development of the database and Web-based application and later changes requested by the program, e.g., increased reporting tools, additional validations) |
$609,600 |
|
System maintenance, updates, and technical support (system requires updates each year before opening the collection; maintenance is required to keep the system current with technology, e.g., database servers, operating systems) |
$260,800 |
|
Data collection opening and support (e.g., online and telephone support to respondents and contacting respondents to encourage completion of their responses), reporting (as defined by the Division of Undergraduate Education), and followup activities (e.g., providing data to other consultants) |
$323,150 |
|
Three-Year Total for All Operational Activities |
$1,193,550 |
The annualized cost was computed as one-third of the total three-year costs; thus, the annualized cost to NSF for the STEP data collection is $397,850.
The previously reported total annual response burden for this collection was 4,972 person-hours for 161 respondents; the current request for 3,295 person-hours for 177 respondents is a decrease of 1,677 person-hours. The number of respondents has increased, and the estimates of burden hours per respondent are based on actual burden data collected during the initial data collection cycle. The initial collection cycle collected data on up to four reporting periods. The collection cycles covered by this request will collect data on up to two reporting periods at most (i.e., baseline and current year).
There have been no major changes in the instrument that would affect the burden. Minor changes have been made to the wording of questions to clarify the desired responses. In addition, some text boxes have been replaced with check boxes to help standardize responses.
Data collection is scheduled to begin around November each year, and award sites will have approximately 60 days to enter data; extensions will be granted by NSF program officers as necessary. Once the data collection has been completed, agency staff can access the data through the online system as needed.
Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. Macro International, the contractor that manages the online data collection system and database, is forbidden contractually from publishing results unless NSF has made a specific exception. In short, all products of the collections are the property of NSF and NSF is the exclusive publisher of the information being gathered. Often it is only after seeing the quality of the information collected that NSF decides the format (raw or analytical) and manner (in the NSF-numbered product Online Document System (ODS) or simply a page on the NSF Web site) in which to publish.
The data from this collection will be used for internal review purposes and to monitor the STEP projects, as well as for reporting to Congress and OMB (e.g., the GPRA and PART reviews). Reports to NSF management, PIs, and Congress dealing with characteristics and performance of the STEP program will include statistical tables and charts generated from the database. At this time, NSF has no set timeline for publishing interim reports from this study.
Not Applicable
No exceptions apply.
The sample size is the entire universe of respondents. Over the three-year clearance period, we estimate that there will be an average of 177 respondents per year.
|
Population |
Estimated Universe Size |
Sample Size |
|
STEP respondents |
177 |
177 |
This data collection uses a Web-based instrument. The PIs and institution leaders associated with each STEP award will be required to provide project information each year during the duration of their NSF funding. The latest program solicitation for the STEP program can be found here: http://www.nsf.gov/publications/pub_summ.jsp?ods_key=nsf07570.
This data collection is a census, so no sampling is required.
Not Applicable
Not Applicable
Not Applicable
Not Applicable
The first year of data collection had a 100 percent response rate and NSF anticipates that the rate will remain the same. Considerable effort is made for follow up for non-response and incomplete responses. This is achieved by sending e-mail every three weeks to respondents who have not logged into the system, and by notifying all respondents still entering data when the system closing date is one week away. Approximately 60 percent of respondents receive at least one of these follow-up e-mail messages. Examples of e-mail messages that will be sent to the STEP respondents are included in appendix D. The collection is part of reporting required of awardees.
The questions in this collection were reviewed by STEP PIs during the development of the system. In addition, many of the items and response categories follow formats that are already in use within other tasks, such as the Noyce Program Monitoring System, also cleared as part of the EHR Generic Clearance (OMB 3145-0136), including demographic questions. User feedback is also taken into consideration during system updates.
Agency
Susan H. Hixson, National Science Foundation, (703) 292-4623
William P. Neufeld, National Science Foundation, (703) 292-5148
Bernice Anderson, National Science Foundation (703) 292-5151
Contractors
Macro International of Bethesda, MD will be responsible for data collection and analysis under the direction of Lea Mesner, (301) 657-3070.| File Type | text/html |
| File Title | Supporting Statement |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |