REQUEST FOR CLEARANCE
National Science Foundation
Directorate of Education and Human Resources
Division of Graduate Education
Integrative Graduate Education and Research Traineeship Program
Monitoring System (IGERT)
Attachment C
This request for Office of Management and Budget (OMB) review asks for a renewal of clearance for the monitoring data collection for the Integrative Graduate Education and Research Traineeship Program (IGERT) that is part of the National Science Foundation's (NSF) Directorate for Education and Human Resources (EHR) Generic Clearance (OMB 3145-0136), which expires on March 31, 2011. The EHR Generic Clearance includes collections of information about NSF's education and training (E&T) activities.
The IGERT program was initiated in 1998 to catalyze a cultural change in graduate education, for students, faculty, and institutions, by establishing innovative new models for graduate education and training in a fertile environment for collaborative research that transcends traditional disciplinary boundaries. It is also intended to facilitate diversity in student participation and preparation, and to contribute to a world-class, broadly inclusive, and globally engaged science and engineering workforce. IGERT awards are made to academic institutions in the United States and its territories that grant the Ph.D. degree and have research and training programs in science and engineering. The awards to each institution may be up to $500,000 per year for five years, with an additional $200,000 available for appropriate instrumentation or research materials during the first year of the award. The majority of the funds must be used for doctoral student stipends, training and educational activities, and related expenditures, such as student travel, publication costs, and recruitment. More information about the IGERT program can be found here.
Data collected from IGERT awards through the monitoring system are needed by NSF for project and program monitoring, to fulfill policy and program reporting needs, and to serve as preliminary work for future impact assessment and evaluation activities. The data collected as part of OMB 3145-0136 allow NSF officials to document the overall program investment in individual alliances, and make future funding and program policy decisions.
Crosswalks of questions on the data collection instruments are included in appendices B1 and B2.
The information collected in this task is required for effective administration, communication, and program and project monitoring; for meeting reporting requirements; for measuring attainment of NSF's program, project and strategic goals as laid out in NSF’s Strategic Plan; and as a baseline for future program evaluations.
In recent guidance from the Director of OMB, M-10-32, the need for rigorous evaluations and the objectives of evaluations of programs were clearly outlined, including the use of evaluation resources. Because the collection of data contained in this monitoring effort contributes to the formal evaluation of the program and provides regular measures of program performance by accumulating operating information from each project in the program, this guidance is particularly pertinent to this request.
“Improving and coordinating the use of existing evaluation resources. In addition to the voluntary evaluation initiative, agencies should continue to carefully assess, report on, and allocate the base funds and resources that the agencies have for conducting evaluation. Agencies are encouraged to share information beyond what is requested in guidance and consult with OMB’s Resource Management Offices (RMOs) to coordinate and improve the design, implementation, and utilization of evaluations.”
These directives establish an ongoing need for NSF to engage in an interactive process of collecting information and using it to improve program services and processes.
The IGERT program also uses the data to fulfill reporting requirements. As a part of its performance assessment activities, NSF relies on the judgment of external experts to maintain high standards of program management. Directorate and Office advisory committees (ACs) meet twice a year, while Committees of Visitors (COVs) for divisions or programs meet once every three years. Data collected in the IGERT monitoring system may be used to report to these committees on program activities. In addition, NSF is required to measure the attainment of its program, project and strategic goals, as identified by the President’s Accountable Government Initiative, by the Government Performance and Results Act (GPRA) Modernization Act of 2010, and by the NSF’s Strategic Plan. Data collected in the IGERT monitoring system help NSF management examine their progress towards the Foundation’s goals and respond to these reporting requirements.
The monitoring system includes two Web-based forms: the principal investigator (PI) form and the trainee form. The PI form collects basic information on program participants and program implementation and management. The IGERT trainee form collects information on trainee demographics, activities and accomplishments during the funding period, as well as information that will allow NSF to conduct future follow up with trainees. In order to encourage frank responses to questions, trainees have sole access to their own forms, although PIs can monitor trainees' completion of each section of the forms via a management screen. Together these forms allow for a comparison of the project as designed and implemented by the PI and faculty, and the project as it is experienced by the participant trainees.
The proposed changes to the IGERT system are in development and up-to-date screenshots of the system are not yet available. Screenshots of the previously cleared version of the system are included as appendices A1 and A1; however, please see appendices B1 and B2 for detailed crosswalks of data elements that indicate which data elements were added since the last clearance.
Finally, the data can also be used as a preliminary step in more detailed future evaluation efforts. EHR makes these data available to NSF staff, EHR contractors with responsibility for the collection, and DGE program managers and their staff and contractors. Information collected may also be disseminated, in aggregate form, to current and prospective applicants to the IGERT program upon request to and approval from the NSF.
EHR tends to favor Web-based systems because they can facilitate respondents' data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough editing of all submitted data for completeness, validity and consistency. Editing is performed as data are entered. Most invalid data cannot enter the system, and questionable or incomplete entries are called to respondents' attention before they are submitted to NSF. Web-based systems employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus and predefined charts and graphics. All these features facilitate the reporting process, provide useful and rapid feedback to the data providers and reduce burden.
The data for this monitoring effort are collected by 508-compliant Web-based instruments. The structure allows respondents to move between a menu screen and a screen addressing individual topics. The question format is primarily quick-response checkboxes, with text boxes provided for the addition of specific, outstanding examples. Respondents may enter and leave their questionnaires as often as they desire and continue to change their responses until they submit their forms. Respondents have access to an online glossary to assist them in understanding the specific meaning of terms in the context of these forms. Additionally, since the collection is Web-based, minor changes in wording and displays have been easily made in response to user feedback.
This system does not duplicate other NSF efforts. Comparable data are not currently being collected on an annual basis for the IGERT program. In addition, the collection is coordinated with the NSF FastLane Project Reports system (OMB 3145-0058) to ensure that the two collections do not collect similar data. As much as possible, data from other NSF monitoring collections are used to pre-fill IGERT items, further minimizing overall response burden. Additionally, aggregate data are being shared with NSF-funded researchers as appropriate, thereby minimizing the possibility that other researchers will duplicate these efforts in their own future collections.
No information is to be collected from small businesses.
Without this information NSF will be unable to document the effectiveness or outcomes of the IGERT program. The Foundation will be unable to disseminate information to other projects and institutions about successful approaches to the integration of research and education and graduate student training. Additionally, without this feedback NSF would have no way of making systematic modifications to the IGERT program (e.g., adequacy of funding amount, duration of award, institutional supports needed). Moreover, NSF will be unable to comply fully with the GPRA and other reporting requirements or with the congressional mandate that the Foundation evaluate its science, technology, engineering, and mathematics (STEM) education programs.
The data collection will comply with 5 CFR 1320.6.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register November 9, 2010, Volume 75, Number 216, pages 68829-68830. No comments were received.
During the initial system development PIs from IGERT awards reviewed the system; their responses to the PI questionnaire and their assessments of the trainee questionnaire were taken into account in the development of the system. Changes in the system since initial development are informed by ongoing consultations with the respondents, ICF Macro (the contractor that designed the Web interface and database system) and Abt Associates, Inc. (the contractor that performs program evaluations). ICF Macro currently maintains the instruments and databases and provides technical support to respondents as needed.
No payments or gifts will be provided to respondents.
Data collected under this task are only available to the respondents, NSF, and the firms hired to manage the data and data collection software. Data are processed according to Federal and State privacy statutes. To protect privacy, only composite data or graphical representations will be released to the public.
For the collection covered by this clearance request, when respondents are presented with the first screen of the instrument, they are additionally instructed as follows:
"The Federal Government has a continuing commitment to monitor its awards to identify and address any inequities based on gender, race, ethnicity, or disability of the PIs/co-PIs, trainees, or other participants.
Information from this data collection system will be retained by NSF, a Federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). All individually identifiable information supplied by individuals or institutions to a Federal agency may be used only for the purposes outlined in the system of records notice and may not be disclosed or used in identifiable form for any other purpose, unless otherwise compelled by law. These are confidential files accessible only to appropriate NSF officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form, or data explicitly requested as "for general use," will be made available to anyone outside of the NSF for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c. The Social Security number (SSN) and date of birth will be maintained in accordance with the requirements of the Privacy Act of 1974. Submission of the SSN is voluntary. It is used for survey quality control, program evaluation, and for matching with other data sets maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998)."
The IGERT questionnaire requests that each trainee provide his or her name, SSN, phone number, a personal e-mail address (e.g., aol.com or hotmail.com), and contact information for a person likely to know how to reach him or her in three years. These data are collected in order to ensure consistent monitoring and to permit followup studies that examine the long-term effect of the IGERT program on individuals' success. The IGERT questionnaire also collects the name, telephone number, e-mail address, fax number, disability status, and citizenship of each PI, co-PI, or advisor. Additionally, trainee Graduate Record Exam (GRE) scores are collected. Respondents have the option of not providing information that they consider privileged, such as disability status, by marking the "not reported" option on the form or by leaving the SSN field blank. Because the program requires that all IGERT trainees be U.S. citizens or permanent residents, the question of IGERT trainee citizenship is directly addressed both in the PI form (by asking the PI to verify the IGERT-required citizenship/residency requirement for each trainee) and in the trainee form (by not providing a "not reported" option for citizenship and also checking the genesis of trainees by requesting date and place of birth). All information will be maintained in accordance with the requirements of the Privacy Act of 1974. Individualized data are provided only to IGERT program staff and to contractors from Abt Associates, Inc. conducting site evaluations authorized by NSF. Any public reporting of the data is in aggregate form.
The total number of annual respondents is 3,756 (141 project PIs and 3,615 IGERT trainees and associates) and the annual burden hour total is 10,524. The Web-based collection is an annual activity of the IGERT program. There are currently 141 IGERT awards, and data is collected from each award site. PIs complete the PI questionnaire; all IGERT trainees are required to complete the IGERT trainee questionnaire, and the associates are requested but not required to complete the IGERT associate questionnaire. We anticipate that new awards may be added, but at about the same rate that active awards expire; thus, on average, the number of respondents will remain constant over time. The annualized burden was computed by taking the number of respondents from the current data collection cycle and estimating their response burden, based on a question in the Web-based data collection asking how long it takes respondents to complete the form. The burden estimates for each type of respondent are outlined below:
|
Type of |
Average Number of Respondents |
Burden Hours Per Respondent |
Total Annual Burden Hours |
|
Project PIs |
141 |
49 |
6,909 |
|
IGERT Trainees and Associates |
3,615 |
1 |
3,615 |
|
Total |
3,756 |
10,524 |
As mentioned above respondents will be project PIs and IGERT trainees and associates. The estimated total annual response burden is 10,524 hours. There is a different Web-based form for each respondent. The annual burden by form was calculated as follows:
|
Form Type |
Respondent Type |
Number of Respondents |
Burden Hours Per Respondent |
Total Burden Hours |
|
Principal Investigator form |
Project PIs |
141 |
49 |
6,909 |
|
Trainee and associate form |
IGERT Trainees and Associates |
3,615 |
1 |
3,615 |
|
Total |
|
3,756 |
10,524 |
The overall annualized cost to the respondents is estimated to be $312,510. The following table shows the annualized estimates of costs to respondents. These estimated hourly rates are based on a report in the April 16, 2010, edition of The Chronicle of Higher Education (2010). (“What Professors Earn.” The Chronicle of Higher Education, 56(31), A10, Washington, D.C.: The Chronicle of Higher Education, Inc.). According to the report, the average salary of an associate professor across all types of doctoral-granting institutions (public, private, church-related) was $83,511. When divided by the number of standard annual work hours (2,080), this calculates to approximately $40 per hour.
|
Respondents |
Number of Respondents |
Hours per Respondent |
Average Hourly Rate |
Total Annual Costs |
|
Project PIs |
141 |
49 |
$40 |
$276,360 |
|
IGERT Trainees and Associates |
3,615 |
1 |
$10 |
$36,150 |
|
Total |
3,756 |
$312,510 |
There is no overall annual cost burden to respondents or record-keepers that results from the monitoring of the IGERT program other than the time spent responding to the data collection instrument. It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested is from records that are maintained as part of normal educational or training practice. Furthermore, the majority of PIs are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, PIs who are the primary respondents to the IGERT data collection make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance costs under OMB 3145-0058, and free software (e.g., Microsoft Explorer or Mozilla Firefox) to respond. The information requested of trainees is typical of doctoral student educational and research portfolios and would be maintained as part of normal practice. Thus, there are no capital and startup costs or operation and maintenance costs to respondents or record-keepers.
Computing the annualized cost to NSF for the IGERT data collection was done by taking the budgets for three years and calculating the costs for each of the following operational activities involved in producing, maintaining, and conducting the IGERT data collection:
|
Operational Activities |
Cost Over Three Years |
|
System Development (includes initial development of the database and Web-based application and later changes requested by the program, e.g., increased reporting tools, additional validations) |
$664,796 |
|
System Maintenance, Updates, and Technical Support (system requires updates each year before opening the collection; maintenance is required to keep the system current with technology, e.g., database servers, operating systems) |
$332,360 |
|
Data Collection Opening and Support (e.g., online and telephone support to respondents and contacting respondents to encourage completion of the questions), Reporting (as defined by the DGE), and Followup Activities (e.g., providing data to other consultants) |
$398,753 |
|
Three-Year Total for All Operational Activities |
$1,395,909 |
The annual costs of the clearance was computed as one-third of the total three-year costs; therefore, the annualized cost to NSF for the IGERT data collection is $465,303.
The previously reported total annual response burden for this collection was 9,440 hours for 2,136 PIs and trainees; the current request for 10,524 hours for 3,756 respondents is an increase of 1,084 hours. This increase is due to gradual growth in the numbers of awards and trainees involved in the IGERT program.
Some changes have been made to the instruments since the last clearance, as some data elements were added or revised and others were deleted. However, the overall length of the questionnaire has not changed and we do not anticipate that these changes will affect the respondents’ burden. See the crosswalk of data elements in the appendices for more information on the changes to the questionnaire.
Data collection is scheduled to begin in March, and award sites will have 90 days to enter data; extensions are granted by NSF program officers as necessary. Once the data collection has been completed, agency staff can access the data through the online system as needed.
Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. ICF Macro, the contractor that manages the data collection Web site and database, is forbidden contractually from publishing results unless NSF has made a specific exception. In short, all products of the collections are the property of NSF and NSF is the exclusive publisher of the information being gathered. Often it is only after seeing the quality of the information collected that NSF decides the format (raw or analytical) and manner (in the NSF-numbered product Online Document System (ODS) or simply a page on the NSF Web site) in which to publish.
The data from this collection will be used for internal review purposes and to monitor the IGERT projects, as well as for reporting to Congress (e.g., the GPRA Annual Performance Plan). Reports to NSF management, PIs, and Congress dealing with characteristics and performance of the IGERT program will include statistical tables and charts generated from the database. At this time, NSF has not set timeline for publishing interim reports from this study.
Not applicable
No exceptions apply.
The sample size is the entire universe of respondents. There are currently 3,756 PIs and trainees working on IGERT awards and this number is expected to remain stable throughout the clearance period.
|
Population |
Estimated Universe Size |
Sample Size |
|
IGERT respondents |
3,756 |
3,756 |
This data collection uses a Web-based system. Each IGERT project provides descriptive data each year for the duration of their NSF funding. The data are primarily useful for program management, monitoring, and descriptive analysis.
NSF understands the limitations of the data collection, particularly in terms of using the data to determine program effectiveness. Data collected through the IGERT monitoring system are not used to determine the ultimate effectiveness of its STEM educational interventions, but are used in program planning and management, to report on agency activities and goals, and to lay the groundwork for future evaluations.
This data collection is a census, so no sampling is required.
Not applicable
Not applicable
Not applicable
Not applicable
Past collections have had 100 percent response rates from PIs and trainees and 60 percent from the associates (associate questionnaires are not required) and NSF anticipates that the rate will remain the same. This is achieved by sending e-mail messages every three weeks to award sites that have not logged into the system, and by notifying all award sites still entering data when the system closing date is one week away. Approximately 60 percent of award sites receive at least one of these followup e-mail messages. Examples of the e-mail messages announcing the opening of the system and reminding awardees to log in and enter data are included in appendix C. The collection is part of reporting required of awardees; principal investigators are responsible for ensuring that individual data are collected from trainees, and will have access to status information on the Web site indicating which trainees have not responded.
This system has been operational since 2000 and has been tested extensively. In addition, many of the items and response categories follow formats that are already in place at NSF.
Agency
Carol Van Hartesveldt, National Science Foundation, (703) 292-8112
Contractors
ICF Macro will be responsible for data collection and analysis under the direction of Lea Mesner, (301) 657-3070.
| File Type | text/html |
| File Title | Supporting Statement |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |