Self-Evaluation Indicator System (SEIS) for Historically Black Colleges and Universities Undergraduate Program (HBCU-UP) Awardees
Attachment H
This request for Office of Management and Budget (OMB) review asks for a renewal of clearance of the program monitoring data collection for the National Science Foundation (NSF) Division of Human Resource Development (HRD) Self-Evaluation Indicator System (SEIS) for Historically Black Colleges and Universities-Undergraduate Program (HBCU-UP) under the Directorate for Education and Human Resources (EHR) Generic Clearance OMB 3145-0136, which expires on January 31, 2008.
SEIS data collected includes undergraduate student enrollment, retention, and graduation, faculty demographics, and other HBCU-UP project implementation activities such as course development.
The Historically Black Colleges and Universities Undergraduate Program (HBCU-UP) "seeks to enhance the quality of undergraduate science, technology, engineering, and mathematics (STEM) education at Historically Black Colleges and Universities as a means to broaden participation in the Nation's STEM workforce. The program provides support for the implementation of comprehensive institutional strategies to strengthen STEM teaching and learning in ways that improve access to and retention of underrepresented groups in STEM disciplines. Typical project implementation strategies include STEM course and curricular reform and enhancement, faculty professional development, supervised research and other active learning experiences for STEM undergraduates, student support, scientific instrumentation to improve STEM instruction, and other activities that meet institutional needs." (Ref.-http://www.ehr.nsf.gov/hrd/hbcu.asp).
Tthe National Science Foundation Act of 1950, as amended, and related legislation, 42 U.S.C. 1861 et seq., and additional authority by the Science and Engineering Equal Opportunities Act (42 U.S.C. 1885), and Title I of the Education for Economic Security Act (20 U.S.C. 3911 to 3922) authorize NSF to administere programs such as HBCU-UP in order to support activities designed to increase the participation of women and minorities and others under-represented in science and technology
NSF GPRA Strategic Plan, FY 2001 - 2006, Appendix 7 states "HBCU-UP provides funds to improve the quality of undergraduate science, mathematics, engineering and technology programs through curricular reform and enhancement, faculty development, research experiences for undergraduates, upgrading of scientific instrumentation, and improvement of research infrastructure. A program goal is to increase the number of baccalaureate recipients." (Ref.- http://www.nsf.gov/pubs/2001/nsf0104/app7.htm )
The NSF HBCU-UP Program Solicitation (NSF-02-162) outlined the monitoring component of the projects: "An evaluation and assessment plan is required within the Project Description so that project development and implementation can be monitored at all stages. One of the key objectives of the HBCU-UP is to improve the quality of undergraduate STEM education through the development, adaptation and implementation of effective educational techniques and practices to enhance STEM instruction. Accordingly, proposed evaluation and assessment plans should include indicators of progress that address the extent to which: (1) educational techniques and practices shown to be effective elsewhere are adapted or modified for use at the awardee institution; (2) a plan has been developed to assess the effectiveness of the educational techniques or practices implemented; (3) faculty at the awardee institution have been prepared to use the modified educational techniques or practices; (4) modified techniques or practices have been incorporated into the curriculum; (5) innovative courses or program components are developed; (6) the effectiveness of implemented educational techniques, practices, courses or components is assessed; and (7) project activities affect student learning and student access to quality STEM education as defined by measurable quantitative student-based outcomes such as:
· number of STEM majors involved in active learning activities or research activities;
· number of STEM majors who have enrolled in and successfully completed newly developed or revised courses or programs;
· rates of successful completion of STEM gate-keeper courses;
· student retention in STEM disciplines;
· number of STEM graduates with Grade Point Averages of 3.0 or higher;
· number of STEM students matriculating into 4-year colleges; and
· number of graduates that enter the STEM workforce."
(Ref-http://www.nsf.gov/pubs/2002/nsf02162/nsf02162.htm#TOC)
NSF asks HBCU-UP awardee institutions to complete and submit SEIS as an annual reporting requirement. (Ref.- http://www.ehr.nsf.gov/EHR/HRD/Reporting.htm)
This program began OMB-approved monitoring in 2003. Prior to that no agency or individual was collecting data on the extent to which programmatic or project outcomes were being achieved. There remain no other data collectors or data providers available except from the current data collection activity that is the subject of this request.
This is a reinstatement of a data collection task. The primary purpose is program monitoring and project self-assessment. The monitoring information will be used by the EHR directorate for analytical and policy support of the HBCU-UP program. In addition, the institutions will use the data for formative evaluation of their individual projects.
This is not an evaluation study that uses statistical comparisons across sites. The primary purpose is to collect program and individual project monitoring data from the HBCU-UP grantees to insure that each site is meeting its individual project goals. NSF may use the data to determine whether planned funding levels are appropriate or need to be revised. NSF has no plans to compare individual projects against each other. There is a separate program evaluation that is using the SEIS data as baseline descriptive data for the award portfolio among other sources.
SEIS is a Microsoft Excel workbook. It is designed to require only basic data entry skills; no Excel experience is required. Data collection tables are arranged on individual worksheets. Embedded in the worksheets are formulas which automatically sum totals. The user navigates through the workbook by hyperlinks embedded in the table of contents or tabs located at the bottom of the individual worksheets. The workbook is distributed on CD-ROM or email attachment. The completed SEIS is submitted via e-submission using Systemic Research's SEIS Management System (http://www.systemic.com/hbcu), or email attachment. The submitted SEIS is available to view or download from the SEIS Management System. The SEIS Management System is password protected- institutions can only access their own data; NSF program officers have access to all of the institutions' data.
SEIS does not duplicate other NSF efforts. SEIS data is part of the Annual Report to NSF. Institutions do not duplicate SEIS data in the report submitted through NSF's administrative database - FastLane Projects Reports system (OMB Control Number 3145-0058).
No information is to be collected from small businesses.
If the information is not collected, NSF will be unable to document the effectiveness, impacts, or outcomes of the HBCU-UP program. It will not be able to meet its accountability requirements or assess the degree to which projects are meeting their goals. NSF will be unable to comply fully with the congressional mandate that the Foundation evaluate its science, technology, engineering, and mathematics education programs, especially those that target underserved populations.
The data collections will continue to comply with 5 CFR 1320.6.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register June 7, 2004, Volume 69, Number 109, page 31846. No comments were received.
No payments or gifts will be provided to respondents.
Data collected are available to NSF officials and staff, and Systemic Research staff. Data are processed according to Federal and State privacy statues. Detailed procedures for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). That system limits access to personally identifiable information to authorized users, Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c. The information requested may be disclosed to qualified researchers and contractors in order to coordinate programs and to a Federal agency, court or party in a court, of Federal administrative proceeding, if the government is a party.
Individual institution data will be released to the public only with written permission from the institutions. A signed Data Release form is required (a copy is included in this clearance). The release specifies to whom the data may be made available to.
No questions of a sensitive nature are asked of respondents.
SEIS data typically are collected using an institution's existing management information system form the institution's Office of Institutional Research.
This survey is an annual census of all active HBCU awards. Institutions funded under the HBCU-UP program can change from year to year depending on several factors, 1) when an existing awards has expired or will expire (no longer receiving NSF-funding) and 2) which awards are newly funded. A third (3rd) factor is that the only institutions eligible to be funded or to apply to this program are HBCU institutions. Authorship (and the right to revise the list is outside of NSF's control. The eligible applicant institutions list is maintained by the Department of Education at:http://www.ed.gov/about/offices/list/ocr/edlite-minorityinst-as-vi.html.
The number of of active HBCU awards in 2007 is 56.
There are two types of respondents from these 56 awards: 1) Data Managers and 2) Principals Investigators (PI) of HBCU-UP awards. The average annual burden is calculated as follows:
|
Respondent Type |
Number of Respondents |
Burden Hours per Respondent |
Annual Person Hour Total |
|
Data Managers |
56 |
30 |
1,680 |
|
PIs |
56 |
6 |
336 |
|
Total |
112 |
36 |
2,016 |
The estimated total annual response burden is 2,016 person hours. This burden was calculated based on 56 responding institutions. On average it takes a Data Manager 30 hours annually to complete the instrument. It takes a PI on average 6 hours. The annual burden by SEIS form was calculated as follows:
|
Instrument |
Respondents Average Number per Year |
Annual Response Burden |
|
|
SEIS--Data Manager |
Data Managers |
56 |
56 x 30 hours = 1,680 hours |
|
SEIS--PI |
PIs |
56 |
56 x 6 hours = 336 hours |
|
Total |
112 |
2,016 hours . . |
|
The overall annualized cost to the respondents is estimated to be $64,848 and refers only to the time spent responding to the survey. The following estimate hourly wages (Data Managers- $27/hour and PIs- $58/hour) are based on a $120,640 12 month salary for PIs and a $56,160 12 month salary for the data manager. These salaries are based on budget requests for existing HBCU-UP projects. The total cost to respondent is calculated as follows:
|
Respondent Type |
Number, Rate, and burden |
Costs |
|
Data Managers |
56 x $27/hour x 30 hours |
$45,360 |
|
PIs |
56 x $58/hour x 6 hours |
$19,488 |
|
Total |
|
$64,848 |
There is no annual cost burden to respondents or other than time spent responding to SEIS. The information being collected is maintained in the institutions management information system, Office of Institutional Research, or project records. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide that is cleared under OMB 3145-0058. The Grant Proposal Guide requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus Data Managers and PIs who are the respondents to the individual data collection tasks for SEIS make use of standard office equipment (e.g. computer), Internet connectivity that is already required as a start-up cost and maintenance cost under OMB 3145-0058, and free software (e.g. Netscape© or Microsoft Explorer©) to respond.
The annualized cost to the government is approximately $380,747 per year.
The estimated costs include:
Personnel $197,124
Travel $27,000
Participant Support Costs $ 15,640
Other Direct Costs
Materials and Supplies $4,000
Publication Costs $ 27,000
Consultants $ 2,666
Computer Services $ 2,400
Other $ 600
Indirect Costs $104,316
Total Costs $380,747
There are travel costs for the Systemic Research staff (based in Massachusetts) to travel to attend meetings or conferences with the HBCU-UP principal investigators and the National Science Foundation (NSF) to demonstrate or present the survey and its findings.
Changes in burden from the 2003 request include an increase in average number of HBCU-UP awardee institutions and an average 3% increase in salaries for PIs and data managers. Burden is unlikely to increase or decrease significantly in the next three years assuming NSF's award funding for new and exisiting awards is consistent with past practice.
Each year the SEIS data collection will be announced in the spring and will be due in October of the same year. The data reported will cover the previous academic year - fall through the summer. The annual Academic Indicator Report (AIR) with aggregate program data will be published in the spring each year by the grantee organization conducting this project monitoring data collection, Systemic Research, Inc.
Interim individual institution AIRs are available to NSF program officers for program monitoring and institutions shortly after SEIS data is submitted. These draft reports are not published or made available to other than the aforementioned.
Because this project monitoring is being conducted under a grant and not a contract mechanism, NSF is not the publisher of this data. Systemic Research, Inc. is the publisher of the data. Systemic Research, Inc. disseminates AIR to the general public via its website http://www.systemic.com as well as interested audiences such as Minority Serving Institutions and national organizations working with Minority Serving Institutions such as the National Association for Equal Opportunity (NAFEO) via hard copy and CD-Rom. Individual institution's AIRs will be published for those who have indicated "Full Release" on the Data Release form.
Not Applicable
No exceptions apply
This collection is an annual descriptive census of all active HBCU-UP awards in the NSF portfolio.
SEIS data collection will involve on average 56 HBCU-UP awards annually. As this represents the entire universe, no statistical sampling will be employed.
Estimated Size of Universe
|
Population |
Universe Size |
Sample Size |
|
|
SEIS |
|||
|
|
Data Managers |
56 |
56 |
|
|
PIs |
56 |
56 |
|
Total |
112 |
112 |
|
This data collection uses an Excel© based instrument (SEIS). Each respondent will provide data annually. The limitation of the data collection system is that it is self-reported.
This is a census survey collecting descriptive data from HBCU-UP awards. The sample size is the universe active, awarded projects.
Not Applicable
Not Applicable
The SEIS form requests data from the institution that was reported in the previous annual response - the data for past years does not need to be reinput each year - it is automatically included. This allows the awardee to correct or complete any previous year's data if it was not available when the report was submitted in the prior year. SEIS collects data for the institution's most recent previous full academic year (Fall semester to Summer semester - not calendar year).
The ability to provide two years of baseline data in the SEIS format for the period before the start of the HBCU-UP project is the result of requests by the institutions during the development of SEIS. The baseline data provides important context for each institution's HBCU-UP project outcomes and helps to determine individual institutional trends due to the introduction of HBCU-UP project activities.
Not Applicable
NSF asks all HBCU-UP awardees to complete SEIS as a requirement for future funding. SEIS may be submitted using e-submission through the web, or email attachment.
The first step in the development and design of SEIS was the conceptual framework. The framework was an outline of the quantitative and qualitative data that would provide all of the information necessary for fulfilling NSF reporting requirements and self-assessment. As the framework was built the feasibility of obtaining the requested data was considered.
A SEIS prototype was developed based on the conceptual framework. It was introduced at a Advisory Panel Meeting on January 22, 2002 in Atlanta, GA. SEIS was thoroughly reviewed and discussed by the participants. The suggestions and feedback received were incorporated into SEIS Prototype Version 2.
A field test was conducted. SEIS Prototype Version 2 was disseminated to six institutions that volunteered to participate. Each institution entered their data into SEIS. Representatives from the institutions reported they found SEIS to be user friendly and comprehensive. A few technical issues were discovered and resolved. Based on the field test results, SEIS was finalized and the first annual SEIS was disseminated at the HBCU-UP workshop in Atlanta on May 16, 2002.
Agency Unit - Mary F. Sladek (703) 292-5152; Dr. Victor Santiago (703) 292-4673 and Dr. Jessie DeAro (703) 292-5350, National Science Foundation
Grantee - Jason J. Kim and Linda M. Crasco (OMB package preparer), Systemic Research, Inc. (781) 278-0300
Advisory Panel -
Josephine Davis, Fort Valley State University- (478) 825-6244
Johnnye Jones, Jarvis Christian College- (903) 769-5724
Caesar Jackson, North Carolina A & T State University- (903) 769-5724
Shirley McBay, Quality Education for Minorities Network- (202) 659-1818
Nellouise Watkins, Bennet College (retired)- (336) 697-1226
Systemic Research, Inc. will be responsible for data collection and analysis under the direction of Jason J. Kim and Linda M. Crasco (781) 278-0300.
| File Type | text/html |
| File Title | Supporting Statement |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |