REQUEST FOR CLEARANCE
National Science Foundation
Directorate of Education and Human Resources
Division of Human Resource Development
Louis Stokes Alliances for Minority Participation
Monitoring System (LSAMP)
Attachment D
This request for Office of Management and Budget (OMB) review is part of the renewal process for the National Science Foundation (NSF) Directorate for Education and Human Resources (EHR) Generic Clearance (OMB 3145-0136), which expires on March 31, 2011. The EHR Generic Clearance includes collections of information about NSF’s education and training (E&T) activities. This particular request addresses management or monitoring for the Louis Stokes Alliances for Minority Participation (LSAMP) program within EHR’s Division of Human Resource Development (HRD). This task is the oldest task in the EHR Generic. LSAMP was one reason OMB requested that NSF apply to establish an EHR Generic Clearance for program and project monitoring in 1995.
In the early 1990s the LSAMP program began as a multidisciplinary comprehensive undergraduate program designed to increase the quantity and quality of minority students receiving baccalaureate degrees in science, technology, engineering, and mathematics (STEM). In the early 2000s, LSAMP’s programmatic design expanded to include financial support for activities that encourage enrollment in full-time graduate study. For example, the LSAMP Bridge to the Doctorate activity allows eligible, currently funded LSAMP projects to apply to NSF to receive additional funds to support graduate students, particularly those traditionally underrepresented in STEM fields, to pursue and attain a doctorate in a STEM field supported by NSF. The Bridge to the Doctorate activity is described in further detail in the LSAMP Bridge to the Doctorate Supporting Statement (attachment E).
The LSAMP program requires funded projects to propose and maintain the formation of multi-institutional alliances. The LSAMP program funds projects that address processes and factors that promote baccalaureate and graduate degree attainment, preparation for graduate study, and preparation for successful STEM careers outside of the higher education enterprise. You can see the latest LSAMP solicitation here.
Data collected from LSAMP alliances through the monitoring system are needed by NSF for project and program monitoring, to fulfill policy and program reporting needs, and to serve as preliminary work for future impact assessment and evaluation activities. The data collected as part of OMB 3145-0136 allow NSF officials to document the overall program investment in individual alliances, and make future funding and program policy decisions.
System screenshots can be found in appendix A, and a crosswalk of data elements can be found in appendix B.
The information collected in this task is required for effective administration, communication, and program and project monitoring; for meeting reporting requirements; for measuring attainment of NSF’s program, project and strategic goals as laid out in NSF’s Strategic Plan; and as a baseline for future program evaluations.
The primary purpose of this collection is to provide data and information for effective program management and monitoring of program activities. This data collection activity is designed to track the extent to which LSAMP awards meet the objectives of the program. Within the HRD division, this information is used to administer and monitor the progress of the program. The findings are used to recommend, among other things, administrative changes in program functions, level of award support, individual program focus and emphasis, and recruiting efforts.
In recent guidance from the Director of OMB, M-10-32, the need for rigorous evaluations and the objectives of evaluations of programs were clearly outlined, including the use of evaluation resources. Because the collection of data contained in this monitoring effort contributes to the formal evaluation of the program and provides regular measures of program performance by accumulating operating information from each project in the program, this guidance is particularly pertinent to this request.
“Improving and coordinating the use of existing evaluation resources. In addition to the voluntary evaluation initiative, agencies should continue to carefully assess, report on, and allocate the base funds and resources that the agencies have for conducting evaluation. Agencies are encouraged to share information beyond what is requested in guidance and consult with OMB’s Resource Management Offices (RMOs) to coordinate and improve the design, implementation, and utilization of evaluations.”
These directives establish an ongoing need for NSF to engage in an interactive process of collecting information and using it to improve program services and processes.
The LSAMP program also uses the data to fulfill reporting requirements. As a part of its performance assessment activities, NSF relies on the judgment of external experts to maintain high standards of program management. Directorate and Office advisory committees (ACs) meet twice a year, while Committees of Visitors (COVs) for divisions or programs meet once every three years. Data collected in the LSAMP monitoring system may be used to report to these committees on program activities. In addition, NSF is required to measure the attainment of its program, project, and strategic goals by the President’s Accountable Government Initiative, the Government Performance and Results Act (GPRA) Modernization Act of 2010, and the NSF’s Strategic Plan. Data collected in the LSAMP monitoring system help NSF management examine their progress towards the Foundation’s goals and respond to these reporting requirements.
Finally, the data can also be used as a preliminary step in more detailed future evaluation efforts. EHR makes these data available to NSF staff, EHR contractors with responsibility for the collection, and HRD program managers and their staff and contractors.
Under the LSAMP monitoring system, each LSAMP alliance and institution provides annual data using the Web-based data collection system (see appendix A). The following is an overview of the types of information collected:
There are no changes to the LSAMP data collection system. See appendix B for the detailed list of data elements.
EHR typically uses Web-based systems because they can facilitate respondents’ data entry across hardware and software platforms. An innovative feature of many of the individual Web systems designed by ICF Macro for NSF is the thorough editing of all submitted data for completeness, validity, and consistency. Editing is performed as data are entered. Most invalid data cannot enter the system, and questionable or incomplete entries are called to respondents’ attention before they are submitted to NSF.
LSAMP’s system employs user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus, and predefined charts and graphics. All these features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden.
The WebAMP system was first used in the spring of 1998 in response to user requests for improvement over a previously used disk-based instrument and to minimize burden. The 508-compliant Web-based software facilitates respondents’ data entry by ensuring more complete and correct data submissions and thus reducing the need for follow up after a response is submitted to NSF. Fields are also marked with out-of-range indicators, and respondents are warned to check their data if they appear to be out-of-range.
Under WebAMP, respondents see data submitted in previous (if any) collection cycles. Most projects (Alliances) have a multi-year lifecycle (often five years or longer), so this feature makes correcting or completing a previous year’s data, particularly those on student enrollments, far easier and less burdensome than re-entering the data. Additionally, because the collection is Web-based minor bugs or formatting of items can (and have) been easily corrected in response to user feedback.
This system does not duplicate other NSF efforts. Comparable data are not currently being collected on an annual basis for the LSAMP program. In addition, the collection is coordinated with the NSF FastLane Project Reports system (OMB 3145-0058) to ensure that the two collections do not collect similar data. As much as possible, data from other NSF monitoring collections are used to pre-fill LSAMP items, further minimizing overall response burden. Additionally, aggregate data are being shared with NSF-funded researchers as appropriate, thereby minimizing the possibility that other researchers will duplicate these efforts in their own future collections.
No information is to be collected from small businesses.
Without this information, NSF would be restricted in managing and reporting on the activities of awards in the LSAMP program. Without this feedback, NSF would have no way of making systematic modifications to the LSAMP program (e.g., adequacy of funding amount, duration of award, and institutional supports needed). These data will ensure that NSF makes informed decisions about future directions of the LSAMP program. The information requested here is not available elsewhere.
Additionally, without this information NSF would find it difficult to meet agency data requests, as well as GPRA and OMB reporting requirements, and would be unable to comply fully with congressional and presidential mandates that the Foundation asses its STEM education programs.
The data collection will continue to comply with 5 CFR 1320.6.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register November 9, 2010, Volume 75, Number 216, pages 68829-68830. No comments were received.
During the initial system development principal investigators (PIs) from LSAMP awards reviewed the system; their responses to the instrument and their assessments of the institution instrument were taken into account in the development of the system. Changes in the system since initial development are informed by ongoing consultations with the respondents, ICF Macro (the contractor that designed the Web interface and database system), and Abt Associates, Inc. (the contractor that produces reports and presentations of aggregate data). ICF Macro currently maintains the system and database and provides technical support to respondents as needed.
No payments or gifts will be provided to respondents.
Data collected under this task are only available to the respondents, NSF, and the firms hired to manage the data and data collection software. Data are processed according to Federal and State privacy statutes. To protect privacy, only composite data or graphical representations will be released to the public.
For the collection covered by this clearance request, when respondents are presented with the first screen of the Web-based instrument, they are additionally instructed as follows: “Information from this data collection system will be retained by the NSF, a Federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). All individually identifiable information supplied by individuals or institutions to a Federal agency may be used only for the purposes outlined in the system of records notice and may not be disclosed or used in identifiable form for any other purpose, unless otherwise compelled by law. These are confidential files accessible only to appropriate NSF officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form, or data explicitly requested as “for general use,” will be made available to anyone outside of the NSF for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c. The Social Security number (SSN) will be maintained in accordance with the requirements of the Privacy Act of 1974. Submission of the SSN is voluntary. It is used for survey quality control, program evaluation, and for matching with other datasets maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998).”
LSAMP requests information on gender, race/ethnicity, disability (if any), academic discipline, and class is order to monitor the sites’ participant populations. GPA, graduation status, enrollment status, mentoring relationships, financial support indicators, and activity participation are needed to assess the impact of NSF’s grant investment.
Names and SSNs are collected to permit tracking of the program participants across time and place (e.g., from 2-year to 4-year institutions to Ph.D.-granting institutions) within a particular Alliance or across Alliances. Respondents have the option of not providing information that they consider privileged by marking the “not reported” option or by leaving the SSN field blank. In addition, individual participant activity status is requested, not required. Respondents are advised that identifiable data are provided only to LSAMP program staff and NSF contractors conducting studies in compliance with the Privacy Act.
The total number of annual respondents is 529 (80 project PIs/co-PIs and 449 LSAMP institution personnel). The total annual person-hours is 13,754. The Web-based collection is an annual activity of the LSAMP program. There are approximately 40 LSAMP alliances with 2 or more co-PIs and project personnel at alliance institutions. New alliances (and institutions within currently funded alliances) will be added to the program over the next three years. The new institutions enter at approximately the same rate that alliances or institutions leave the program or project as their funding expires.
The annualized burden for the component instruments in the current task (PI and institution personnel) was calculated by taking the average number of respondents from the previous collection cycles and estimating their response burden, based on a question in the Web-based data collection system asking how long it takes respondents to complete the instrument. The three burden estimates for each type of respondent are outlined below:
|
Respondent Type |
Estimated Average Annual Number of Respondents |
Estimated Average Annual Burden Hours Per Respondent |
Estimated Annual Burden Hour Total |
|
PIs/co-PIs |
80 |
26 |
2,080 |
|
LSAMP institution personnel |
449 |
26 |
11,674 |
|
Total |
529 |
13,754 |
As mentioned above respondents will be project PIs, co-PIs, and other project personnel of the LSAMP program. The total annual response burden is 13,754 hours. The annual burden by form was calculated as follows:
|
Form Type |
Respondent Type |
Number of Respondents |
Burden Hours Per Respondent |
Total Burden Hours |
|
LSAMP data collection form |
PIs, co-PIs, LSAMP institution personnel |
529 |
26 |
13,754 |
|
Total |
|
529 |
26 |
13,754 |
The overall cost to the respondents is estimated to be $223,288. The following table shows the annualized estimates of costs to PI respondents, who are generally university professors. These estimated hourly rates are based on a report in the April 16, 2010, edition of The Chronicle of Higher Education (2010). (“What Professors Earn.” The Chronicle of Higher Education, 56(31), A10, Washington, D.C.: The Chronicle of Higher Education, Inc.). According to the report, the average salary of an associate professor across all types of doctoral-granting institutions (public, private, church-related) was $83,511. When divided by the number of standard annual work hours (2,080), this calculates to approximately $40 per hour.
|
Respondents |
Number of Respondents |
Hours per Respondent |
Average Hourly Rate |
Total Annual Costs |
|
Project PIs |
80 |
26 |
$40 |
$83,200 |
|
Institution Personnel |
449 |
26 |
$12 |
$140,088 |
|
Total |
529 |
$223,288 |
There is no overall annual cost burden to respondents or record-keepers that results from the distance monitoring of the LSAMP program other than the time spent responding to the data collection instrument attached as appendix A to this request.
It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested is from records that are maintained as part of normal educational or training practice. Furthermore, the majority of respondents are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, LSAMP PIs and program personnel make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance costs under OMB 3145-0058, and free software (e.g., Microsoft Explorer or Mozilla Firefox) to respond. Thus, there are no capital and startup costs or operation and maintenance costs to respondents or record-keepers.
Computing the annualized cost to NSF for the LSAMP data collection was done by taking the budgets for three years and calculating the costs for each of the following operational activities involved in producing, maintaining, and conducting the LSAMP data collection:
|
Operational Activities |
Cost Over Three Years |
|
System Development (includes initial development of the database and Web-based application, and later changes requested by the program-e.g., increased reporting tools, additional validations) |
$321,382 |
|
System Maintenance, Updates, and Tech Support (system requires updates each year before opening the collection; maintenance is required to keep the system current with technology, e.g., database servers, operating systems) |
$156,105 |
|
Data Collection Opening and Support (e.g., online and telephone support to respondents and contacting respondents to encourage completion of the questions), Reporting (as defined by HRD), and Followup activities (e.g., providing data to other consultants) |
$202,076 |
|
Three-Year Total for All Operational Activities |
$679,563 |
The annualized cost was computed as one-third of the total three-year costs; thus, the annualized cost to NSF for the LSAMP data collection is $226,521.
In this request for renewal, the number of respondents has increased greatly from the 2008 clearance, but the total hour burden has increased only slightly because the average burden hours per person decreased. In the 2008 clearance request, the burden was 13,680 hours for 380 respondents. This renewal request is for 13,754 hours for 529 respondents. The increase of 149 respondents resulted in an increase of 74 burden hours overall. There are no changes to the LSAMP data collection form that would affect burden, however, respondent familiarity with the instrument and the Web-based data collection system has cut down on burden hours over time. For more details on the instruments, see the crosswalk in appendix B.
Data collection begins in June each year and ends in October. NSF program officers extend the October deadline upon request of the respondents. Once the data collection has been completed, agency staff can access the data through the on-line system as needed.
Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. ICF Macro, the contractor that manages the data collection Web site and database, is forbidden contractually from publishing results unless NSF instructs them to. In short, all products of the collections are the property of NSF and NSF is the exclusive publisher of the information being gathered.
The data from this collection primarily are used for internal review purposes and to monitor the LSAMP alliances, as well as for baseline data in NSF-contracted third-party program evaluations and descriptive analysis studies used in reporting to Congress (e.g., the GPRA Annual Performance Plan). Reports to NSF management, PIs, OMB, and Congress deal with characteristics and performance of the LSAMP program and may include statistical tables and charts generated from the LSAMP database.
Data from the LSAMP data collection may be used for NSF reports addressing the goal of increasing minority participation in STEM education and research. For example in the year 2000 NSF, the National Aeronautics and Space Administration (NASA), and the National Institutes of Health (NIH) participated in a joint project (Study of Services for Underrepresented Students) that described the activities supported by these programs that share a joint goal of increasing the participation of traditionally underrepresented minorities in undergraduate study in STEM fields. The final report highlighted methods that promote the achievement of traditionally underserved students in STEM fields. That report was turned into an NSF publication, A Description and Analysis of Best Practice Findings of Programs Promoting Participation of Underrepresented Undergraduate Students in Science, Math, Engineering, and Technology Fields, December 2000, Westat (NSF 01-31). NSF 01-31 makes passing references to the LSAMP data.
During the 2001-2004 clearance period, in accordance with OMB approval, NSF provided the historic LSAMP database (Please note that the LSAMP data collection instrument is sometimes called MARS, which refers to the pre-Web method for delivering the instrument) to NSF’s contractor, the Urban Institute. The Urban Institute’s evaluative study of the LSAMP program was cleared through OMB under OMB 3145-0190, and the Final Report on the Evaluation of the NSF LSAMP program was released in November 2005; more information on the report is available here.
Not applicable
No exceptions apply.
The sample size is the entire universe of LSAMP projects that consist of an annual average of 40 multi-year grants and cooperative agreements made by NSF to an eligible institution of higher education (IHE). That lead awardee has many partner IHEs as sub or collaborative awardees. The individual respondents come from both the individual project’s lead institution and other partnering IHEs. The annual average of individual respondents is 529. As above mentioned in section A, the individual respondent types include a project’s PIs/co-PIs, other project personnel, and data coordinators. This annual number of 529 is expected to remain stable throughout the clearance period.
|
Population |
Estimated Universe Size |
Sample Size |
|
LSAMP Project Participants |
529 |
529 |
This data collection uses a Web-based instrument. Participating individuals from each LSAMP project provide descriptive data each year for the duration of their NSF funding. The data are primarily useful for program management, monitoring, and descriptive analysis.
NSF understands the limitations of the data collection, particularly in terms of using the data to determine program effectiveness. Data collected through the LSAMP system are not used to determine the ultimate effectiveness of its STEM educational interventions, but are used in program planning and management, to report on agency activities and goals, and to lay the groundwork for future evaluations.
This data collection is a census, so no sampling is required.
Not applicable
Not applicable
Not applicable
Not applicable
Past collections have had 100 percent response rates, and NSF anticipates that the rate will remain the same. The collection is part of the reporting required of LSAMP programs to maintain their NSF funding. Additionally, considerable effort is made to follow up with alliances and institutions that have not provided complete reports. E-mail reminders are sent at regular intervals during the collection cycle, and phone calls are made to alliance personnel as the end of the collection cycle approaches. Examples of the e-mail messages announcing the opening of the system and reminding respondents to log in and enter data are included in appendix C.
This system has been operational since 1998. Most alliance PIs tested the system while it was in development and provided valuable feedback. Additionally, respondents continually provide feedback on system improvements. Most of the items and response categories utilized in this system follow formats that are already in place in other NSF monitoring systems.
Agency
A. James Hicks, National Science Foundation, (703) 292-4668
Contractors
ICF Macro will be responsible for data collection and analysis under the direction of Lea Mesner, (301) 657-3070.
| File Type | text/html |
| File Title | Supporting Statement |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |