REQUEST FOR CLEARANCE
National Science Foundation
Directorate of Education and Human Resources
Division of Human Resource Development
Louis Stokes Alliances for Minority Participation Bridge to the Doctorate
Monitoring System (LSAMP BD)
Attachment E
This request for Office of Management and Budget (OMB) review is part of the renewal process for the National Science Foundation (NSF) Directorate for Education and Human Resources (EHR) Generic Clearance (OMB 3145-0136), which expires on March 31, 2011. The EHR Generic Clearance includes collections of information about NSF’s education and training (E&T) activities. This particular request addresses management or monitoring for the Louis Stokes Alliances for Minority Participation Bridge to the Doctorate (LSAMP BD) initiative within EHR’s Division of Human Resource Development (HRD).
In the early 1990s the LSAMP program began as a multidisciplinary comprehensive undergraduate program designed to increase the quantity and quality of minority students receiving baccalaureate degrees in science, technology, engineering, and mathematics (STEM). In the early 2000s, LSAMP’s programmatic design expanded to include financial support for activities that encourage enrollment in full-time graduate study. For example, the LSAMP BD initiative allows eligible, currently funded LSAMP projects to apply to NSF to receive additional funds to support graduate students, particularly those traditionally underrepresented in STEM fields, to pursue and attain a doctorate in a STEM field supported by NSF.
The LSAMP program requires funded projects to propose and maintain the formation of multi-institutional alliances. The LSAMP program funds projects that address processes and factors that promote baccalaureate and graduate degree attainment, preparation for graduate study, and preparation for successful STEM careers outside of the higher education enterprise. You can see the latest LSAMP solicitation here.
Data collected from LSAMP BD data respondents through the monitoring system are needed by NSF for project monitoring, to fulfill policy and program reporting needs, and to serve as preliminary work for future impact assessment and evaluation activities. The data collected as part of OMB 3145-0136 allow NSF officials to document the overall program investment in individual alliances, and make future funding and program policy decisions.
System screenshots can be found in appendix A, and a crosswalk of data elements can be found in appendix B.
The information collected in this task is required for effective administration, communication, and program and project monitoring; for meeting reporting requirements; for measuring attainment of NSF’s program, project, and strategic goals as laid out in NSF’s Strategic Plan; and as a baseline for future program evaluations.
The primary purpose of this collection is to provide data and information for effective program management and monitoring of program activities. This data collection activity is designed to track the extent to which LSAMP BD awardees meet the objectives of the initiative. Within the HRD division, this information is used to administer and monitor the progress of the LSAMP BD initiative. The findings are used to recommend, among other things, administrative changes, level of award support, and recruiting efforts.
In recent guidance from the Director of OMB, M-10-32, the need for rigorous evaluations and the objectives of evaluations of programs were clearly outlined, including the use of evaluation resources. Because the collection of data contained in this monitoring effort contributes to the formal evaluation of the program and provides regular measures of program performance by accumulating operating information from each project in the program, this guidance is particularly pertinent to this request.
“Improving and coordinating the use of existing evaluation resources. In addition to the voluntary evaluation initiative, agencies should continue to carefully assess, report on, and allocate the base funds and resources that the agencies have for conducting evaluation. Agencies are encouraged to share information beyond what is requested in guidance and consult with OMB’s Resource Management Offices (RMOs) to coordinate and improve the design, implementation, and utilization of evaluations.”
These directives establish an ongoing need for NSF to engage in an interactive process of collecting information and using it to improve program services and processes.
The LSAMP BD initiative also uses the data to fulfill reporting requirements. As a part of its performance assessment activities, NSF relies on the judgment of external experts to maintain high standards of program management. Directorate and Office advisory committees (ACs) meet twice a year, while Committees of Visitors (COVs) for divisions or programs meet once every three years. Data collected in the LSAMP BD monitoring system may be used to report to these committees on program activities. In addition, NSF is required to measure the attainment of its program, project, and strategic goals by the President’s Accountable Government Initiative, the Government Performance and Results Act (GPRA) Modernization Act of 2010, and the NSF’s Strategic Plan. Data collected in the LSAMP BD monitoring system help NSF management examine their progress towards the Foundation’s goals and respond to these reporting requirements.
Finally, the data can also be used as a preliminary step in more detailed future evaluation efforts. EHR makes these data available to NSF staff, EHR contractors with responsibility for the collection, and HRD program managers and their staff and contractors.
Under the LSAMP BD monitoring system, each institution participating in the LSAMP Bridge to the Doctorate initiative provides annual data using the Web-based data collection system (see appendix A) for BD students and comparable students (Peer Comparison students). The following is an overview of the types of information collected:
See appendix B for a detailed list of data elements.
EHR typically uses Web-based systems because they can facilitate respondents’ data entry across hardware and software platforms. An innovative feature of many of the individual Web systems designed by ICF Macro for NSF is the thorough editing of all submitted data for completeness, validity, and consistency. Editing is performed as data are entered. Most invalid data cannot enter the system, and questionable or incomplete entries are called to respondents’ attention before they are submitted to NSF.
The LSAMP BD Web-based instrument employs user-friendly features such as data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, and standard menus. Undergraduate data on the Baseline Data screen is pre-populated from LSAMP. All of these features facilitate the reporting process, provide useful and rapid feedback to the data respondents, and reduce burden.
The LSAMP BD system was first used in the fall of 2007. The 508-compliant Web-based software facilitates respondents’ data entry by ensuring more complete and correct data submissions and thus reducing the need for follow up after a response is submitted to NSF. Fields are also marked with out-of-range indicators, and respondents are warned to check their data if they appear to be out-of-range.
Since LSAMP BD data are reported on an annual basis , respondents see data submitted in previous (if any) collection cycles. This feature enables data respondents to view past data responses and to request corrections if necessary. Additionally, because the collection is Web-based, any issues or formatting of items can (and have) been easily corrected in response to user feedback.
This system does not duplicate other NSF efforts. Comparable data are not currently being collected on an annual basis for the LSAMP BD initiative. In addition, the collection is coordinated with the NSF FastLane Project Reports system (OMB 3145-0058) to ensure that the two collections do not collect similar data. As much as possible, data from other NSF monitoring collections are used to pre-fill LSAMP BD items, further minimizing overall response burden. Additionally, aggregate data are being shared with NSF-funded researchers as appropriate, thereby minimizing the possibility that other researchers will duplicate these efforts in their own future collections.
No information is to be collected from small businesses.
Without this information, NSF would be restricted in managing and reporting on the activities of awards in the LSAMP BD initiative. Without this feedback, NSF would have no way of making systematic modifications to the LSAMP BD initiative. These data will ensure that NSF makes informed decisions about future directions of the LSAMP BD initiative. The information requested here is not available elsewhere.
Additionally, without this information NSF would find it difficult to meet agency data requests, as well as GPRA and OMB reporting requirements, and would be unable to comply fully with congressional and presidential mandates that the Foundation asses its STEM education programs.
The data collection will continue to comply with 5 CFR 1320.6.
The notice inviting comments on the EHR Generic Clearance (OMB 3145-0136) was published in the Federal Register November 9, 2010, Volume 75, Number 216, pages 68829-68830. No comments were received.
During the initial system development principal investigators (PIs) from LSAMP BD awards reviewed the system; their responses to the instrument and their assessments of the institution instrument were taken into account in the development of the system. Changes in the system since initial development are informed by ongoing consultations with the respondents, ICF Macro (the contractor that designed the Web interface and database system), and the Urban Institute (the contractor that produces reports and presentations of aggregate data). ICF Macro currently maintains the system and database and provides technical support to respondents as needed.
No payments or gifts will be provided to respondents.
Data collected under this task are only available to the respondents, NSF, and the firms hired to manage the data and data collection software. Data are processed according to Federal and State privacy statutes. To protect privacy, only composite data or graphical representations will be released to the public.
For the collection covered by this clearance request, when respondents are presented with the first screen of the Web-based instrument, they are additionally instructed as follows: “Information from this data collection system will be retained by the NSF, a Federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). All individually identifiable information supplied by individuals or institutions to a Federal agency may be used only for the purposes outlined in the system of records notice and may not be disclosed or used in identifiable form for any other purpose, unless otherwise compelled by law. These are confidential files accessible only to appropriate NSF officials, their staffs, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form, or data explicitly requested as “for general use,” will be made available to anyone outside of the NSF for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c. The Social Security number (SSN) will be maintained in accordance with the requirements of the Privacy Act of 1974. Submission of the SSN is voluntary. It is used for survey quality control, program evaluation, and matching with other datasets maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998).”
LSAMP BD requests information on gender, race/ethnicity, disability (if any), intended major, and degree in order to monitor the sites’ participant populations. GPA, graduation status, and financial support indicators are needed to assess the impact of NSF's grant investment.
Names and Social Security numbers are collected to permit tracking of the program participants across time and for followup activity. Respondents have the option of not providing information that they consider privileged by marking the “not reported” option or by leaving the SSN field blank. Respondents are advised that identifiable data are provided only to LSAMP BD program staff and NSF contractors conducting studies in compliance with the Privacy Act.
The total number of Bridge to the Doctorate data coordinators involved in the LSAMP BD module is 48 and the annual burden hour total is 3,456.
The Web-based collection is an annual activity of the LSAMP BD initiative. There are approximately 48 LSAMP BD participating institutions. New institutions participating in the BD initiative will be added to the program over the next three years. New institutions are selected to participate in the LSAMP BD initiative at approximately the same rate as institutions leave the initiative as their funding expires.
The annualized burden for the component instruments in the current task was calculated by taking the average number of data respondents for each participating BD institution from the previous collection cycles and the actual reported response burden for the BD and Peer Comparison student data, based on a question in the Web-based data collection system asking how long it takes data respondents to complete the instrument.
As mentioned above respondents for the LSAMP BD module will be data coordinators. The total annual response burden is 3,456 hours. The annual burden by form was calculated as follows:
|
Form Type |
Respondent Type |
Number of Respondents |
Burden Hours Per Respondent |
Total Burden Hours |
|
Bridge to the Doctorate Module |
Bridge to the Doctorate Data Coordinators |
48 |
72 |
3,456 |
|
Total |
48 |
72 |
3,456 |
The overall cost to the data respondents is estimated to be $62,208. The following table shows the annualized estimates of costs to data respondents:
|
Respondents |
Number of Respondents |
Hours per Respondent |
Average Hourly Rate |
Total Annual Costs |
|
|
Bridge to the Doctorate Data Coordinators |
48 |
72 |
$18 |
$62,208 |
|
|
Total |
48 |
72 |
$18 |
$62,208 |
|
There is no overall annual cost burden to respondents or record-keepers that results from the distance monitoring of the LSAMP BD initiative other than the time spent responding to the data collection instrument attached as appendix A to this request.
It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested is from records that are maintained as part of normal educational or training practice. Furthermore, the majority of respondents are active or former grantees or participants in programs or projects once funded by NSF. In order to be funded by NSF, institutions must follow the instructions in the NSF Grant Proposal Guide (GPG) that is cleared under OMB 3145-0058. The GPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane, an Internet-based forms system. Thus, LSAMP BD data respondents and personnel make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance costs under OMB 3145-0058, and free software (e.g., Microsoft Explorer or Mozilla Firefox) to respond. Thus, there are no capital and startup costs or operation and maintenance costs to respondents or record-keepers.
Computing the annualized cost to NSF for the LSAMP BD data collection was done by taking the budgets for three years and calculating the costs for each of the following operational activities involved in producing, maintaining, and conducting the LSAMP BD data collection:
|
Operational Activities |
Cost Over Three Years |
|
System Development (includes initial development of the database and Web-based application, and later changes requested by the program-e.g., increased reporting tools, additional validations) |
$202,737 |
|
System Maintenance, Updates, and Tech Support (system requires updates each year before opening the collection; maintenance is required to keep the system current with technology, e.g., database servers, operating systems) |
$98,476 |
|
Data Collection Opening and Support (e.g., online and telephone support to respondents and contacting respondents to encourage completion of the questions), Reporting (as defined by HRD), and Followup activities (e.g., providing data to other consultants) |
$127,475 |
|
Three-Year Total for All Operational Activities |
$428,688 |
The annualized cost was computed as one-third of the total three-year costs; thus, the annualized cost to NSF for the LSAMP BD data collection is $142,896.
In the previous request for renewal, the number of LSAMP BD data coordinators was 35. With additional institutions participating in the Bridge to the Doctorate data collection, the number of data coordinators is now 48. The average burden hours per respondent previously reported was based on the best estimate at the time as data collection activities had not yet begun. We are now using the average of actual burden hours per respondent reported by the Bridge to the Doctorate data coordinators. The increase in respondents and average burden hours per respondent result in an increase in total burden from 700 hours to 3,456 hours.
Data collection begins in late October each year and ends in December. NSF program officers extend the December deadline upon request of the respondents. Once the data collection has been completed, agency staff can access the data through the online system as needed.
Like many agencies, NSF is reducing its reliance on formal (i.e., traditional) publication methods and publication formats. ICF Macro, the contractor that manages the data collection Web site and database, is forbidden contractually from publishing results unless NSF instructs them to. In short, all products of the collections are the property of NSF and NSF is the exclusive publisher of the information being gathered.
The data from this collection primarily are used for internal review purposes and to monitor the LSAMP BD awards, as well as for baseline data in NSF-contracted third-party program evaluations and descriptive analysis studies used in reporting to Congress (e.g., the GPRA Annual Performance Plan). Reports to NSF management, PIs, OMB, and Congress deal with characteristics and performance of the LSAMP program (which includes the LSAMP BD initiative) and may include statistical tables and charts generated from the LSAMP BD database.
Data from the LSAMP BD collection may be used for NSF reports addressing the goal of increasing minority participation in STEM education and research. For example in the year 2000 NSF, the National Aeronautics and Space Administration (NASA), and the National Institutes of Health (NIH) participated in a joint project (Study of Services for Underrepresented Students) that described the activities supported by these programs that share a joint goal of increasing the participation of traditionally underrepresented minorities in undergraduate study in STEM fields. The final report highlighted methods that promote the achievement of traditionally underserved students in STEM fields. That report was turned into an NSF publication, A Description and Analysis of Best Practice Findings of Programs Promoting Participation of Underrepresented Undergraduate Students in Science, Math, Engineering, and Technology Fields, December 2000, Westat (NSF 01-31). NSF 01-31 makes passing references to the LSAMP BD data.
Not applicable
No exceptions apply.
The sample size is the entire universe of LSAMP BD participating institutions that consist of an annual average of 48 cooperative agreements made by NSF to an eligible institution of higher education. The annual average of individual respondents is 48. As mentioned above in section A, the individual that responds to the LSAMP BD questionnaire is a data coordinator. This annual number of 48 is expected to remain stable throughout the clearance period.
|
Population |
Estimated Universe Size |
Sample Size |
|
LSAMP BD Data Coordinators |
48 |
48 |
This data collection uses a Web-based instrument. The data are primarily useful for program management, monitoring and descriptive analysis.
NSF understands the limitations of the data collection, particularly in terms of using the data to determine program effectiveness. Data collected through the LSAMP BD system are not used to determine the ultimate effectiveness of its STEM educational interventions, but are used in program planning and management, to report on agency activities and goals, and to lay the groundwork for future evaluations.
This data collection is a census, so no sampling is required.
Not applicable
Not applicable
Not applicable
Not applicable
Past collections have around an 80 percent response rate, and NSF anticipates that the rate will increase in future collections. The collection is part of the reporting required of LSAMP programs to maintain their NSF funding. Additionally, considerable effort is made to follow up with institutions that have not provided complete reports. E-mail reminders are sent at regular intervals during the collection cycle and phone calls are made to award personnel as the end of the collection cycle approaches. Examples of the e-mail messages announcing the opening of the system and reminding respondents to log in and enter data are included in appendix C.
This system has been operational since 2007. Data respondents continually provide feedback on system improvements. Most of the items and response categories utilized in this system follow formats that are already in place in other NSF monitoring systems.
Agency
A. James Hicks, National Science Foundation, (703) 292-4668
Contractors
ICF Macro will be responsible for data collection and analysis under the direction of Lea Mesner, (301) 657-3070.
| File Type | text/html |
| File Title | Supporting Statement |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |