Supporting Statement (Part A; Control No. 3145-0226, 2022 renewal)
REQUEST FOR RENEWAL OF EHR
PROGRAM MONITORING DATA COLLECTIONS
Forms Clearance Package
Submitted by:
National Science Foundation
2415 Eisenhower Avenue
Alexandria, VA 22230
The National Science Foundation (NSF) is an independent federal agency that supports research at the frontiers of knowledge, across all fields of science and engineering (S&E) and S&E education. NSF’s mission is “to promote the progress of science to advance the national health, prosperity and welfare; to secure the national defense; and for other purposes.”1 NSF is “the funding source for approximately 27 percent of the total federal budget for basic research conducted at U.S. colleges and universities.”2 The Foundation awards “grants and cooperative agreements to more than 2,000 colleges, universities, K-12 school systems, businesses, informal science organizations, and other research organizations throughout the U.S.”3
Within NSF, the mission of the Directorate for Education and Human Resources (EHR) is “to achieve excellence in U.S. science, technology, engineering, and mathematics (STEM) education at all levels and in all settings (both formal and informal) in order to support the development of a diverse and well-prepared workforce of scientists, technicians, engineers, mathematicians, and educators and a well-informed citizenry that has access to the ideas and tools of science and engineering.”4 To these ends, EHR provides support for research and implementation activities that prepare the next generation of STEM professionals and attract and retain more Americans to STEM careers; develop a robust research community that can conduct rigorous research and evaluation to support excellence in STEM education and integrate research and education; increase the technological, scientific, and quantitative literacy of all Americans so that they can exercise responsible citizenship and live productive lives in an increasingly technological society; and broaden participation and close achievement gaps in all STEM fields.5
This request seeks renewal of OMB 3145-0226 for five data collections that have similar elements and purposes and provide essential information for program monitoring purposes. The collections contain items in two categories of programs (i.e., scholarship/fellowship programs and implementation, development, and research programs).
Data collected by EHR program monitoring systems are used for program planning, management, evaluation, and audit purposes. Summaries of monitoring data are used to respond to queries from Congress, the public, NSF’s external merit reviewers who serve as advisors, including Committees of Visitors (COVs), and NSF’s Office of the Inspector General. These data are needed for effective administration, program and project monitoring, evaluation, and measuring attainment of NSF’s program and strategic goals, consistent with the Government Performance and Results Act (GPRA) Modernization Act of 2010 and NSF’s Strategic Plan.
The five program-specific collections included in this request (see Exhibit 1) are designed to assist in management of specific programs, divisions, or multi-agency initiatives and to serve as data resources for program evaluations. Two other collections included in the prior request for renewal—the Science, Technology, Engineering and Mathematics Program (STEP) and the Integrative Graduate Education and Research Traineeship Program (IGERT)—are not included in this renewal request. Because of changes in program focus and emphasis since this collection was last cleared, these legacy programs have either been replaced by new programs, combined with others, ended, or no longer require program-specific monitoring data collections. Accordingly, the level of monitoring activity and total burden for the collections covered by this approval have decreased.
Exhibit 1: Collections covered by this request
Program |
Type of Program |
Centers of Research Excellence in Science and Technology (CREST) and Historically Black Colleges and Universities Research Infrastructure for Science and Engineering (HBCU-RISE) Monitoring System |
Implementation, Development, & Research |
Louis Stokes Alliances for Minority Participation (LSAMP) Monitoring System |
Implementation, Development, & Research; Scholarships and Fellowships |
Louis Stokes Alliances for Minority Participation Bridge to the Doctorate (LSAMP-BD) Monitoring System |
Scholarships and Fellowships |
Robert Noyce Teacher Scholarship Program (Noyce) Monitoring System |
Scholarships and Fellowships |
Scholarships in Science, Technology, Engineering, and Mathematic (S-STEM) Monitoring System |
Scholarships and Fellowships |
EHR is responsible for analyzing and evaluating the Directorate’s STEM education and human resource development and research programs and they activities they support.
EHR Monitoring Systems Clearance
Since the original request for this collection, EHR has continued to refine and enhance strategies for generating evidence to inform assessments of, and decisions regarding, EHR programs and portfolios of investments. EHR periodically reviews and assesses the ways in which monitoring data are used within the Directorate, and plausible alternative sources of information (including NSF enterprise data) to meet EHR’s (and other stakeholders’) needs for evidence on the Directorate’s STEM workforce development, broadening participation and institutional capacity, and STEM education research programs and investments. The Directorate’s Evaluation and Monitoring Group (EMG) regularly consults and collaborates with EHR program staff and leadership, colleagues in NSF’s Evaluation and Assessment Capability (EAC), and others to: establish evidence-building agendas; develop strategies and processes to address questions about EHR programs and investments; and provide guidance in the development of common metrics and scalable approaches to provide robust, timely evidence for a wide range of improvement, management, evaluative, transparency, and accountability purposes.
Issues Addressed in the Initial Collection Request
The initial request that created OMB 3145-0226 addressed the extent to which monitoring data in the collection were used in two ways:
Do monitoring systems collect data needed to assess programs?
These monitoring systems provide data required to assess the progress of projects in each program. The monitoring data also contribute to the overall assessment of program performance. In the case of programs that are primarily fellowship or scholarship programs, collection of information about participants in those programs is essential to any future tracking of their progress and determination of the impact of participation in the program. Programs with implementation and/or development goals utilize detailed information about the initial efforts of individual projects to track the potential impact of those efforts in successive locations. The monitoring systems collect project-level information on the scale, scope, and state of each project along with information on types of activities implemented; results, such as publications and number of students and/or faculty involved in the project; and partners. This information is essential for program management and reporting purposes; it is used to document the development, implementation, adaptation, dissemination, and results of supported activities. In addition, the program monitoring system data are an important source of information for program evaluation.
To what extent are monitoring data used to shape questions for a third-party evaluation?
NSF policy requires the development of a management plan to accompany every program announcement and solicitation. A plan for monitoring and assessing program activities is a required element of the program management plan; this may include plans for data collection and external evaluation. As noted in the previous clearance request, and reiterated above, EHR programs rely on their monitoring data collections to contribute to and inform such third-party evaluations. Without these data, third-party evaluators could be required to collect data about program participants and program projects after awards had been completed rather than during the period of performance of an award.
Circumstances of Data Collection
To fulfill its planning and management responsibilities, and to answer queries from Congress, OMB, and NSF management, EHR needs current and standardized information about projects in NSF’s Education and Training System of Records portfolio. This information is specifically important to support studies and evaluations by EHR, and studies by other NSF organizational units for project monitoring and effective program administration. The information is retained in accordance with the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). The Education and Training System of Records has several purposes, including:
providing a source of information on demographic and educational characteristics and employment plans of participants in NSF-funded educational projects, in compliance with Foundation responsibilities to monitor scientific and technical resources enabling NSF to monitor the effectiveness of NSF-sponsored projects and identify outputs of projects funded under NSF awards for management and for reporting to the Administration and Congress, (e.g., further to the GPRA Modernization Act of 2010, 5 U.S.C. 306 and 39 U.S.C. 2801-2805 and other requirements); and
creating public use files (which contain no personally identifiable information) for research purposes.
The data collected under this request are focused on initiative-specific, division-specific, and program-specific quantitative and qualitative data collection activities. Data from these collections are focused on participant demographic detail (particularly for scholarship and fellowship programs) and activities and outputs (i.e., the accomplishments of program grantees [projects] in terms of specific objectives). These descriptive data collections provide essential information for documenting progress toward NSF’s major performance goals, as described in NSF’s Strategic Plan (NSF 22-068): empower STEM talent to fully participate in science and engineering; create new knowledge about our universe, our world, and ourselves; benefit society by translating knowledge into solutions; and excel at NSF operations and management.
A.2. Purposes and Uses of the Data
Data collected under this request are required for effective program administration, program and project monitoring, evaluation, and for measuring attainment of NSF’s program and strategic goals as laid out in NSF’s Strategic Plan. This section describes how data to be collected under the clearance authority will be used for internal program management and administration; as a data source for NSF’s performance assessment activities, including COVs and Directorate and Office Advisory Committees; for documenting the attainment of NSF’s program and strategic goals; and as a foundation for rigorous assessment of the effectiveness of STEM education programs.6
Program Management and Administration
One of the primary uses of data from the EHR program monitoring clearance is for the general oversight of project and program activities by EHR staff. EHR has a limited number of staff members who must monitor hundreds of projects. Large-scale data collection is an efficient and effective mechanism for program officers to track project activities. The monitoring systems that fall under OMB 3145-0226 allow program officers and other NSF staff to integrate pre-existing data from the NSF administrative data system and newly generated data in a coherent and timely manner, providing information needed to adjust program portfolios. This kind of monitoring may stimulate respondents to iteratively refine their projects’ activities, facilitate changes in program guidelines and/or NSF funding levels to a particular project, and result in improved benefits to participants in NSF projects.
Data for Performance Assessment
Data from the monitoring systems contribute to NSF’s performance assessment activities and support the larger NSF evaluation model. NSF relies on the judgment of external experts to maintain high standards of program management, to provide advice for continuous improvement of NSF performance, and to ensure openness to the research and education community served by the Foundation. COV reviews are conducted at regular intervals of approximately four years for programs and offices that recommend or award grants, cooperative agreements, and/or contracts and whose main focus is the conduct or support of NSF research and education in science and engineering.7 COV reviews provide NSF with external expert judgments in two areas: (1) assessments of the quality and integrity of program operations and program-level technical and managerial matters pertaining to proposal decisions; and (2) comments on how the results generated by awardees have contributed to the attainment of NSF’s mission and strategic outcome goals. Data collected via the EHR program monitoring systems are used to inform the development of materials for the EHR Committees of Visitors.
Another central use of the EHR program monitoring data collected subject to this approval is to report on progress toward and document attainment of NSF program and strategic goals. The Foundation’s FY 2022–2026 Strategic Plan describes four strategic goals: (1) Empower STEM talent to fully participate in science and engineering.; (2) Create new knowledge about our universe, our world and ourselves (3) Benefit society by translating knowledge into solutions; and (4) Excel at NSF operations and management. EHR contributes to the attainment of these goals through programs of activity that:
prepare the next generation of STEM professionals and attract and retain more Americans to STEM careers;
develop a robust research community that can conduct rigorous research and evaluation that will support excellence in STEM education and that integrates research and education;
increase the technological, scientific, and quantitative literacy of all Americans so that they can exercise responsible citizenship and live productive lives in an increasingly technological society; and
broaden participation (individuals, geographic regions, types of institutions, STEM disciplines) and close achievement gaps in all STEM fields.8
The five EHR programs whose awards and activities are the subject of the data collections described in this renewal request play critical roles in the attainment of these objectives. Much of the information that enables EHR to monitor their progress and report on their accomplishments (e.g., consistent with the performance and improvement requirements of the GPRA Modernization Act of 2010) is derived from the data elements collected in the monitoring systems under OMB 3145-0226.
A Foundation for Future Evaluations
EHR places a strong emphasis on evidence-based decision making and is committed to generating robust evidence to inform the development, management, and assessment of its programs and portfolios of investment. This is consistent with the Administration's "strong and enduring commitment to…engaging in high-quality evaluations to learn and improve”.9
EHR has encouraged the use of monitoring data in its evaluation activities, creating a foundation for robust assessments of program activities. The monitoring systems used to collect data under this collection are not evaluative studies. However, the data they generate play a role in program assessment and program-level management reviews to ensure that programs are administered properly and in accordance with federal guidelines and agency missions. Data gathered through OMB 3145-0226 contribute to the formal evaluation of programs (e.g., facilitate the design and conduct of rigorous evaluation studies) and provide regular measures of program performance. Ways in which the monitoring data which are the subject of this renewal request might be used in evaluation include:
creating a universe data set with which to compare and establish representativeness of sample data;
providing data with which to verify/assess quality of evaluation data; and
providing data with which to establish population baseline and/or trend data.
With continued collection of these data, EHR will be able to leverage program monitoring system data from prior years and more efficiently and effectively track outputs, outcomes, and progress towards the attainment of program and Directorate goals over time. More generally, access to these data is critical to realizing the commitments to building and using evidence that are central to the Administration’s vision for results-driven government and the provisions of the Foundations for Evidence-Based Policymaking Act of 2018 (Pub.L. 115-435).
All the collections included under this clearance request use Web-based data collection systems to minimize data duplication and respondent burden. EHR favors Web-based systems because they facilitate respondents’ data entry across computer platforms. One innovative feature of many of the individual Web systems is the thorough reviewing and editing of all submitted data for completeness, validity, and consistency. Editing and validation are performed as data are entered. Most invalid data cannot be entered into the system, and questionable or incomplete entries are called to respondents’ attention before they are submitted to NSF.
EHR program monitoring Web-based data collection systems employ user-friendly features such as automated tabulation, data entry with custom controls such as checkboxes, data verification with error messages for easy online correction, standard menus, and predefined charts and graphics. These features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce burden.
All collections in the EHR program monitoring clearance comply with Section 508, the 1998 amendment to the Federal Rehabilitation Act of 1973, which mandates that the electronic and information technology used by federal agencies be made accessible to all people with disabilities.
The EHR program monitoring clearance does not duplicate efforts undertaken by the Foundation, other federal agencies, or other data collection agents. For example, NSF grants require the submission of annual and final project reports in accordance with OMB 3145-0058. Recipients of NSF grants, such as PIs, create and submit annual and final project reports through Research.gov. Data collected under the EHR program monitoring clearance are unique and not available in either the NSF annual or final reporting system. The submission of project information in annual and final reports based on the RPPR format does not remove the need for additional information that monitoring systems provide on a program-specific basis.
None of the five collections included in this request for renewal of the EHR program monitoring clearance collect information from small businesses.
Data collected for the EHR program monitoring clearance are used to manage programs, monitor projects, inform project and program evaluations, coordinate with federal and non-federal education partners, provide Congress with information about government-supported activities, and report for GPRA and other requirements. In many cases, the data need to be collected annually to inform the NSF management and evaluation processes. Data collected under the EHR program monitoring clearance can be used by NSF management to document and measure NSF’s success at achieving both strategic outcome goals and internal annual performance goals.
If the information were not collected, NSF would be unable to document the implementation of project activities and outcomes of its programs. It would be unable to meet its accountability requirements or assess the degree to which projects and programs are meeting their goals.
All data collections will comply with 5 CFR 1320.6. All collections under the EHR program monitoring clearance ask respondents for data annually, apart from the S-STEM monitoring system, which asks respondents to submit data each semester/quarter. See attachment E1 for more information on the frequency of this collection.
The notice inviting comments on the EHR program monitoring clearance (OMB 3145-0226) was published in the Federal Register February 15, 2022, Volume 87, Number 31, pages 8616-8618. No substantial comments were received in response to the Federal Register notice.
When developing collection instruments, EHR routinely consults with research and evaluation experts, PIs, and educators affected by EHR investments. The purpose of these consultations is to assess the relevance, availability, and clarity of items. As suggested by OMB guidelines, these consultations also enable EHR staff to obtain a reliable estimate of the respondent burden generated by new instruments. For data collections conducted earlier under the EHR program monitoring clearance, consultations have included knowledgeable outsiders such as representatives of EHR contractors responsible for technical and evaluation tasks and fellows who work at the Foundation as guests under programs such as the American Association for the Advancement of Science (AAAS) Science & Technology Policy Fellowships Program.
To date no payments or gifts have been provided to respondents. There are no plans to provide incentives to respondents because the value of program and project monitoring surveys is of value to the respondents as well as NSF. Program monitoring can be used by projects as a foundation for project-level evaluation.
Respondents are informed that any information on specific individuals is maintained in accordance with the Privacy Act of 1974. Each data collection instrument displays both OMB and Privacy Act notices.
Respondents are told that data collected for the EHR program monitoring clearance are available to NSF officials and staff, evaluation contractors, and the contractors hired to manage the data and data collection software. Data are processed according to federal and state privacy statutes. Detailed procedures followed by EHR for making information available to various categories of users are specified in the Education and Training System of Records (63 Fed. Reg. 264, 272 January 5, 1998). This system limits access to personally identifiable information to authorized users. Data submitted are used in accordance with criteria established by NSF for monitoring research and education grants and in response to Public Law 99-383 and 42 USC 1885c.
The information requested through NSF monitoring systems may be disclosed to qualified researchers and contractors to coordinate programs and to a federal agency, court, or party in court or federal administrative proceedings, if the government is a party.
Currently the five collections in the EHR program monitoring clearance request information from respondents, including name, address, Social Security Number (SSN), date of birth (DOB), and/or grade point average (GPA). These data are collected to monitor the award sites and evaluate the success of the award programs. Information of this nature is also used to track recipients of funding and training. However, in all collections that request SSN, SSN is a voluntary field. Responses to all items of a sensitive nature are voluntary. Respondents may choose not to provide information that they deem as privileged, such as SSN, address, or DOB. Any individual-level data that are collected are provided only to program staff and consultants conducting studies using the data as authorized by NSF. Any public reporting of data is in aggregate form. Exhibit 2 shows which individual collections currently include questions of a sensitive nature (designated with an ‘X’ in the appropriate column) and which propose to continue collecting such information (designated with an ‘X’ highlighted in yellow).
Exhibit 2: Questions of a sensitive nature currently collected and proposed to continue
Attachments |
Collection Title |
Address |
DOB |
GPA |
Name |
SSN |
A1-A2 |
Centers of Research Excellence in Science and Technology (CREST) and Historically Black Colleges and Universities Research Infrastructure for Science and Engineering (HBCU-RISE) Monitoring System |
X |
|
|
X |
|
B1-B2 |
Louis Stokes Alliances for Minority Participation (LSAMP) Monitoring System |
|
|
X |
X |
X |
C1-C2 |
Louis Stokes Alliances for Minority Participation Bridge to the Doctorate (LSAMP-BD) Monitoring System |
X |
|
X |
X |
X |
D1-D2 |
Robert Noyce Teacher Scholarship Program (Noyce) Monitoring System |
|
X |
X |
X |
|
E1-E2 |
Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM) Monitoring System |
X |
X |
X |
X |
|
As shown in Appendix A and in Exhibit 3 below, the annual response burden for the five collections under OMB 3145-0226 is 19,133 hours (for 1,893 respondents and 2,537 responses). Given the diversity of respondent types, the methods used to arrive at individual collection burden estimates are described in detail in attachments A1 through E1.
Exhibit 3: Respondents, responses, and annual hour burden
Attachment |
Collection Title |
No. of Respondents |
No. of Responses |
Annual Hour Burden |
A1 |
Centers of Research Excellence in Science and Technology (CREST) and Historically Black Colleges and Universities Research Infrastructure for Science and Engineering (HBCU-RISE) Monitoring System |
46 |
46 |
1,476 |
B1 |
Louis Stokes Alliances for Minority Participation (LSAMP) Monitoring System |
643 |
643 |
10,288 |
C1 |
Louis Stokes Alliances for Minority Participation Bridge to the Doctorate (LSAMP-BD) Monitoring System |
53 |
53 |
530 |
D1 |
Robert Noyce Teacher Scholarship Program (Noyce) Monitoring System |
511 |
511 |
4,599 |
E1 |
Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM) Monitoring System |
640 |
1,280 (640 respondents x2 responses/yr.) |
2,240 |
|
Total |
1,893 |
2,533 |
19,133 |
Exhibit 4 presents an example illustrating how the hour burden was estimated for the CREST/HBCU-RISE monitoring system (detailed in Attachment A1).The estimated average number of annual respondents is 46 (33 CREST center PIs/program coordinators and 13 HBCU-RISE award PIs/program coordinators), with an estimated annual response burden of 1,476 hours. The Web-based data collection is an annual activity of the CREST/HBCU-RISE program. The respondents are either PIs or program coordinators. Generally, one PI or program coordinator per award completes the questionnaire. The estimated annual hour burden per respondent was determined using the burden information reported by respondents from the last two collection cycles.
Exhibit 4: Sample calculation of burden hour estimate
Respondent Type |
Estimated Average Annual No.
of |
Estimated Average Annual Burden Hours Per Respondent |
Estimated Annual Burden Hour Total |
CREST center PIs/program coordinators |
33 |
40 |
1,320 |
HBCU-RISE award PIs/program coordinators |
13 |
12 |
156 |
Total |
46 |
32.08 |
1,476 |
Details on the burdens of each form can be found in attachments A1 through E1. Exhibit 5 provides an example of how this burden was estimated for the CREST/HBCU-RISE monitoring system (details in Attachment A1):
Exhibit 5: Sample calculation of hour burden estimates by form
Form Type |
Respondent Type |
No. of Respondents |
Burden Hours Per Respondent |
Total Burden Hours |
CREST/HBCU-RISE data collection form |
PIs/program coordinators |
46 |
32.08 |
1,476 |
Total |
|
46 |
|
1,476 |
As shown in Appendix A, the total annual cost to respondents generated by the five ongoing data collections is currently estimated to be $818,650. Following is an example of the method used to calculate cost burden for the CREST/HBCU-RISE monitoring system (details in Attachment A1):
The overall annualized cost to the respondents is estimated to be $78,228. Exhibit 6 shows the annualized estimate of cost to PI/program coordinator respondents, who are generally university professors. This estimated hourly rate is based on a report from the American Association of University Professors, “The Annual Report on the Economic Status of the Profession, 2020–2021,” Survey Report Table 1. According to this report, the average salary across all academic ranks and across all types of doctoral-granting institutions (public, private-independent, religiously affiliated) was $109,428. When divided by the number of standard annual work hours (2,080), this calculates to approximately $53 per hour.
Exhibit 6: Sample calculation of estimated annualized cost to respondents
Respondent Type |
No. of Respondents |
Burden Hours Per Respondent |
Average Hourly Rate |
Estimated Annual Cost |
PIs/Program Coordinators |
46 |
32.08 |
$53 |
$78,228 |
Total |
46 |
|
|
$78,228 |
The costs to respondents generated by each data collection are described in attachments A1 through E1.
There is no overall annual cost burden to respondents or record-keepers that results from the EHR program monitoring clearance other than the time spent responding to online questionnaires that are described in specific detail in attachments A1 through E1. It is usual and customary for individuals involved in education and training activities in the United States to keep descriptive records. The information being requested is from records that are maintained as part of normal educational or training practice. Furthermore, most respondents are active or former grantees or participants in programs or projects funded by NSF. In order to receive funding, institutions must follow the instructions in the NSF Proposal and Award Policies and Procedures Guide (PAPPG) that is cleared under OMB 3145-0058. The PAPPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane or Research.gov. Thus, PIs, K-12 administrators, faculty members, and college students, who are the primary respondents to the individual data collections within the EHR program monitoring clearance, make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance cost under OMB 3145-0058, and free software (e.g., Microsoft Explorer) to respond.
As shown in Appendix A, the total annual cost to the Federal Government of the five ongoing data collections is currently estimated to be $1,102,681. Details of the cost of each collection can be found in Appendix A. Following is an example of the calculation of cost to the Federal Government for the CREST/HBCU-RISE data collection (details in Attachment A1). More details on the costs of existing collections can be found in attachments A1 through E1.
Computing the annualized cost to NSF for the CREST/HBCU-RISE data collection was done by taking the projected budget for the next three years and calculating the cost for each of the following operational activities involved in producing, maintaining, and conducting the data collection (see Exhibit 7).
Exhibit 7: Sample calculation of estimated annualized cost to the Federal Government
Operational Activities |
Cost Over Three Years |
System Development (includes initial development of the database and Web-based application, and later changes requested by the program, e.g., increased reporting tools, additional validations) |
$164,197 |
System Maintenance, Updates, and Technical Support (system requires updates each year before opening the collection; maintenance is required to keep the system current with technology, e.g., database servers, operating systems) |
$295,556 |
Data Collection Opening and Support (e.g., online and telephone support to respondents and contacting respondents to encourage completion of the questions), Reporting (as defined by HRD), and Follow-up Activities (e.g., providing data to other consultants) |
$197,037 |
Three-Year Total for All Operational Activities |
$656,790 |
The annualized cost was computed as one-third of the total three-year cost; thus, the annualized cost to NSF for the CREST/HBCU-RISE data collection is $218,930.
A.15.1. Proposed changes to item wordings
Working with the contractor that administers the program monitoring collection systems that are the subject of this renewal request to examine consistency of select items across collections, EHR proposes to revise to align across the collections items that request information on individuals’ field/discipline (e.g., field of study). In general, the item revisions are designed to follow the formats described below. Additionally, the CREST collection intends to amend item response categories for one question (revising by reducing the list of programs about which respondents are prompted to provide information). While we note these proposed changes here, none are anticipated to have an impact on the time required to complete the surveys, the estimated respondent burden hours required by form, the aggregate hour burdens, or the estimated annualized cost to respondents.
Field/discipline items: Currently field/discipline is described in a variety of ways (e.g., area of study, discipline, major, field), with various response options, across the data collections. EHR proposes to align item wording (with minor adjustments as necessary to specific wordings to ensure the questions remain relevant to different program requirements), format, and response options (based on a searchable list of from the National Center for Science and Engineering Statistics Survey of Earned Doctorates or the National Center for Education Statistics Classification of Instructional Programs10). With OMB approval, these items would be revised as indicated in Exhibit 9.
Exhibit 8: Proposed new wording for field/discipline items
What is this person’s [“DISCIPLINE OR FIELD OF STUDY”/”MAJOR”] for the current degree or certificate program?
[SEARCHABLE LIST OF SED OR CIP FIELDS WITH “OTHER” OPTION] If other, please specify: [TEXT BOX] [CHECK BOX] Undecided + Add another [“DISCIPLINE OR FIELD OF STUDY”/”MAJOR”]
|
Additionally, the decision to discontinue reporting of select information in the LSAMP collection (e.g., requesting counts of student enrollment in specific disciplines in the aggregate only, rather than by race, ethnicity, or gender) results in changes to the wording of 22 items on the LSAMP collection.
Detailed information on the proposed changes to currently approved item wordings are included in Attachments A1 and B1. As instrumentation would not be changed without OMB approval, these revised item wordings are not reflected in the screenshots (Attachments A2 and B2) of the extant instrumentation.
A.15.2. Proposed new items
One collection (CREST) proposes to add an item to its currently approved instruments. While we note the proposed change here, it is not anticipated to have an appreciable impact on the time required to complete the surveys, thus has no implications for the estimated respondent burden hours required by form, the aggregate hour burdens, or the estimated annualized cost to respondents.
CREST proposes to add an item to facilitate more accurate assessment of supported participants’ research contributions. The item asks respondents to (a) provide or (b) indicate they prefer not to provide an Open Researcher and Contributor ID (ORCID) if they have one, or (c) to indicate if they do not have an ORCID.
Detailed information on the revision proposed is included in Attachment A1. As instrumentation would not be changed without OMB approval, these revised item wordings are not reflected in the screenshots (Attachments A2) of the extant instrumentation.
A.15.3. Items proposed for discontinuation
NSF plans during the period covered by this renewal request to discontinue asking 163 currently approved items. These include items requesting information on race, ethnicity, gender, disability status, and other information specific to individuals. The items in question are flagged in Attachments A1-E1.
A.15.4. Proposed changes to task burdens and numbers of respondents
The current inventory numbers at OMB for the EHR program monitoring clearance covers individual collection tasks. The OMB inventory records show a total number of responses of 7,784 and total hours of 32,698, including responses from two discontinued collections (IGERT and STEP). Hours associated with these discontinued collections have been removed from the ‘previously cleared burden’ column total in Exhibit 9.
This renewal includes five individual collection tasks and requests 2,533 annual responses and 19,133 total hours annually; details can be found in Appendix A. The change in burden is due to shifts in the number of respondents and changes in the number of items asked across the collections. Exhibit 9 shows the changes in burden in the individual collections.
Exhibit 9: Hour changes in task burdens
Collection Title |
Previously Cleared Burden |
Currently Requested Burden |
Change in Burden |
Centers of Research Excellence in Science and Technology (CREST) and Historically Black Colleges and Universities Research Infrastructure for Science and Engineering (HBCU-RISE) Monitoring System |
1,648 |
1,476 |
(172) |
Louis Stokes Alliances for Minority Participation (LSAMP) Monitoring System |
16,250 |
10,288 |
(5,962) |
Louis Stokes Alliances for Minority Participation Bridge to the Doctorate (LSAMP-BD) Monitoring System |
1,008 |
530 |
(478) |
Robert Noyce Teacher Scholarship Program (Noyce) Monitoring System |
6,050 |
4,599 |
(1,451) |
Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM) Monitoring System |
4,900 |
2,240 |
(2,660) |
NSF Burden Estimate Total |
29,856 |
19,133 |
(10,723) |
The total change of burden is a decrease of 10,723 hours.
Changes in the hour burden are accompanied by changes in the number of respondents. Exhibit 10 shows the changes in total number of respondents.
Exhibit 10: Changes in number of respondents
Collection Title |
Previously Cleared No. of Respondents |
Currently Requested No. of Respondents |
Change in No. of Respondents |
||||
Centers of Research Excellence in Science and Technology (CREST) and Historically Black Colleges and Universities Research Infrastructure for Science and Engineering (HBCU-RISE) Monitoring System |
42 |
46 |
4 |
||||
Louis Stokes Alliances for Minority Participation (LSAMP) Monitoring System |
625 |
643 |
18 |
||||
Louis Stokes Alliances for Minority Participation Bridge to the Doctorate (LSAMP-BD) Monitoring System |
56 |
53 |
(3) |
||||
Robert Noyce Teacher Scholarship Program (Noyce) Monitoring System |
550 |
511 |
(39) |
||||
Scholarships in Science, Technology, Engineering, and Mathematics (S-STEM) Monitoring System |
700 |
640 |
(60) |
||||
NSF Respondent Estimate Total |
|
1,893 |
(80) |
The change in respondents is due largely to variations in the numbers of awards active from year to year. In future years, EHR anticipates burden will similarly be affected by variations in award populations. NSF will notify OMB when there are significant changes to the burden.
The data collections that are the subject of this renewal request are utilized for multiple purposes, described in sections A.1 (Circumstances Requiring the Collection of Data) and A.2 (Purposes and Use of the Data). Most of the data the EHR program monitoring clearance collects are used for internal purposes, e.g., informing how NSF manages, documents, evaluates, and measures its performance as an agency. NSF’s GPRA Performance Report or an individual division’s annual report to the NSF Director may use information from the collection to report to Congress. This is an annual cycle.
The data collection efforts included under this request are administered by third-party contractors that deliver (1) analytical reports, (2) the raw data from the collections, or (3) both. Third parties are contractually forbidden from publishing results unless NSF has made a specific exception. All products of the collections are the property of NSF.
When reports on studies that employ monitoring data or documents presenting analyses of monitoring data are approved for publication, distribution is likely to be electronic in nature. For content authored by NSF or by a third party at NSF’s request, the agency rarely uses paper to publish the information. NSF publishes most documents electronically only using the agency’s Web site, from requests for proposals to evaluation or statistical reports, using an archive called an ODS. Public reports on studies that make use of monitoring data are typically made available from the EHR main Web page, part of the NSF main public Web page.
EHR recurring studies based on monitoring data are requested by program staff and are done to monitor, manage, and communicate with and about the individuals supported by NSF’s investments in education and training. In most cases the primary purpose for each recurring study is program management. These studies generate data that enable both NSF and the funded education and training projects to improve management and performance. Typically, recurring studies generate information that NSF uses as inputs to other reports, and therefore EHR cites no specific publication plans other than internal or general use to meet reporting requirements.
EHR uses data from recurring studies to provide information that can be mined for program evaluation purposes, such as identifying best practices in the education of graduate and undergraduate students, or as a baseline for summative evaluation reports.
Not applicable
No exceptions apply.
1 NSF. (2022). NSF Strategic Plan for Fiscal Years (FY) 2022-2026: Leading the World in Discovery and Innovation, STEM Talent Development and the Delivery of Benefits from Research, NSF 22-068. Alexandria, VA: NSF. Retrieved June 2022 from https://www.nsf.gov/pubs/2022/nsf22068/nsf22068.pdf.
2 National Science Foundation. (2022). About NSF: At a Glance. Retrieved June 2022 from https://www.nsf.gov/about/glance.jsp.
3 National Science Foundation. (2022). How We Work. Retrieved June 2022 from https://www.nsf.gov/about/how.jsp.
4 National Science Foundation. (2022). About Education and Human Resources (EHR). Retrieved June 2022 from https://www.nsf.gov/ehr/about.jsp.
5 Ibid.
6For general information on NSF performance assessment activities see https://www.nsf.gov/about/performance/.
7See “Committee of Visitors (COV),” retrieved June 2022 from https://www.nsf.gov/od/oia/activities/cov/. COV reports are available at https://www.nsf.gov/od/oia/activities/cov/covs.jsp.
8See “About Education and Human Resources (EHR),” retrieved June 2022 from https://www.nsf.gov/ehr/about.jsp.
9 Office of Management and Budget (OMB). (2022). “Building and using evidence to improve government effectiveness,” Chapter 6 of Analytical Perspectives: Budget of the U.S. Government, Fiscal Year 2023. Washington, D.C.: OMB. Retrieved June 2022 from https://www.whitehouse.gov/wp-content/uploads/2022/03/ap_6_evidence_fy2023.pdf.
10 For information on the Survey of Earned Doctorates (SED), see https://www.nsf.gov/statistics/srvydoctorates/; for information on the Classification of Instructional Programs (CIP), see https://nces.ed.gov/ipeds/cipcode/Default.aspx?y=56.
Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Supporting Statement |
| Author | Greenberg, Kevin |
| File Modified | 0000-00-00 |
| File Created | 2022-07-29 |