Part A of supporting statement (2024_04_24)

6

Supporting Statement PART A

A.1. Circumstances that Make Data Collection Necessary

Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The National Science Foundation's (NSF's) Division of Graduate Education (DGE) in the Directorate for STEM Education (EDU) administers the NSF Research Traineeship (NRT) program. The NRT program is designed to encourage the development and implementation of bold, new, and potentially transformative models for STEM graduate education training. The NRT program seeks to ensure that graduate students in research-based master's and doctoral degree programs develop the skills, knowledge, and competencies needed to pursue a range of STEM careers. NRT is dedicated to providing effective training of STEM graduate students in high priority interdisciplinary or convergent research areas through the use of a comprehensive traineeship model that is innovative, evidence-based, and aligned with changing workforce and research needs. The NRT program also focuses on broadening participation in STEM learning and careers, particularly among those individuals traditionally underrepresented and underemployed in the STEM workforce, including but not limited to, women, persons with disabilities, and racial and ethnic minorities.

Initially, NRT awardees provided NSF with information on their activities through periodic research performance progress reports (RPPRs, OMB Control No. 3145-0221). However, the NRT program requires (for program management and assessment/evaluative purposes) additional (and more finely grained) information beyond that requested/specified in the RPPR format. This information is required for effective administration, communication, program and project monitoring and evaluation, and for measuring attainment of NSF's program, project, and strategic goals, consistent with the GPRA Modernization Act of 2010¹, the Foundations for Evidence-Based Policymaking Act of 2018², and the NSF's Strategic Plan³. Without these data, NSF is unable to document the implementation of NRT project activities and outcomes and to provide satisfactory responses to requests from OSTP and Congress. The NRT monitoring system (also referred to as the NRT reporting system) addresses these needs. In 2022, the NRT monitoring system (OMB Control No. 3145-0263) began collecting data to enable awardees to satisfy RPPR reporting requirements while submitting additional information critical for program management and accountability purposes. The NRT reporting system uses internet-based information and communication technologies to collect, review, and validate specific data on NRT awards.

EDU is committed to ensuring the efficiency and effectiveness with which respondents provide and NSF staff can access and analyze data on funded projects within the NRT programs. This request proposes to renew the data collection for the NRT monitoring system (OMB Control No. 3145-0263). Some modifications to the survey instruments are proposed to improve the quality of the data collection and to improve the user experience. Other EDU programs have also implemented data collection via monitoring systems to allow for more detailed information to be collected for program management and accountability purposes. The establishment of the NRT monitoring system was following a practice successfully employed by the precursor program to the NRT, the Integrative Graduate Education and Research Traineeship (IGERT) data collection system (OMB Control No.3145-0226⁴).

The NRT solicitation⁵ alerts those submitting proposals that if they receive a NRT award, then the awarded “PIs will be required to submit annual and final project reports that differ from the standard reporting format contained in Research.gov. Instructions for preparing and submitting such reports will be provided to the PI.”

This renewal request proposes to continue the established practice of collecting information on an annual basis, according to the RPPR data collection schedule, using the NRT monitoring system.

A.2. Purpose and Use of the Information evidence

Indicate how, by whom, how frequently, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

Data collected under this request are required for effective program administration and to measure the attainment of the NRT program’s goals⁶. NSF will use the data from this collection for program planning, management, accountability, and audit purposes (e.g., to respond to queries from the Congress, the public, NSF's external merit reviewers who serve as advisors, including Committees of Visitors (COVs), the NSF's Office of the Inspector General, and as a basis for either internal or third-party evaluations of individual programs).

This section describes how data to be collected under the clearance authority will be used for (1) internal program management and administration; (2) as a data source for NSF performance assessment activities, including Committees of Visitors (COVs) and Directorate and Office Advisory Committees; (3) for documenting the attainment of NSF’s program and strategic goals; and (4) as a foundation for rigorous assessment of the effectiveness of the NRT program.⁷

The Foundation’s FY 2022–2026 Strategic Plan describes four strategic goals: (1) Empower, empower STEM talent to fully participate in science and engineering; (2) Discover, create new knowledge about our universe, our world and ourselves; (3) Impact, benefit society by translating knowledge into solutions; and (4) Excel, excel at NSF operations and management. NRT contributes to the attainment of these goals as well as the Division of Graduate Education’s Strategic Goals⁸ through programs of activity that:

• prepare the next generation of STEM professionals and attract and retain more Americans to STEM careers;

• develop a robust research community that can conduct rigorous research and evaluation that will support excellence in STEM education and that integrates research and education;

• increase the technological, scientific and quantitative literacy of all Americans so that they can exercise responsible citizenship and live productive lives in an increasingly technological society; and

• broaden participation (individuals, geographic regions, types of institutions, STEM disciplines) and close achievement gaps in all STEM fields.⁹

The NRT monitoring system allows NSF to further assess any findings related to trainee preparation for careers in STEM, barriers to the implementation of a project, and strategies for improving recruitment and retention of trainees in the NRT program. Data collected will be analyzed by subgroups of critical interest to NSF, including underrepresented minorities and women. These findings may inform future improvements to the program.

At the time of renewal submission, two rounds of data collection have been completed under OMB Control No. 3145-0263. These data enabled the program, for the first time, to describe the number and demographic characteristics of trainees that received stipends from the NRT awards. Common project activities related to interdisciplinary training, career preparation, and recruitment and retention are becoming apparent. In addition, the feedback collected directly from trainees regarding their experience with the NRT provides program staff with rich insight into how the program is received from the trainee perspective. The NRT program proposes to continue this data collection to continue the collection of data necessary to evaluate the program and to respond to requests to describe the populations benefiting from it.

The NRT monitoring system includes surveys to be completed by the different project participants:

• Project Survey: Completed by PI. The PI may also enable the Project Coordinator (PC) and/or Co-PI(s) to access and enter data. This instrument collects information on project activities, accomplishments, participants, and products for a single reporting year. A subset of data, including participant lists, are carried over in subsequent years to reduce burden. Information collected from participants (e.g., name and email) is used to initiate account set up for access to relevant surveys. Accompanying this is the Coordinator Instrument - this information is collected under the Project Survey and entered by the PI. It consists of contact information only, which is collected for the purpose of initiating an account.

• PI Survey: Completed by PI. This instrument collects basic contact and demographic data on the PI. All information carries over in subsequent years to reduce burden.

• Faculty Survey: Completed by Co-PIs and other Faculty named as participants in the project. This instrument collects basic contact and demographic data on the PI. All information carries over in subsequent years to reduce burden.

• Trainee Survey: Completed by trainees identified as participants in the Project Survey. This instrument collects demographic data, education and research background, future plans, as well as information on their activities, accomplishments, and experiences in the program. A subset of information carries over to subsequent years to reduce burden for trainees that are active across multiple reporting years.

This renewal of the existing OMB Control No. 3145-0263 collection, with minor modifications, requests information on (1) Staff and project participants; (2) project implementation characteristics; and (3) project outputs. Collection of these data will continue to satisfy RPPR requirements to provide program and project-level monitoring of grant award progress and outcomes annually. These data will continue to satisfy participant reporting requirements for Federal STEM education investments to address requirements of the America COMPETES Reauthorization Act of 2010. The NRT program is part of the inventory¹⁰ of Federal STEM education programs. In addition, these data may be used to provide the necessary foundation to determine individual-level treatment and control groups for future third-party study or for internal evaluation, to identify well-matched comparison groups, and to measure baseline for pre- and post-NSF-funding level impacts.

NSF uses data from the survey to inform improvements to the effectiveness and efficiency of the NRT program. Since this collection is primarily used for program monitoring and management, a census rather than sampling design is necessary.

At the project level, funding for continuing grants can be adjusted based on an individual project's responses to some of the surveys. Unless otherwise provided for in the original award notice, each increment of a continuing grant will be funded at the level indicated in the original award notice without a formal request, subject to NSF’s judgment of satisfactory progress, availability of funds, and receipt and approval of the required annual report. NSF makes every attempt to honor continuing grant commitments¹¹.

A.3. Use of Information Technology and Burden Reduction

Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.

The NRT monitoring system is administered entirely online via the web. NSF favors web-based systems because they facilitate respondents’ data entry across computer platforms. One innovative feature of the web system is the thorough reviewing and editing of all submitted data for completeness, validity, and consistency. Editing and validation are performed as data are entered. Invalid data cannot be entered into the system, and questionable or incomplete entries are called to the respondents’ attention before they are submitted to NSF.

The NRT monitoring system employs user-friendly features such as data entry with controls like checkboxes and radio buttons, data verification with error messages for easy online correction, standard menus, and tooltips for questions requiring additional, clarifying information. To further reduce burden, programmed skip patterns will allow respondents to only answer questions relevant to them and their experiences; in addition, the system carries over any data that can be pre-populated after the initial award year to further reduce burden in subsequent years. The surveys also allow respondents to save their progress at any point, allowing them to return to continue the surveys at their convenience. These features facilitate the reporting process, provide useful and rapid feedback to the data providers, and reduce the burden for the respondents.

NSF reduces burden with the use of application programming interfaces (APIs) to allow respondents to retrieve data stored elsewhere and to import it into the relevant sections of the NRT data collection. For example, respondents can retrieve a product’s information by entering a product’s digital object identifier (DOI). A DOI is a unique, persistent identifying number for a document published online. By entering the DOI for their products, respondents can quickly complete this section with minimal effort and the burden reduction is significant. The system also works with BibTeX. Respondents can enter DOI or BibTeX information one at a time or do a bulk upload to import multiple products at one time.

All data collected for the NRT program under this clearance will comply with Section 508, the 1998 amendment to the Federal Rehabilitation Act of 1973, which mandates that the electronic and information technology used by federal agencies be made accessible to all people with disabilities. The NRT Tech Support team is available to assist users that require additional support to access and/or complete the surveys.

A.4. Efforts to Identify Duplication and Use of Similar Information

Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purpose described in item 2 above.

The NRT program monitoring clearance will not duplicate efforts undertaken by the Foundation, other federal agencies, or other data collection agents. The data collected under the NRT program monitoring clearance are not duplicated in the NSF RPPR system. The data collected via the NRT monitoring system will satisfy the RPPR requirements.

A.5. Impacts on Small Businesses or Other Small Entities

If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

NSF has determined that the requirements for this information collection do not adversely impact small businesses or other small entities. The respondent populations do not include these entities.

A.6. Consequences of Collecting the Information Less Frequently

Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

The information collection proposed for this study consists of a set of surveys, including a project-level survey and additional survey instruments tailored for a specific respondent population. Respondents will complete annual surveys for the award's lifetime and will be used to satisfy RPPR requirements. If these data were collected less frequently, it would not satisfy the conditions of receiving a NSF award, which requires annual reporting on progress and activities. If these data were not collected at all or less frequently, NSF would:

• be unable to document the implementation of NRT project activities and outcomes.

• be unable to provide satisfactory responses to requests from OSTP, CoSTEM, and Congress that require demographic data.

• not have critical information that can be used to refine the NRT program, including examining participant satisfaction with the NRT program, variation of experiences across critical population subgroups, managing efforts for recruitment and retention, and an assessment of whether satisfaction, quality, and fairness have improved over time.

• have difficulty verifying that it has met its accountability requirements under the ‘‘Foundations for Evidence-Based Policymaking Act of 2018’’ or assess the degree to which projects and programs, such as NRT, are meeting their goals.

• not meet annual reporting requirements for NSF grants that are required to track performance progress, in addition to being unable to compare outputs and outcomes across various government agencies.

A.7. Special Circumstances Relating to the Guideline of 5 CFR 1320.6 – Public Protection

Explain any special circumstances that would cause an information collection to be conducted in a manner:

There are no special circumstances for the proposed data collection. The collection of information will be conducted annually in a manner consistent with the guidelines in 5 CFR 1320.6.

A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside Agency

If applicable, provide a copy and identify the date and page number of publications in the Federal Register of the agency’s notice, soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting form, and on the data elements to be recorded, disclosed, or reported.

1. Federal Register Notice and Comments

In accordance with the Paperwork Reduction Act of 1995, the public was given an opportunity to review and comment through the 60-day Federal Register Notice, published on December 21, 2023 (Vol. 88, No. 88, pages 88420-88421). One comment was received, but it was not relevant to the collection, so we propose to proceed with the request for renewal of this data collection (OMB Control No. 3145-0263).

2. Consultations Outside of the Agency

During the initial development of the NRT monitoring system, NSF consulted with Creative Business Solutions, Inc. (CBS) on the survey content and design, administration of the survey, and the final online user interface. Due to the similarities between the NRT program and its precursor program (IGERT), survey content was adapted from and modeled after many of the IGERT survey items (OMB Control No. 3145-0226).

Prior to receiving initial clearance for the NRT monitoring system data collection, the collection was pilot tested to obtain user feedback to ensure that the questions could be answered in a reasonable amount of time (i.e., that the burden estimates are accurate) and that the directions and question content were easy to understand and follow. The pilot test involved nine individuals who were either PIs or PCs. Members from the contracting team and NSF moderated video call “Think Aloud” exercises with two of the nine pilot testers so they could watch them use and reflect on the data collection in real time. Data obtained from both the system comments and the Think Aloud exercises were used to guide system development.

The current NRT monitoring system (OMB Control No. 3145-0263) allows for feedback and comments from users used to improve the user experience and reduce respondent burden. Users may provide feedback through comment boxes available throughout the system, through reaching out to technical support by phone or email, or by contacting NSF program staff. Additional approaches to gather feedback for improvement purposes include collecting feedback from PIs during meetings and conferences; comprehensive reviews by NSF staff; and testing performed by the system developers.

Any input on the system received from users is reviewed by NSF and CBS staff during regular check-ins. Where appropriate, the feedback has been used to improve the guidance provided to users and/or system functionality and features. In addition, this feedback has informed the revised data collection. For example, some respondents to the Project Survey noted that the Professional Skills Activities and Trainee Preparation sections appeared duplicative; therefore, this renewal proposes to reorganize the question order and remove questions to reduce the perception of duplication in data that was collected in the former Professional Skills Activities and Trainee Preparation sections. These are now in a single section. This led NSF and CBS staff to evaluate the sequence and content of the survey questions in these sections and to propose, in this renewal, to modify the sequence of these questions and to remove some checklist responses, thus providing clarity and further reducing burden. Moreover, CBS implemented an email reminder feature for PIs and PCs to use to contact personnel directly, as well as provided additional means of importing and managing products relevant to an award. After reviewing feedback gathered through CBS’s technical support system, it and NSF decided to alter the workflow process for PIs and PCs to manage their personnel and initiate report preparation in a more streamlined way without altering any content. This has allowed for timelier reporting for all respondents.

A.9. Explanation of Any Payment or Gift to Respondents

Explain any decision to provide any payment or gift to respondents, other than remuneration of contractors or grantees.

No payment or gift will be offered to survey respondents.

A.10. Assurance of Confidentiality Provided to Respondents

Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

All respondents are shown the following survey privacy notice when logging in to the system:

The Federal Government has a continuing commitment to monitor its awards to identify and address any inequities based on gender, race, ethnicity, or disability of the principal investigators (PIs), co-PIs, trainees, or other participants.

Information from this data collection system will be retained by the NSF, a Federal agency, and will be an integral part of its Privacy Act System of Records in accordance with the Privacy Act of 1974 and maintained in the Education and Training System of Records 63 Fed. Reg. 264, 272 (January 5, 1998). All individually identifiable information supplied by individuals or institutions to a Federal agency may be used only for the purposes outlined in the system of records notice and may not be disclosed or used in identifiable form for any other purpose, unless otherwise compelled by law. These are confidential files accessible only to appropriate NSF officials, their staff, and their contractors responsible for monitoring, assessing, and evaluating NSF programs. Only data in highly aggregated form or data explicitly requested "for general use" will be made available to anyone outside of NSF for research purposes. Data submitted will be used in accordance with criteria established by NSF for monitoring research and education grants, and in response to Public Law 99-383 and 42 USC 1885c.

A.11. Justification for Sensitive Questions

Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

Questions of a sensitive nature have been kept to an absolute minimum in the proposed collection. The only data that are sensitive in nature relate to respondent demographics (e.g., race, ethnicity, gender, disability, and veteran status.) and date of birth. Respondents may choose not to provide information that they deem privileged, such as demographic questions involving race, ethnicity, gender, disability, and veteran status. Although respondents are required to respond to these questions, they have the option of selecting “Do not wish to provide” for each demographic question and to leave the date of birth field blank.

Demographic information requested is based on government-wide standard categories currently in use on a variety of forms such as the U.S. Census and would only pertain to the people who have directly worked on the award. To better align with the White House “Best Practices” for collecting self-reported sexual orientation and gender identity (SOGI) data, and in accordance with the June 2022 Executive Order 14075 on Advancing Equality for Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex Individuals, this renewal request proposes to slightly modify two demographic questions relating to Gender and Veteran status; these modifications will allow for a more authentic, representative collection of NRT awardees, as demonstrated with NSF’s Education and Training Application (ETAP) program (OMB 3145-0248). The demographic data will be used for data analysis and reporting; it will not be used as part of the progress report evaluation. The data will be collected, maintained, and used in accordance with the Privacy Act of 1974, and any other applicable OMB and agency policies and practices.

NSF is committed to monitoring and identifying any real or apparent inequities based on gender, race, ethnicity, disability, or veteran status of the project participants. As such, these demographic data will allow the Foundation to gauge whether opportunities in science and technology for the NRT program are fairly reaching and benefiting everyone regardless of demographic category.

Otherwise, survey questions are limited to asking respondents to report their behavior (i.e., involvement in STEM activities, professional development activities and training) and their perceptions of their experiences (i.e., benefits, professional impact) as relates to the NRT program. Respondents are also asked to provide their names, an email address, institutional information, and funding information.

Respondents may also submit their Open Research and Contributor ID (ORCiD) if they wish. Submission of an ORCiD is optional. NRT asks for an ORCiD rather than sensitive information such as an SSN. As part of this renewal request, we are proposing to begin collecting date of birth (DOB) as an optional field from trainees only, in order to support future matching with other data sources, such as the National Student Clearinghouse (NSC), to track student educational outcomes. The ORCiD is a public, digital identifier used by academics and researchers to distinguish themselves from others who might share the same name. This id remains with the scholar or academic for their entire career. The ORCiD is also tied directly to publications and other academic products that they have produced. ¹²

Trainees will be notified at the beginning of the web survey that certain information will not be accessible by their PIs or other project personnel; however, a PI will be able to see if Trainees have or have not completed and submitted a section of their survey. Any individual-level data that are collected are provided only to program staff and contractors conducting studies using the data as authorized by NSF. Any public reporting of data is in aggregate form.

A.12. Estimates of Hour Burden Including Annualized Hourly Costs

Provide estimates of the hour burden of the collection of information. The statement should:

• Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. If this request for approval covers more than one form, provide separate hour-burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

• Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories.

As shown below in Table A.12.1 the NRT monitoring system consists of a set of five surveys.

Table A.12.1 – Surveys and Burden Comprising the NRT Monitoring System

Survey	Respondent Responsible	Response Frequency	Burden Estimate	Number of Respondents	Total Annual Hour Burden
Project Survey	PI/PC	Once annually	24 hours	120	2,880 hours
PI Survey	PI	Once annually	15 minutes	120	30 hours
Co-PI Survey	Co-PI	Once annually	10 minutes	1440	240 hours
Faculty Survey	Faculty	Once annually	10 minutes	1200	200 hours
Trainee Survey	Trainee	Once annually	1.5 hours	1200	1,800 hours

The estimated project survey burden listed in Table A.12.1 are based on the burden hours for RPPR respondents (OMB 3145-0221). The burden estimated for RPPR ranges from 2-16 hours. The NRT project survey respondents’ burden is somewhat higher than the RPPR burden, as NRT collects some additional information beyond the scope of RPPR. The burden for the PI, Co-PI, Faculty, and Trainee surveys were initially based on the burden reported by IGERT respondents. The NRT respondents’ burden was considered similar in scope to the IGERT respondents’ burden as the scope of the data collections are similar. Feedback from NRT respondents that was directly collected by the contractor indicates that these estimates appear in line with the current experience. The estimates indicated in the table reflect the burden for new participants. However, respondent data collected by the contractor indicates that returning respondents spend less time completing the instruments in subsequent years. For example, the average estimated time reported by PIs to complete their PI survey in the initial year of the collection was ~16 minutes whereas in Year 2 it declined to ~7 minutes. NRT award duration is typically five years, so the burden displayed is estimated for initial reporting years. For later reporting years (Years 2-5), less time is expected to complete the surveys.

Estimates of annualized cost to respondents are shown below in Table A.12.2, with appropriate wage rate categories used and identified.

Table A.12.2 – Estimates of Annualized Cost to Respondents Using the NRT Monitoring System

Survey	Respondent Responsible	Burden Estimate (hours)	Hourly Wage	Annual Cost Per Respondent	Number of Respondents	Total Annual Burden (hours)	Total Annual Cost
Project Survey	PI/PC	24 hours	$38.8713	$932.88	120	2,880	$111,946
PI Survey	PI	0.25 hours	$38.8713	$9.71	120	30	$1,165
Co-PI Survey	Co-PI	0.17 hours	$38.8713	$6.47	1440	40	$9317
Faculty Survey	Faculty	0.17 hours	$38.8713	$6.47	1200	200	$7,764
Trainee Survey	Trainee	1.5 hours	$16.3514	$24.52	1200	1,800	$29,424

For the NRT program, it is expected that all faculty respondents (including PIs, Co-PIs, PCs, and Faculty) will be working at an academic institution, likely in a teaching and/or research capacity. Therefore, for the purpose of cost estimates for Faculty, the annual mean wage for postsecondary teachers from Bureau of Labor Statistics has been used, which is $80,840¹³. Assuming a 40-hour workweek over the course of 52 weeks annually, the hourly wage for this occupation is approximately $38.87.

For the purpose of cost estimates for Trainees, the current minimum stipend rate for NRT Trainees has been used, which is $34,000¹⁴. Assuming a 40-hour workweek over the course of 52 weeks annually, the hourly wage for this occupation is approximately $16.35.

A.13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

Provide estimates of the total annual cost burden to respondents or record keepers resulting from the collection of information, (do not include the cost of any hour burden shown in items 12 and 14). The cost estimates should be split into two components: a) a total capital and start-up cost component annualized over its expected useful life, and b) a total operation and maintenance and purchase of services component.

There is no overall annual cost burden to respondents or record-keepers that results from the NRT program monitoring other than the time spent responding to the online survey. It is usual, customary, and expected that individuals involved in education and training activities in the United States keep descriptive records, and the information being requested is from records that are maintained as part of normal educational or training practices. In order to receive funding, institutions must follow the instructions in the NSF Proposal and Award Policies and Procedures Guide (PAPPG) that is cleared under OMB 3145-0058. The PAPPG requires that all applicants submit requests for NSF funding and that all active NSF awardees do administrative reporting via FastLane or Research.gov. Thus, the PIs, PCs, faculty members, and trainees, who are the primary respondents to the individual data collections within the NRT program monitoring clearance, make use of standard office equipment (e.g., computers), Internet connectivity that is already required as a startup cost and maintenance cost under OMB 3145-0058, and free software (e.g., Google Chrome, Microsoft Edge or other web browsers) to respond.

A.14. Annualized Cost to Federal Government

Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information.

The total annual cost to the Federal Government of the NRT data collection is currently estimated to be $979,291. Computing the annualized cost to NSF for the NRT data collection was done by calculating the actual percent of the budget (93%) in the most recent contract year that was dedicated to operation of the system and estimating that the percent of the budget will be the same for the next three years. The total three-year cost is estimated to be $2,937,875.

The annualized cost was computed as one-third of the total three-year cost; thus, the annualized cost to NSF for the NRT data collection is $979,291.

A.15. Explanation for Program Changes or Adjustments

Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-1.

The increase in burden hours reflects an increase in the number of respondents to the survey. In the initial request, the program estimated ~102 awards per year and this has been revised to better reflect the current active NRT award volume of ~120 awards per year. The annualized cost to respondents has also increased, due to wage increases and increased respondents. Similarly, the annualized costs to the Federal Government have also changed commensurate with the higher respondent volume.

A.16. Plans for Tabulation and Publication and Project Time Schedule

For collections of information whose results are planned to be published, outline plans for tabulation and publication.

Typically, recurring studies generate information that NSF uses as inputs to other reports, and therefore NRT cites no specific publication plans other than internal or general use to meet reporting requirements.

The data collection that is the subject of this request will be utilized for multiple purposes that are described in sections A.1 (Circumstances Requiring the Collection of Data) and A.2 (Purposes and Use of the Data). The data that the NRT monitoring system collects are not published as a stand-alone product. The data are an input to how NSF manages, documents, evaluates, and measures its performance as an agency. Aggregate reports on the NRT participants may be shared with division/directorate/NSF leadership and with the NRT Coordinating Committee (NCC), which includes representative program officers from the other seven NSF Directorates. Aggregate reports may also be shared publicly, on NSF websites, or in presentations. When these data are shared outside NSF, NSF staff will comply with NSF internal guidance (Staff Memorandum OD 18-10) on sharing of non-public information. For data included as part of an evaluation, the evaluation activities will be aligned with the NSF Evaluation Policy¹⁵.

The data collection effort included under this request is administered by a third-party contractor that will deliver analytical reports and the raw data from the collection. NSF holds all vetting rights, and third parties are contractually forbidden from publishing results unless NSF has made a specific exception. All products of the collection are the property of NSF; after the products are delivered, NSF determines whether the quality of the products deserves publication.

When reports on studies that employ monitoring data or documents presenting analyses of monitoring data are approved for publication, distribution is likely to be electronic in nature.

A.17. Reason(s) Display of OMB Expiration Date is Inappropriate

If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The agency plans to display the expiration date of OMB approval on all forms/questionnaires associated with this information collection.

A.18. Exceptions to Certification for Paperwork Reduction Act Submissions

Explain each exception to the certification statement identified in Item 19 “Certification for Paperwork Reduction Act.”

There are no exceptions to the certification statement.

1 The GPRA Modernization Act of 2010 may be found at: https://www.congress.gov/111/plaws/publ352/PLAW-111publ352.pdf

2 The Foundations for Evidence-Based Policymaking Act of 2018 may be found at: https://www.congress.gov/bill/115th-congress/house-bill/4174/text

3 The NSF’s 2022-2026 Strategic Plan may be found at: https://www.nsf.gov/pubs/2022/nsf22068/nsf22068.pdf

4 The IGERT data collection was part of a larger multiple program data collection under OMB Control No. 3145-0226.

5 At the time of submission to the Federal Register and OMB, the current NRT solicitation and notice of reporting requirements may be found at: https://www.nsf.gov/pubs/2021/nsf21536/nsf21536.htm#reportreq. However, the program anticipates producing an updated solicitation within the next fiscal year. The most current NRT solicitation may be found at: https://new.nsf.gov/funding/opportunities/national-science-foundation-research-traineeship.

6 As described in NSF 21-536, “The goals of the program are to: Catalyze and advance cutting-edge interdisciplinary or convergent research in high priority areas; Increase the capacity of U.S. graduate programs to produce diverse cohorts of interdisciplinary STEM professionals with technical and transferable professional skills for a range of research and research-related careers within and outside academia; and Develop innovative approaches and knowledge that will promote transformative improvements in graduate education.”

7 For general information on NSF performance assessment activities see https://www.nsf.gov/about/performance/.

8 See “Division of Graduate Education 2023 – 2028 Strategic Plan,” at https://www.nsf.gov/edu/dge/2023%20DGE%20Strategic%20Plan.pdf

9 See “About STEM Education (EDU),” at https://www.nsf.gov/edu/about.jsp.

10 The inventory of Federal STEM education programs is included as part of the annual Progress Report on the Implementation of the Federal STEM Education Strategic Plan. The 2022 report may be viewed at: https://www.whitehouse.gov/wp-content/uploads/2023/02/Final_2022_CoSTEM_Progress_Report.pdf. Data for NRT participants was reported beginning with the 2023 inventory, which is not yet published.

11 See https://new.nsf.gov/policies/pappg for more information on NSF’s proposal and award policies and procedures guide (PAPPG)

12 Source: https://support.orcid.org/hc/en-us

13 Source of Hourly Wage https://www.bls.gov/ooh/education-training-and-library/postsecondary-teachers.htm.

14 Source of Hourly Wage https://www.nsf.gov/pubs/2021/nsf21536/nsf21536.htm

15 The NSF’s Evaluation Policy was published in April 2023 and is available at https://new.nsf.gov/od/oia/eac/evaluation-policy-april-2023