Supporting Statement for 3145-0233-2020

Supporting Statement for Paperwork Reduction Submission

Grantee Reporting Request for Emerging Frontiers in Research and Innovation (EFRI) program

A. Justification

A.1. Circumstances Requiring the Collection of Data

This request for OMB approval is in regard to building on a pilot project which tested the design and implementation of the grantee reporting requirements developed by the Offices of Emerging Frontiers in Research and Innovation (EFRI) and Program Evaluation in the Office of the Assistant Director (OAD) in the Directorate for Engineering (ENG) at the National Science Foundation (NSF). The pilot study evaluated a questionnaire for its effectiveness in capturing the data detailed in the EFRI logic model while streamlining the collection process and minimizing the burden to the Principal Investigator (PI), including eliminating any redundancies and overlap with indicators already collected in the NSF’s standard, annual reporting mechanism (the Research Performance Progress Report (RPPR)).

The development of an EFRI-specific reporting system stemmed from the desire to obtain, on a voluntary basis, the best data available on the outcomes of the funded research. EFRI is a unique program at NSF that was motivated by the vision of the Directorate for Engineering to be the global leader in advancing the frontiers of fundamental engineering research. EFRI serves a critical role in focusing research on important emerging areas in a timely manner. EFRI awards receive support of up to a total of $2,000,000 spread over four years. The EFRI program has made approximately 200 EFRI awards since 2007. Because of the unique and innovative nature of the EFRI awards, we are seeking to collect additional longitudinal information from the awardees about their research that allows us to capture the unique goals and purpose of the program. This is very important to enable appropriate and accurate evaluation of the program and to determine whether the specific goals of the program are being met.

The indicators provided in the following table are both quantitative and descriptive. The questions have been designed applying sophisticated question-response techniques, such as prompted selections to minimize efforts on behalf of the respondents. Therefore, please reference the questionnaire, in the attached PDF, which is a mock-up demo that displays the complete question-answer design that demonstrates the full scope of each question. Other indicators that are also identified as important measures for the EFRI program, but could be collected through other means, without asking the PIs, are not included in this list because they will not be part of the reporting system.

Following on obtaining OMB approval to pilot an EFRI reporting system, a pilot study was conducted which demonstrated feasibility of this reporting system. The current request is to deploy this reporting system to collect information from EFRI grantees during their award and up to five years post award expiration. The results from the completed pilot project will be used to improve the design and content of the EFRI reporting requirements instrument so that it minimizes the burden to the respondents while ensuring that valuable information is collected.

Reporting System Questions Only

Questions	Indicator
PROJECT OUTPUTS/ KNOWLEDGE TRANSFER ACROSS DISCIPLINES
Please list all collaborators/partners that you consider to be interdisciplinary	Number of Interdisciplinary collaborations (or percentage of grants with this characteristic)
Number of accepted publications with co-authors from different disciplines that have resulted from this award	Number of papers with co-authors from different disciplines
Please list all achievements of any of the PIs and Co-PIs involved in the EFRI research in the appropriate category	List of PI achievements (awards, leadership, promotions, etc.)
INNOVATION OF IDEAS IN AREAS OF GREATER OPPORTUNITY
Has this research generated additional funding from other sources (other NSF programs or other agencies) for all PIs and Co-PIs involved in this EFRI project research?	Number of follow-on grants supported by other agencies (or percentage of grants with this characteristic) after an EFRI award
Please list any researchers who have spent more than 10% of their time in a laboratory at a different institution (including EFRI partner laboratories)	Number of researchers (PIs, post-docs, graduate, undergraduate and other research assistants) exchanged across laboratories (inter-disciplinary, inter-institutional)
Please indicate the nature of the collaboration and whether the partner is international (check all that apply)	Number of grants with International collaborations (or percentage of grants with this characteristic)
Please check the Grand Challenges and/or National Needs that this EFRI research initially set out to address as indicated in the grant proposal	National need(s)/ grand challenge(s) addressed by the completed research or research in progress (have there been any changes in the scope of the research?)
Does the research project currently address the same set of Grand Challenges and/or National Needs that the research initially set out to address?
If not, please indicate below the set of Grand Challenges and/or National Needs the research currently addresses.
POTENTIAL FOR TRANSLATIONAL RESEARCH
Has this research generated additional funding from other sources (other NSF programs or other agencies) for all PI’s and Co-PI’s involved in this EFRI project research?	Number of PIs with additional continuation funding at larger scale from other agencies (or percentage of grants with this characteristic)
Please list any companies or start-ups that were developed directly or indirectly based on the research funded by EFRI	Number of start-up companies whose formation an EFRI award contributed (# of jobs, including founder)
Please provide the following information about any licensing activity that has resulted from your EFRI research (Technology Being Licensed, Type of License, Year of Licensing Agreement.)	Licensing activity
Please list any companies that have demonstrated interest in partnership activities related to your EFRI research. Please gather this information from all leadership in the EFRI grant.	Number of companies interested in partnering with EFRI awardees
Please list new research capabilities resulting from EFRI research that are being utilized and adopted by the research community and their appropriate categories	Number of new techniques, tools, resources, equipment, methods of research, models, etc.
INNOVATIVE RESEARCH METHODS OR DISCOVERIES ARE INTRODUCED TO THE CLASSROOM
Please indicate if this EFRI research has led to curriculum changes at any educational level, collegiate or pre-collegiate	Number of topics that have generated curriculum changes or inclusions of modules to teach methods, discoveries or innovations funded by EFRI at college level and at PK-12 levels
PROJECT RESULTS ADVANCE THE FRONTIER / CREATION OF NEW FIELDS OF STUDY
Please provide the following information about former EFRI-supported students who have become involved in high-risk/ high-reward research (Name, Current Title, Field of Research.)	Number of EFRI students who become involved in high risk research in their portfolio after graduation
Please list any activities of new research communities that have formed around EFRI topics	Creation of communication medium on new topic, i.e., new conferences, new journals, working groups, congressional hearings, etc.
FOSTERING PARTICIPATION OF UNDEREPRESENTED GROUPS IN SCIENCE
Self-disclosed response from email solicitation to students	Number of participants and research assistants from underrepresented groups in research activities

Survey questions emailed to students

Demographic Information asked in same format as has already been approved through NSF’s Graduate Research Fellowships Program (GRFP) including questions on gender, veteran status, race, ethnicity, and disability.	Percentage of participants from underrepresented groups who stated their EFRI experience (lab and interactions with research group) influenced positively their decisions and career path
Please list any significant achievements that resulted from your EFRI research involvement.
Please list any outreach activities you participated in for work related to your EFRI project
Is there any other information you would like to share with us regarding your experience with this EFRI project?
Please consider how strongly you agree or disagree with the following statement: When I began my involvement with this EFRI research, I had a clear career goal
Please rate your interest in pursuing each of the following career pathways at the beginning and at the conclusion of your EFRI research experience (Faculty at a research-intensive university, Faculty at a teaching-intensive university, Research career, non-academic (industry, pharmaceutical, biotech, government, start-up, etc.), Non-research career (consulting, policy, science writing, patent law, business, etc.))

A.2. Purpose and Use of Data

In this project, grantees from EFRI awards will be asked to provide data to NSF and its authorized representatives (contractors or grantees). The data collected will be used for NSF internal reports, historical data, assessing program impact and recommending changes to strengthen the program to ensure the program remains responsive to a changing environment in order to secure future funding for continued EFRI program maintenance and growth.

A.3. Use of Automation

The questionnaire has been tested for the most effective method of data collection. Online collection tools will be utilized as mechanisms for data collection.

A.4. Efforts to Identify Duplication

No other federal agencies or organization within NSF collects data pertaining to the EFRI program. Care was taken in the development of the questionnaire to identify and eliminate indicators already collected in NSF’s standard, annual reporting mechanism, the (the Research Performance Progress Report (RPPR)).

A.5. Small Business Consideration

N/A

A. 6. Consequences of Less Frequent Collection

The optimal collection frequency will be tested during deployment of the questionnaire.

A.7. Special Circumstances for Collection

N/A

A. 8. Federal Register Notice and Outside Consultation

The agency’s notice relating to the intent to renew the data collection was published in the Federal Register on February 20, 2020 at 85 FR 9811, and no comments were received.

A. 9. Gifts or Remuneration

N/A

A.10. Assurance of Confidentiality

All data collected is intended for internal use only. Any information that will be exposed externally will be presented in aggregate, and no names of individuals will be included. In the event that an individual’s particular achievements, reported as a part of the questionnaire, are chosen to be highlighted and presented outside of the NSF, the Engineering directorate will seek the permission and approval from both the individual in question and OMB.

A 11. Questions of a Sensitive Nature

No questions of any sensitive nature are asked.

A. 12. Estimate of Burden

This request for the project to deploy the questionnaire pertains to 100 EFRI awards and is expected to occur annually or at a less regular frequency. Therefore, the maximum number of reports will be 100 annually. Based on data from the pilot study, we estimate the burden of collecting the data necessary to answer the questionnaire, in terms of man-hours per award, as follows:

1. Lead PI – 1 hour

2. Co-PI’s – 20 mins per co-PI

3. Students graduate/undergraduate (material collection) – 30 minutes per student

Total hours per award for this confined pilot project are estimated to be 7 hours.

Total number of hours for 100 awards annually: approximately 700 hours.

A. 13. ANNUALIZED COST TO RESPONDENTS

Estimated cost per award, is as follows:

Expense category	Unit cost	Units	Total value of time
1. Lead PI	$100/hour	1 hour	$100
2. Co-PIs	$100/hour	20 mins [*3 (average number of Co-PIs on EFRI awards)]	$100
3. Students graduate/undergraduate	$16/hour	30 minutes [*10(average number of students working on EFRI awards)]	$80
4. Fringe benefits (30%) on items 1-2 (based on averages)			$60
Total annual cost per award			$340
Total cost for 100 awards			$34,000

A. 13. Annual cost burden [not included in hour cost]

There are no additional costs beyond the estimated hours of burden shown above.

A. 14. Annualized Cost to the Federal Government (Payment to the Evaluators)

The reporting project will be outsourced to a contractor. The cost of the contract is expected to be roughly $150,000.

A. 15. Changes in Burden

This collection builds on a pilot project previously approved by OMB, and now involves all recipients rather than a selection.

A. 16 Publication of Collection

N/A

A. 17 Approval to Not Display OMB Expiration Date

N/A

A. 18 Exception to Item 19 of OMB Form 83-I Certification Statement

N/A

B. STATISTICAL METHODS

Not applicable

Attachments:

Attachment I: EFRI monitoring system (in PDF form).

Attachment 2: EFRI Program solicitation (NSF Number 19-599)

5