3145-0233 Supporting Statement for EFRI 2017

Supporting Statement for Paperwork Reduction Submission

Grantee Reporting Request for Emerging Frontiers in Research and Innovation (EFRI) program (OMB Clearance Number 3145-0233)

A. Justification

A.1. Circumstances Requiring the Collection of Data

This request for OMB approval is in regard to the implementation of a grantee reporting questionnaire that was piloted tested by the Emerging Frontiers in Research and Innovation (EFRI) and Program Evaluation Offices in the Offices of the Assistant Director (OAD) in the Directorate of Engineering (ENG) at the National Science Foundation (NSF). The questionnaire was evaluated for its effectiveness in capturing the data detailed in the EFRI logic model while streamlining the collection process and minimizing the burden to the Principle Investigator (PI), including eliminating any redundancies and overlap with indicators already collected in the NSF’s standard, annual reporting mechanism (the Research Performance Progress Report (RPPR)).

The development of an EFRI-specific reporting system stems from the following: a desire to obtain, on a voluntary basis, the best data available on the outcomes of the funded research. EFRI is a unique program at NSF that was motivated by the vision of the Directorate for Engineering to be the global leader in advancing the frontiers of fundamental engineering research. EFRI serves a critical role in focusing research on important emerging areas in a timely manner. EFRI awards receive support of up to a total of $2,000,000 spread over four years. The EFRI program has awarded 215 research proposals since 2007. Because of the unique and innovative nature of the EFRI grants, we are seeking to collect additional longitudinal information from the grantees about their research that allows us to capture the unique goals and purpose of the program. This is very important to enable appropriate and accurate evaluation of the program and to determine whether or not the specific goals of the program are being met.

The indicators provided in the following table are both quantitative and descriptive. The following questions have been designed applying sophisticated question-response techniques, such as prompted selections to minimize efforts on behalf of the respondents. Therefore, please reference the questionnaire, attached below the table, which is a mock-up demo that displays the complete question-answer design that demonstrates the full scope of each question. Other indicators that are also identified as important measures for the EFRI program, but could be collected through other means, without asking the PIs, are not included in this list because they will not part of the reporting system

Reporting System Questions Only

Questions	Indicator
PROJECT OUTPUTS/ KNOWLEDGE TRANSFER ACROSS DISCIPLINES
Please list all collaborators/partners that you consider to be interdisciplinary	Number of Interdisciplinary collaborations (or percentage of grants with this characteristic)
Number of accepted publications with co-authors from different disciplines that have resulted from this award	Number of papers with co-authors from different disciplines
Please list all achievements of any of the PIs and Co-PIs involved in the EFRI research in the appropriate category	List of PI achievements (awards, leadership, promotions, etc)
INNOVATION OF IDEAS IN AREAS OF GREATER OPPORTUNITY
Has this research generated additional funding from other sources (other NSF programs or other agencies) for all PIs and Co-PIs involved in this EFRI project research?	Number of follow on grants supported by other agencies (or percentage of grants with this characteristic) after an EFRI award
Please list any researchers who have spent more than 10% of their time in a laboratory at a different institution (including EFRI partner laboratories)	Number of researchers (PIs, post-docs, graduate, undergraduate and other research assistants) exchanged across laboratories (inter-disciplinary, inter-institutional)
Please indicate the nature of the collaboration and whether the partner is international (check all that apply)	Number of grants with International collaborations (or percentage of grants with this characteristic)
Please check the Grand Challenges and/or National Needs that this EFRI research initially set out to address as indicated in the grant proposal	National need(s)/ grand challenge(s) addressed by the completed research or research in progress (have there been any changes in the scope of the research?)
Does the research project currently address the same set of Grand Challenges and/or National Needs that the research initially set out to address?
If not, please indicate below the set of Grand Challenges and/or National Needs the research currently addresses.
POTENTIAL FOR TRANSLATIONAL RESEARCH
Has this research generated additional funding from other sources (other NSF programs or other agencies) for all PI’s and Co-PI’s involved in this EFRI project research?	Number of PIs with additional continuation funding at larger scale from other agencies (or percentage of grants with this characteristic)
Please list any companies or start-ups that were developed directly or indirectly based on the research funded by EFRI	Number of start-up companies whose formation an EFRI award contributed (# of jobs, including founder)
Please provide the following information about any licensing activity that has resulted from your EFRI research (Technology Being Licensed, Type of License, Year of Licensing Agreement.)	Licensing activity
Please list any companies that have demonstrated interest in partnership activities related to your EFRI research. Please gather this information from all leadership in the EFRI grant.	Number of companies interested in partnering with EFRI awardees
Please list new research capabilities resulting from EFRI research that are being utilized and adopted by the research community and their appropriate categories	Number of new techniques, tools, resources, equipment, methods of research, models, etc.
INNOVATIVE RESEARCH METHODS OR DISCOVERIES ARE INTRODUCED TO THE CLASSROOM
Please indicate if this EFRI research has led to curriculum changes at any educational level, collegiate or pre-collegiate	Number of topics that have generated curriculum changes or inclusions of modules to teach methods, discoveries or innovations funded by EFRI at college level and at PK-12 levels
PROJECT RESULTS ADVANCE THE FRONTIER / CREATION OF NEW FIELDS OF STUDY
Please provide the following information about former EFRI-supported students who have become involved in high-risk/ high-reward research (Name, Current Title, Field of Research.)	Number of EFRI students who become involved in high risk research in their portfolio after graduation
Please list any activities of new research communities that have formed around EFRI topics	Creation of communication medium on new topic, i.e., new conferences, new journals, working groups, congressional hearings, etc.
FOSTERING PARTICIPATION OF UNDEREPRESENTED GROUPS IN SCIENCE
Self-disclosed response from email solicitation to students	Number of participants and research assistants from underrepresented groups in research activities

Survey questions emailed to students

Demographic Information asked in same format as has already been approved through NSF’s Graduate Research Fellowships Program (GRFP) including questions on gender, veteran status, race, ethnicity, and disability.	Percentage of participants from underrepresented groups who stated their EFRI experience (lab and interactions with research group) influenced positively their decisions and career path
Please list any significant achievements that resulted from your EFRI research involvement.
Please list any outreach activities you participated in for work related to your EFRI project
Is there any other information you would like to share with us regarding your experience with this EFRI project?
Please consider how strongly you agree or disagree with the following statement: When I began my involvement with this EFRI research, I had a clear career goal
Please rate your interest in pursuing each of the following career pathways at the beginning and at the conclusion of your EFRI research experience (Faculty at a research-intensive university, Faculty at a teaching-intensive university, Research career, non-academic (industry, pharmaceutical, biotech, government, start-up, etc.), Non-research career (consulting, policy, science writing, patent law, business, etc.))

A.2. Purpose and Use of Data

Grantees from closed EFRI awards will be asked to provide data to NSF and its authorized representatives (contractors or grantees). The results of the pilot project were used to improve the design and content of the EFRI reporting requirements instrument minimizing the burden to the respondents while ensuring that valuable information is collected. Data collected will be used for NSF internal reports, historical data, assessing program impact and recommending changes to strengthen the program to ensure the program remains responsive to a changing environment in order to secure future funding for continued EFRI program maintenance and growth.

A.3. Use of Automation

Online collection will be the mechanism for data collection.

A.4. Efforts to Identify Duplication

No other federal agencies or organization within NSF collect data pertaining to the EFRI program. Care was taken in the development of the questionnaire to identify and eliminate indicators already collected in NSF’s standard, annual reporting mechanism, the (the Research Performance Progress Report (RPPR)).

A.5. Small Business Consideration

N/A

A. 6. Consequences of Less Frequent Collection

The optimal collection frequency will be tested in the testing of the questionnaire.

A.7. Special Circumstances for Collection

N/A

A. 8. Federal Register Notice and Outside Consultation

The agency’s notice, as required by 5 CFR 1320.8(d), was published in the Federal Register on December 19, 2016, at 81 FR 91960, and no comments were received.

A. 9. Gifts or Remuneration

N/A

A.10. Assurance of Confidentiality

All data collected is intended for internal use only. Any information that will be exposed externally will be presented in aggregate, and no names of individuals will be included. In the event that an individual’s particular achievements, reported as a part of the questionnaire, are chosen to be highlighted and presented outside of the NSF, the Engineering directorate will seek the permission and approval from both the individual in question and OMB.

A 11. Questions of a Sensitive Nature

No questions of any sensitive nature are asked.

A. 12. Estimate of Burden

This request pertains to cohorts of 20 EFRI awards every 2 years after the end of the award, on years 5, 7 and 9 from the award date . Therefore, the total number of reports could achieve 100 over a 10 year period. We estimate the burden of collecting the data necessary to answer the questionnaire, in terms of man-hours per award, as follows:

1. Lead PI – 6 hours x 3 reports = 18

2. Co-PI’s – 2 hours x 3 reports = 6

3. Students graduate/undergraduate (material collection) – 30 minutes per student x 3 = 1.5 hours

Total hours per award are estimated to be 25.5 hours.

Total number of hours for a cohort of 20 awards over a cycle of 10 years: approximately 510 hours.

A. 13. ANNUALIZED COST TO RESPONDENTS

Estimated cost per award, is as follows:

Expense category	Unit cost	Units	Total value of time
1. Lead PI	$100/hour	6 hours	$600
2. Co-PIs	$100/hour	2 hours[*3 (average number of Co-PIs on EFRI awards)]	$600
3. Students graduate/undergraduate	$16/hour	30 minutes[*10(average number of students working on EFRI awards)]	$80
4. Fringe benefits (30%) on items 1-2 (based on averages)			$360
Total annual cost per award			$1,640
Total cost for 20 awards			$32,800

A. 13. Annual cost burden [not included in hour cost]

There are no additional costs beyond the estimated hours of burden shown above.

A. 14. Annualized Cost to the Federal Government (Payment to the Evaluators)

The development of the monitoring system will be outsourced to a contractor. The cost of the contract is expected to be roughly $300,000.

A. 15. Changes in Burden

This is a follow up data collection after a pilot project.

A. 16 Publication of Collection

N/A

A. 17 Approval to Not Display OMB Expiration Date

N/A

A. 18 Exception to Item 19 of OMB Form 83-I Certification Statement

N/A

B. STATISTICAL METHODS

Not applicable

Attachments

Attachment I: Please see the attached power point slides.

Attachment 2: Program solicitation (NSF Number 16-612)

5