3145-0238 Supporting Statement Part A

SUPPORTING STATEMENT FOR PAPERWORK REDUCTION SUBMISSION
ENGINEERING INDUSTRIAL INNOVATIONS AND PARTNERSHIPS (IIP) PROGRAM
MONITORING DATA COLLECTIONS
SUPPLEMENTARY INFORMATION
TITLE OF COLLECTION: Program Monitoring Data Collections for Industrial
Innovation and Partnerships Division
Type of Request: Intent to seek approval to renew an information collection.
Section A. Justification
This request is for approval of a group of information collections intended to monitor
outputs, and short-term, intermediate and long-term outcomes of NSF-Directorate of
Engineering investments in research and innovation in the Division of Industrial Innovation
and Partnerships (IIP).
IIP serves the entire Foundation by fostering partnerships to advance technological
innovation and plays an important role in the public-private partnerships by investing in
science and engineering research across all disciplines that have the potential for high
impact in meeting national and societal needs. IIP focuses on leveraging federal, small
business, industrial, university, state and community college resources. Genuine
partnerships between academe and industry are an important aspect of IIP programs and
should facilitate the types of infrastructure that can sustain and nurture the spread of
innovative activity.
IIP is home to the two congressionally mandated small business research programs, the
Small Business Innovation Research (SBIR) program and the Small Business Technology
Transfer program. IIP also manages the Partnerships for Innovation: Accelerating
Innovation Research (PFI:AIR) as well as the Partnerships for Innovation: Building
Innovation Capacity (PFI:BIC) program, which stimulate innovation by building
partnerships across the scientific, engineering, and business community. In addition, the IIP
leverages industrial support through the Industry-University Cooperative Research Centers
(I/UCRC) program. IIP also manages another NSF-wide program, Innovation Corps program
(I-Corps), which equips scientists with the entrepreneurial tools needed to transform
discoveries with commercial realization potential into innovative technologies.
These survey questionnaires, individually tailored for different programs, will provide
essential information for program monitoring. Data collected by ENG IIP program
monitoring collections will be used for program planning, management, and evaluation.
Summaries of monitoring data are used to respond to queries from Congress, the public,
NSF's external merit reviewers who serve as advisors, including Committees of Visitors
(COVs), and NSF's Office of the Inspector General. These data are needed for effective
administration, program and project monitoring, evaluation, and for measuring attainment
of NSF's program and strategic goals, as identified by the President's Accountable
1

Government Initiative, the Government Performance and Results Act (GPRA) Modernization
Act of 2010, and NSF's Strategic Plan.
The eight (8) program-specific collections included in this request are designed to assist in
management of specific programs and to serve as data resources for current and future
program evaluations. As such, expected outcomes could vary according to the nature of the
program funding, field of study, and other program characteristics.

Office

Programs
Innovation Corps (I-Corps)
Partnerships For Innovation: Accelerating
Innovation Research (PFI:AIR)
Partnerships For Innovation: building
Innovation Capacity (PFI:BIC)

Industrial Innovation and Partnerships
(IIP)

Small Business Innovation Research
(SBIR)/Small Business Technology
Transfer (STTR)
Grant Opportunities for Academic Liaison
with Industry (GOALI)

A.1 Circumstances Requiring the Collection of Data
The NSF Division of Industrial Innovation and Partnerships is responsible for analyzing and
evaluating program research and development activities in IIP’s funded portfolios.
Precedent for the ENG IIP Monitoring Systems Clearance
Data from NSF administrative databases can be incorporated with information gathered
through initiative-specific, division-specific, and program-specific data collections. NSF-IIP
uses these data for monitoring, managing, and evaluating its investment in programs,
initiatives, and activities.
Attached is a crosswalk that details the topics each of the questions address and how they
vary between collections. These questions are based on the current understanding of
data management needs and potential use in evaluation activities, including the
development of metrics associated with several program-level logic models.
2

Circumstances of data collection
To fulfill its planning and management responsibilities, and to answer queries from
Congress, OMB, and NSF management, IIP needs current and standardized information
about the short-and long-term outcomes of projects in the IIP portfolio. This information is
specifically important to support studies and evaluations by IIP, and collection of these data
serves several purposes, one of which is to:
•

Provide a source of information on the outcomes of the research investments in terms
of advancements in science, benefits to the institutions, researchers, students and
society and educational and/or career impact on participants/students in NSFfunded projects, in compliance with Foundation responsibilities to monitor scientific
and technical resources enabling NSF to monitor the effectiveness of NSF-sponsored
projects and identify outputs of projects funded under NSF awards for management
and for reporting to the Administration and Congress.

The collections have been assembled for these IIP programs based on their logic models,
theory of change or management needs. The data collected under these surveys is focused
on initiative-specific, division-specific, and program-specific quantitative and qualitative
indicators. The question items elicit participants’ details and activities, outputs (i.e., the
accomplishments of program grantees (projects) in terms of specific program objectives),
outcomes, and impacts (longitudinal (post-award) accomplishments of program grantees).
A.2 Purposes and Use of the Data
The information collected under this request is required for effective program
administration, program and project monitoring, evaluation, and for measuring attainment
of NSF’s program and strategic goals as laid out in NSF’s Strategic Plan. This section describes
how the data to be collected will be used for internal program management and
administration; as a data source for IIP’s performance assessment activities, including
Committees of Visitors and Directorate and Office Advisory Committees (ACs); and as a
foundation for the rigorous research required to evaluate the effectiveness of IIP programs.
The primary objective of this clearance is to allow IIP programs to collect outcome and
output data from grantees, their partners and students, which will enable the evaluation of
the impact of its investments in engineering research over time.
A.3 Use of Information Technology to Reduce Burden
All of the collections included under this clearance request use web-based data collection
systems to minimize data duplication and respondent burden. In some cases, program
officers may call the respondent for follow-up or clarification, and enter data gathered in
these conversations into the web-based data collection system.

3

A.4 Efforts to Identify Duplication
The ENG IIP Program Monitoring Clearance does not duplicate efforts undertaken by the
Foundation, other federal agencies, or other data collection agents. Please note, while there
is public data on award information, they are usually a few years out of date.
A.5 Small Business
In the ENG IIP Program Monitoring Clearance, the IIP Division will collect information from
small businesses. These businesses are partners to current and/or former awardees or are
run/owned by current and/or former awardees. The only impact of this data collection on
the business will be the time required for respondents to complete the survey or have a
conversation with data collectors.
A.6 Consequences of Not Collecting the Information
Data collected for the IIP Program Monitoring Clearance will be used to manage programs,
monitor projects, inform project and program evaluations, coordinate with federal and nonfederal partners, provide Congress with information about government-supported activities,
and report for GPRA and other requirements.
If additional information were not collected, IIP would be unable to document outputs and
outcomes of its programs that occur many years after the award is made (given the nature
of scientific discovery). It would be unable to meet its accountability requirements or assess
the degree to which projects and programs are meeting their goals over time.
A.7 Special Circumstances Justifying Inconsistencies with Guidelines in 5 CFR 1320.6
All data collections will comply with 5 CFR 1320.6. All collections under the ENG IIP Program
Monitoring Clearance could ask respondents for data annually, or occasionally at shorter
intervals for some data collected during the life of the award, or at longer intervals for postaward monitoring. Owing to many of the potential outcomes and impacts of investments in
engineering research are realized years after the award is made (particularly for
fundamental research investments), it is necessary to capture some of these outcomes and
impacts via post-award monitoring. Post-award data collections are voluntary. Analysis
of non-response bias will be implemented every time inferences about a program are to be
drawn from the data. Post-award monitoring systems, including the frequency of postaward data collection are tailored to the expected timeline of outcomes and impacts from
each individual program. In most cases, we expect to collect post-award data at 1-year, 3year, and 5-year intervals, with some post-award systems capturing a 4th data point at 10
years post-award. These collections for programs in the IIP division, expect that important
indicators for outcomes and impacts may become apparent at shorter intervals post-award,
and in these cases, data collection intervals could be more frequent, but will extend over a
shorter total period post-award.

4

A.8 Federal Register Notice Consultation Outside the Agency
The agency’s notice, as required by 5 CFR 1320.8(d), was published in the Federal Register
on February 14, 2018, at 83 FR 6616, and no comments were received.
Consultations have included knowledgeable outsiders such as representatives of IIP
contractors, like VentureWell, responsible for technical and evaluation tasks and fellows
who work at the Foundation as guests under programs such as the American Association for
the Advancement of Science - Science & Technology Policy Fellows Program.
A.9. Payments or Gifts to Respondents
N/A
A.10. Assurance of Confidentiality
Respondents will be informed that any information on specific individuals is maintained in
accordance with the Privacy Act of 1974. Every data collection instrument will display both
OMB and Privacy Act notices.
Respondents will be told that data collected are available to NSF officials and staff, evaluation
contractors, and the contractors hired to manage the data and data collection software. Data
will be processed according to federal and state privacy statutes. The system will limit access
to personally identifiable information to authorized users. Data submitted will be used in
accordance with criteria established by NSF for monitoring research and education grants
and in response to Public Law 99-383 and 42 USC 1885c.
The information requested through NSF monitoring systems may be disclosed to qualified
researchers and contractors for evaluation purposes and to a federal agency, court, or party
in court or federal administrative proceedings, if the government is a party.
A.11 Questions of a Sensitive Nature
Some of the proposed question items in the IIP Program Monitoring Clearance request
information from respondents, including name, contact information, and demographic data
(see table below). These data are collected to monitor the award sites and evaluate the
success of the award programs. Information of this nature is also used to track recipients of
funding and training. Responses to all items of a sensitive nature are voluntary. Respondents
may choose not to provide information that they deem as privileged. Any individual-level
data that are collected will be provided only to program staff and consultants conducting
studies using the data as authorized by NSF. Any public reporting of data will be in aggregate
form, and all personal identifiers will be removed.
Post-award monitoring data collections are voluntary and respondents will clearly be given
the choice of non-response. The table below shows which questions of a sensitive nature
are included in the IIP collections.
5

Table 1. Questions of a Sensitive Nature
Collection Title

Name

Contact
Information

Grant Opportunities for Academic
Liaison with Industry (GOALI)

X

X

Innovation Corps (I-Corps)

X

X

Innovation Corps (I-Corps) PreCourse Survey

X

X

Innovation Corps (I-Corps) PostCourse Survey

X

X

X

X

X

X

X

X

X

X

Partnerships For Innovation:
Accelerating Innovation Research
(PFI:AIR)
Partnerships For Innovation:
building Innovation Capacity
(PFI:BIC)
Small Business Innovation Research
(SBIR)/Small Business Technology
Transfer (STTR)
SBIR Baseline Monitoring Survey

Demographic

X

A.12 Estimates of Response Burden
A.12.1. Number of Respondents, Frequency of Response, and Annual Hour Burden
While output and short-term outcome monitoring can happen yearly, long-term outcomes
will only be monitored at longer intervals (potentially 1, 3, 5, and 10 years after the end of
the award). The table below shows the ideal scenario for a hypothetical year, in which all
respondents in all the collections participate in the collection.
For IIP, many awards are made in translational research, such that we might expect a shorter
and more condensed timeline of outcomes and impacts. In these cases, it is necessary to
collect very limited monitoring data from awardees at short intervals (e.g. annually) during
the life of the award. The data collected could serve two purposes: 1) to provide formative
feedback to awardees on possible mid-course corrections as challenges arise to assist them
in optimizing the impact of their NSF funding, and 2) to provide trend data to NSF program
6

staff such that they might be able to identify patterns that indicate or suggest characteristics
of successful awardees. Identification of trends and patterns can allow NSF to eventually
provide training or resources to future awardees to enhance these characteristics of success.
If data collections are deemed to occur more frequently, the burden to awardees will be
limited to no more than 20 minutes of the respondents’ time in each instance.
Table 2. Respondents, Responses, and Annual Hour Burden
Collection Title

No. of
Annual No. of
Annual Hour
Respondents Hours/Respondent Burden

Grant Opportunities for Academic
Liaison with Industry (GOALI)

200

1

200

Innovation Corps (I-Corps)
Longitudinal Collection

700

1

700

Innovation Corps (I-Corps) PreCourse Survey Questionnaire

800

0.5

400

Innovation Corps (I-Corps) PostCourse Survey Questionnaire

800

0.5

400

Partnerships For Innovation:
Accelerating Innovation Research
(PFI:AIR)

200

1

200

Partnerships For Innovation:
building Innovation Capacity
(PFI:BIC)

30

1

30

Small Business Innovation
Research (SBIR)/Small Business
Technology Transfer (STTR)

900

1.5

1,350

SBIR Baseline Monitoring Survey

800

2

1,600

Total

4,430

8.5

4,880

As shown in Table 3 below, the annual response burden for the collections under this request
is 4,880 hours. This collection will happen at 1, 3, and 5, and 10 years after the award so the
total number of hours is 4,880 x 4 = 19,520 hours/10-year period, an average of 1,952 hours
per year.

7

For life-of-award monitoring, the data collection burden to awardees will be limited to no
more than 120 minutes of the respondents’ time in each instance, but will most likely average
60 minutes of the respondents’ time in each instance.
The respondents are PIs, partners or students. For some programs, like I-Corps, the burden
already includes a response from 3 members of the team in the pre-and post-course surveys.
For all others, one PI or assignee per award completes the questionnaire.
A.12.2. Estimates of Annualized Cost to Respondents for the Hour Burdens
The following table shows the annualized estimate of costs to PI/program coordinator
respondents, who are generally university professors. This estimated hourly rate is based on
a report from the American Association of University Professors, “Annual Report on the
Economic Status of the Profession, 2017-18,” Academe, March–April 2018, Survey Report
Table 1. According to this report1, the average salary of an associate professor across all
types of doctoral-granting institutions (public, private-independent, religiously affiliated)
was $99,820. When divided by the number of standard annual work hours (2,080), this
calculates to approximately $48 per hour.
When at full implementation, in a year in which all programs monitor their investments and
all respondents participate in the collection, the overall annualized cost to the respondents
is estimated to be $234,240. Since these data collections will not take place every year—
rather, on average, every 2.5 years, annual costs to all the respondents across a ten-year
period is average to be $93,696.
Table 3. Annuitized Cost to Respondent
Respondent
Type

No. of
Respondents

Total
Burden
Hours

Average
Hourly Rate

Estimated
Annual Cost

PIs, Assignees,
Partners or
Students (IIP
Division)

4,430

4,880

$48

$234,240

A.13 Estimate of Total Capital and Startup Costs/Operation and Maintenance Costs to
Respondents or Record Keepers
Not applicable
A.14 Estimates of Costs to the Federal Government
Estimated costs include ~$500,000 to develop one monitoring system for each of the 6
divisions and offices, ~$300,000/year to maintain each of these systems by a contractor, and
1

https://www.aaup.org/file/ARES_2017-18.pdf
8

30% of full-time salary for each of two (2) Evaluation & Assessment staff who are salaried at
~$100,000/year.
A.15. Changes in Burden
The changes in burden are due to revisions to the various collections, which either resulted in less
burden time or additional respondents. One new collection was added to the overall request.
A.16. Plans for Publication, Analysis, and Schedule
At this time, IIP plans to produce a summary or descriptive report every year after
completion of the data collections.
A.17. Approval to Not Display Expiration Date
Not applicable
A.18 Exceptions to Item 19 of OMB Form 83-I
No exceptions apply.

9